Tonghan Wang

Lu Zhang

Huizhen Jia

Baosheng Li

Huazhong Shu

Multiscale contrast similarity deviation: an effective and efficient index for perceptual image quality assessment

Keywords: contrast similarity, image quality assessment, multiscale, standard deviation pooling, full reference

Perceptual image quality assessment (IQA) uses a computational model to assess the image quality in a fashion consistent with human opinions. A good IQA model should consider both the effectiveness and efficiency. To meet this need, a new model called multiscale contrast similarity deviation (MCSD) is developed in this paper. Contrast is a distinctive visual attribute closely related to the quality of an image. To further explore the contrast features, we resort to the multiscale representation. Although the contrast and the multiscale representation have already been used by other

IQA indices, few have reached the goals of effectiveness and efficiency simultaneously. We compared our method with other state-of-the-art methods using six well-known databases. The experimental results showed that the proposed method yielded the best performance in terms of correlation with human judgments. Furthermore, it is also efficient when compared with other competing IQA models.

Introduction

Image quality assessment occupies a very important position in numerous fields and applications, such as image acquisition, compression, transmission and restoration. Since human beings are the ultimate receivers of the visual stimulus, it is essential to develop a perceptual model to closely correlate with the human visual system (HVS).

Objective quality assessment methods can be classified into three types [START_REF] Wang | Modern Image Quality Assessment[END_REF]: (1) full-reference(FR), where an ideal "reference" image is available for comparison; [START_REF] Wang | Why is image quality assessment so difficult[END_REF] reduced-reference (RR), where partial information about the reference image is available; and (3) no-reference (NR), where the reference image is not accessible. This paper focuses on the FR methods. In the past decades, great efforts and huge advances have been made in FR methods. Here we briefly review some representative ones. More comprehensive surveys on FR-IQA metrics can be found in [START_REF] Zhang | A comprehensive evaluation of full reference image quality assessment algorithms[END_REF] , [START_REF] Lin | Perceptual visual quality metrics: A survey[END_REF] and [START_REF] He | Objective image quality assessment: a survey[END_REF]. The traditional metrics such as the peak signal-to-noise ratio (PSNR) and the mean squared error (MSE) did not correlate well with human opinions [START_REF] Wang | Why is image quality assessment so difficult[END_REF]. The later developed FR methods could be generally classified into three types of approaches: the HVS model based ones, the information theoretic ones and the structural ones.

The noise quality measure index (NQM) [START_REF] Damera-Venkata | Image quality assessment based on a degradation model[END_REF], the visual signal-to-noise ratio index (VSNR) [START_REF] Chandler | VSNR: A wavelet-based visual signal-to-noise ratio for natural images[END_REF] and most apparent distortion (MAD) [START_REF] Larson | Most apparent distortion: full-reference image quality assessment and the role of strategy[END_REF] are the three representatives HVS based FR methods. The NQM and the VSNR quantified the effects of different visual signals (e.g. the luminance, the contrast, the frequency content, the interaction between them) on the HVS. The MAD was proposed by Larson and Chandler based on the hypothesis that the HVS used different strategies for high quality and low quality images. However this kind of FR methods is usually not computationally efficient.

The information theoretic approaches include the visual information fidelity (VIF) [START_REF] Sheikh | Image information and visual quality[END_REF] and the information fidelity criteria (IFC) [START_REF] Sheikh | An information fidelity criterion for image quality assessment using natural scene statistics[END_REF]. The VIF took the FR IQA problem as an information fidelity problem and chose the amount of information shared by the reference image and the distorted one as the similarity, which was an extented version of the IFC.

The structural approaches are based on the assumption that the HVS is highly adapted for extracting structural information from the visual scene. As a milestone in the development of IQA models, the structural similarity (SSIM) [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF] surpassed the previous ones since it showed a better correlation with the human perception. Then a lot of SSIM-based metrics have been proposed in the literature [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF][START_REF] Li | Three-component weighted structural similarity index[END_REF][START_REF] Wang | Information content weighting for perceptual image quality assessment[END_REF]. In [START_REF] Gao | A content-based image quality metric[END_REF], Gao et al. presented a content-based metric (CBM), which divided the structural information into edges, textures and flat regions in accordance with the content and then pooled the three parts with different weights to obtain the final image quality. Based on the observation that the visual information in an image is often redundant and the HVS understands an image mainly based on its low-level features, Zhang et al. proposed the feature-similarity (FSIM) index [START_REF] Zhang | FSIM: A feature similarity index for image quality assessment[END_REF] which employed two features (the phase congruency and the gradient magnitude) to compute the local similarity map. Considering the gradients' sensitivity to structure and contrast changes, Liu et al. proposed GSM [START_REF] Liu | Image quality assessment based on gradient similarity[END_REF] based on a gradient similarity. Unlike the SSIM's average pooling, the CBM, the FSIM and the GSM adopted a weighting strategy for the pooling. Note that the weighting pooling may gain more IQA accuracy than those with average pooling to some degree, but it may increase the computational complexity. In addition, this pooling could make the predicted quality scores nonlinear to human opinions [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF]. Based on these considerations, Xue et al. [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF] proposed a gradient based model, based on the observation that the image gradient can effectively capture image local structures to which the HVS is highly sensitive. The gradient magnitude similarity deviation (GMSD) index [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF] firstly the generated image gradient magnitude maps of the reference image and the distorted one, and then computed the similarity map of them, finally took the standard deviation of similarity map as the overall image quality score.

Generally, the effectiveness, namely, high correlation with the human subjective score, is the prerequisite of a good IQA model. The efficiency (low computation cost), however, is the second most important criterion of a good IQA model. Thus the effectiveness and efficiency are two ultimate goals for the design of IQA models, but unfortunately it is hard to reach these two goals simultaneously. Among the above-mentioned methods, the GMSD [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF] had a big success in terms of the two goals, but its performances were a little bit low for certain distortion types (such as the contrast distortion). In this paper, we attempt to make another effort to fill this need and to overcome the problem of the GMSD by proposing an effective and efficient FR-IQA model called multiscale contrast similarity deviation (MCSD). The MCSD is also a structural approach.

The rest of this paper is organized as follows. In Section 2, we present the proposed model in detail. Section 3 shows and discusses the results. We conclude this work in Section 4.

Multiscale contrast similarity deviation (MCSD)

Contrast is a good attribute for characterizing the quality of an image [START_REF] Watson | Visual detection of spatial contrast patterns: evaluation of five simple models[END_REF]. Proper contrast change may even improve the perceptual quality of images. In fact, we can define "high quality" as appropriate contrast and little distortion. The contrast has been widely used in the area of image enhancement [START_REF] Ferraro | Image contrast enhancement via entropy production[END_REF][START_REF] Delmarco | The design of wavelets for image enhancement and target detection[END_REF][START_REF] Panetta | Parameterized Logarithmic Framework for Image Enhancement Systems[END_REF][START_REF] Panetta | Human visual System Based Image Enhancement and Measure of Image Enhancement[END_REF][START_REF] Agaian | Visual Morphology[END_REF][START_REF] Agaian | Transform Coefficient Histogram-Based Image Enhancement Algorithms Using Contrast Entropy[END_REF]. Contrast has previously been utilized in SSIM [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF], where it was used as one of the three featuresluminance, contrast and structure. Here we use the contrast feature alone to design our IQA model. Furthermore, the contrast is sensitive to the spatiotemporal frequency and viewing distance [START_REF] Haun | Is image quality a function of contrast perception?[END_REF], [START_REF] Watson | DCT basis function visibility: Effects of viewing distance and contrast masking[END_REF], which are related to the multi-scale representation to some extent. In fact, the multi-scale method is a convenient way to incorporate image details at different resolutions. Perceptibility of image details depends on viewing distance and sampling density of an image. Furthermore, a natural image might have objects and structures that are relevant at different scales, but the human eye is readily able to identify and process the information presented by it [START_REF] Lindeberg | Scale-space theory: A basic tool for analysing structures at different scales[END_REF]. Thus, processing an image at various scales adds flexibility to the processing technique, and image scales play a very important role in IQA [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF], [START_REF] Saha | Utilizing image scales towards totally training free blind Image quality assessment[END_REF]. In [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF], five scales, namely, the original image scale plus the other four reduced resolution (each down sampled by a factor of 2), were utilized to design the multi-scale SSIM. Very recently, by using proper image scales, the authors of [START_REF] Saha | Utilizing image scales towards totally training free blind Image quality assessment[END_REF] designed a totally training free NR-IQA. For these reasons, we combined the contrast with the multi-scale representation to design our model.

Structure diagram of the MCSD

The structure diagram of the MCSD is shown in Fig. 1. The design choices (all the parameters' settings) for MCSD will be introduced in section 3.2. The MCSD explores the contrast features by resorting to the multi-scale representation. The reason is that multiscale method incorporates image details at different resolutions, and contrast is relevant to the viewing distance. The contrast combined with multiscale representation were widely used in the literature of image enhancement [START_REF] Lu | Contrast enhancement of medical images using multiscale edge representation[END_REF][START_REF] Boccignone | A multiscale contrast enhancement method[END_REF][START_REF] Nercessian | Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of the human visual system[END_REF][START_REF] Mukhopadhyay | A multiscale morphological approach to local contrast enhancement[END_REF][START_REF] Rahman | A Comparison of the Multiscale Retinex with Other Image Enhancement Techniques[END_REF] and image fusion [START_REF] Toet | Multiscale contrast enhancement with applications to image fusion[END_REF][START_REF] Nercessian | Multiresolution Decomposition Schemes Using the Parameterized Logarithmic Image Processing Model with Application to Image Fusion[END_REF][START_REF] Nercessian | Multiscale image fusion using an adaptive similarity-based sensor weighting scheme and human visual system-inspired contrast measure[END_REF], where the multiscale representation was implemented by pyramid decomposition [START_REF] Nercessian | Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of the human visual system[END_REF], [START_REF] Nercessian | Multiresolution Decomposition Schemes Using the Parameterized Logarithmic Image Processing Model with Application to Image Fusion[END_REF], wavelet decompositions [START_REF] Lu | Contrast enhancement of medical images using multiscale edge representation[END_REF][START_REF] Nercessian | Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of the human visual system[END_REF][START_REF] Nercessian | Multiresolution Decomposition Schemes Using the Parameterized Logarithmic Image Processing Model with Application to Image Fusion[END_REF], scale-space approach [START_REF] Boccignone | A multiscale contrast enhancement method[END_REF] and homomorphic transform [START_REF] Rahman | A Comparison of the Multiscale Retinex with Other Image Enhancement Techniques[END_REF]. In view of efficiency, the multiscale representation in the proposed model, however, was operated by iteratively applying a low-pass filter and downsampling the filtered image by a factor of 2, as did in MS-SSIM [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF].

Considering the computational cost, we do not use the original image scale but three reduced resolutions (each down sampled by a factor of 2). For the reference image and distorted image at each scale, we calculate their contrast similarity deviation (CSD), sort of contrast similarity map between them (the derivation of the CSD will be detailed in the next subsection). Then the CSDs at three scales are pooled to have the final score MCSD using the following equation:

nscales 1 CSD j j j MCSD ( 1
)
where means product, nscales is the total scale we use, j is the jth scale, j is the corresponding weight of the jth scale and nscales 1 1 j j .

Contrast similarity deviation

There are different definitions for contrast [START_REF] Pelli | Measuring contrast sensitivity[END_REF], [START_REF] Peli | Contrast in complex images[END_REF], such as Weber contrast, Michelson contrast, RMS (root-mean-square) contrast, etc. Weber contrast is mainly for characters, Michelson contrast is mainly for depicting the gratings, and RMS contrast for natural stimuli [START_REF] Pelli | Measuring contrast sensitivity[END_REF]. Thus we chose the RMS contrast in this paper, as adopted by the SSIM [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF]. Similar to [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF], we use the standard deviation (the square root of variance) as an estimate of the signal contrast:

1/2 2 1 1 1 ( -) ( -1)( -1) M N ij i j c I I M N (2)
where intensities ij I are the i-th j-th element of the 2D image of size M by N, and I

is the average intensity of all pixel values in the image. Whereas, the contrast is computed locally in the paper and it can be written in the following form. (I )

M N ij ij I i j c w (3)
and

1 1 M N I i ji j i j w I (4)
where ij w is local window, namely, a circular-symmetric Gaussian weighting function and then rescaled to unit volume, I is the local mean of image I. The local window size plays an important role in the performance of the IQA model as it will be shown in section 3.2. Contrast maps are computed locally for reference image and its distorted one using Eq. ( 3). We denote by r CM and d CM the contrast maps for the reference image and the distorted one, respectively. Then contrast maps similarity contrast similarity map( CS , which is also a map) for the two images being compared is defined as: (2 ) = ( . . )

r d CM CM a CS CMr CMr a ( 5
)
where "•" indicates element-wise multiplication of two matrices, " 2 . " indicates element-wise square of the matrix, and a is a positive constant to avoid division by zero. Note that r CM and d CM are computed from a local patch of reference image r and the distorted one d, respectively. For grayscale image, contrast is the difference in luminance that makes an object distinguishable. To reflect the contrast change as accurately as possible, we resort to the standard deviation as the pooling strategy since it can reflect the range of image local contrast degradation. This is in line with the contrast since both of them indicate "change". Thus the contrast similarity deviation (CSD) is defined as follows:

( ( , )-)

M N i j CSD CS i j CSM MN [START_REF] Wang | Information content weighting for perceptual image quality assessment[END_REF] where CSM (contrast similarity mean) is the mean pooling of contrast similarity map:

1 1 1 ( , ) M N i j CSM CS i j MN (7)

Algorithm description of the MCSD

To clearly state the workflow of the MCSD, we give its algorithmic description in Algorithm 1.

Algorithm 3 Performance evaluation

databases and evaluation protocols

To test the performance of the proposed MCSD, we used the six well-known databases: LIVE [START_REF] Sheikh | LIVE image quality assessment database release 2[END_REF], CSIQ [START_REF] Larson | Most apparent distortion: Full-reference image quality assessment and the role of strategy[END_REF], TID2008 [START_REF] Ponomarenko | TID2008-A database for evaluation of full-reference visual quality assessment metrics[END_REF], TID2013 [START_REF] Ponomarenko | Color image database TID2013: Peculiarities and preliminary results[END_REF], MICT [START_REF] Horita | MICT image quality evaluation database[END_REF] and IVC [START_REF] Ninassi | Pseudo no reference image quality metric using perceptual data hiding[END_REF]. The information about these databases is summarized in Table 1.

1) LIVE database II [START_REF] Sheikh | LIVE image quality assessment database release 2[END_REF]: It consists of 29 reference images and their degraded version with five types of distortions, i.e., JPEG 2000 compression (JP2K), JPEG compression (JPEG), additive white Gaussian noise (WN), Gaussian blurring (Gblur), and fast fading (FF). The human subject score (DMOS) associated with distorted images are provided.

2) CSIQ IQA database [START_REF] Larson | Most apparent distortion: Full-reference image quality assessment and the role of strategy[END_REF]: It consists of 30 reference images and their degraded versions with six different types of distortions at four to five different levels: JPEG2000, JPEG, Additive Gaussian white noise (AGWN), Gaussian Blurring (GB), global contrast decrements (CTD), and additive pink Gaussian noise (APGN). DMOS associated with distorted images is provided and is in the range [0, 1], where lower DMOS indicates higher quality.

3) The TID2008 database [START_REF] Ponomarenko | TID2008-A database for evaluation of full-reference visual quality assessment metrics[END_REF] We calculated four commonly used performance indices, i.e. the Spearman's rank ordered correlation coefficient (SROCC) and the Kendall rank-order correlation coefficient (KROCC) which measure the prediction monotonicity; the Pearson's linear correlation coefficient (PLCC) which is related to the prediction linearity (considered as the measure of prediction accuracy); and the root mean square error (RMSE) which evaluates the prediction consistency.

Let the vector s denote subjective scores, the vector x denote the original IQA scores predicted by an objective metric, i d denote the difference between the ranks of each pair in x and s , both s and x have n elements, then: x x or i j s s , then they are neither concordant nor discordant; else they are discordant.

To compute the PLCC and the RMSE, we used a logistic regression function to reduce the nonlinearity of predicted scores [START_REF] Sheikh | A statistical evaluation of recent full reference image quality assessment algorithms[END_REF].

2 3 1 4 5 ( ) 1 1 ( ) 1, 2,..., 2 1 i i i x p x i n e ( 10
)
where i (i=1, 2, …, 5) are parameters to be fitted and p i is the element in the vector p that denotes the IQA score after regression. Then we have:

1 2 2 1 1 ( )( ) ( ) (s )
n i i i n n i i i i p p s s PLCC p p s ( 11
)
where p and s are the mean of p and s , respectively.

n i i i RMSE s p n ( 12
)
A value being close to 1 for SROCC, KROCC and PLCC indicates a good performance for quality prediction. Whereas, for RMSE, the smaller the value is, the better prediction consistency it yields.

Implementation of MCSD

Note that several parameters are involved in MCSD design. We thus tested their impact on the overall performance of the proposed model in this section. We draw the plot of the SROCC vs. the local window size and the SROCC vs. the constant a in Eq. (5) in Fig. 2 and Fig. 3, respectively. We investigated their impacts on the performance on CSIQ database. We can see from Fig. 2 that the performance is on a downward trend when the local window size increased. Thus we set it to 2. In other words, we use a 2×2 circular-symmetric Gaussian weighting function with its standard deviation of 1.5 and then rescaled to unit volume. We investigated the constant a from 1 to 130 with a step of 10. From Fig. 3 we can see that the performance reaches its maximum value at the point of 40 on CSIQ database. In our experiment, we set it to 45 considering its overall performance on all the benchmark databases. We set nscales=3, 1 0.65 , 2 0.1 and 3 0.25 based on experiments conducted in the same way as the above procedure.

To further investigate the influences of different pooling strategies on the performance, the simple average (MEAN) pooling strategy is compared with the standard deviation pooling strategy. We conducted an experiment on three six large databases-LIVE [START_REF] Sheikh | LIVE image quality assessment database release 2[END_REF], CSIQ [START_REF] Larson | Most apparent distortion: Full-reference image quality assessment and the role of strategy[END_REF], TID2008 [START_REF] Ponomarenko | TID2008-A database for evaluation of full-reference visual quality assessment metrics[END_REF], TID2013 [START_REF] Ponomarenko | Color image database TID2013: Peculiarities and preliminary results[END_REF], MICT [START_REF] Horita | MICT image quality evaluation database[END_REF] and IVC [START_REF] Ninassi | Pseudo no reference image quality metric using perceptual data hiding[END_REF]. The Spearman's rank ordered correlation coefficient (SROCC), defined in section 3, is used as the evaluation criterion here. The result is shown in Fig. 4, from which we can see that the standard deviation pooling for our model is better than the mean pooling on all of the three six databases.

Performance comparisons

We compared the proposed method MCSD to seven state-of-the-art and representative FR-IQA models, including SSIM [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF], MS-SSIM [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF] IW-SSIM [START_REF] Wang | Information content weighting for perceptual image quality assessment[END_REF] VIF [START_REF] Sheikh | Image information and visual quality[END_REF], MAD [START_REF] Larson | Most apparent distortion: full-reference image quality assessment and the role of strategy[END_REF], FSIM [START_REF] Zhang | FSIM: A feature similarity index for image quality assessment[END_REF], GMSD [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF]. All the IQA models were designed for grayscale image except the FSIM which could handle both color and grayscale image. Note that GMSD is the recently proposed top-performance IQA model. In Table 2, the best three ones are highlighted in boldface for each distortion for SROCC, KROCC, PLCC and RMSE. The source codes of all the other metrics were obtained from the original authors. In Table 3, we show the weighted-average SROCC, KROCC, PLCC, and RMSE results over six databases, where the weight assigned to each database linearly depends on the number of distorted images contained in that dataset. This can provide an overall performance evaluation of the tested IQA indices [START_REF] Wang | Information content weighting for perceptual image quality assessment[END_REF]. The ranking of the weighted-average performances of the evaluated IQA indices based on four different performance metrics, SROCC, KROCC, PLCC, and RMSE, is presented in Table 4. From Table 2, it can be seen that the proposed MCSD performs consistently well on each individual database. It achieves the best on the four largest databases (TID2013, TID2008, CSIQ and LIVE). On IVC and MICT, even though it is not the best, it performs only slightly worse than the best one. While for other competitors, they either work well on one database but poor on other databases. For example, the MAD performs quite well on CSIQ, LIVE, IVC and MICT, but it works quite poor on the two largest databases, TID2013 and TID 2008. The GMSD performs only slightly worse than the proposed MCSD, but it lags the MCSD on MICT, IVC and LIVE databases a lot. From Tables 3 and4 we can see that the MCSD has the best overall performance weighted on the six benchmark databases. The relative bad performance of the MCSD on the MICT will be discussed in the next subsection (after Table 6).

Performance comparisons on individual distortion types

To fully evaluate the IQA models, we also compared their abilities of predicting image quality degradations caused by different types of distortions. We list the results in Table 5 and Table 6. Like the GMSD [START_REF] Xue | Gradient magnitude similarity deviation: a highly efficient perceptual image quality index[END_REF], we also only showed the SROCC scores for space saving purpose. In Table 5 and Table 6, we use boldface font to highlight the top 3 models in each group. It can be seen from Table 5 and Table 6 that the proposed MCSD is among the top 3 models 35 times, followed by the GMSD and the FSIM, which are among the top 3 models 31 times and 23 times, respectively.

We observe in Table 2 that the MCSD has a relatively poor performance on the MICT database. Here from Table 6 we can see that this is because the MCSD has a bad performance on the images with JPEG distortions in the MICT database. Since all the 14 reference images used in the MICT were also used in the TID 2008, TID 2013 and LIVE databases, we also calculated the SROCC values of the MCSD by only using the JPEG distorted images of the 14 images in these three databases and we got good results (e.g. 0.9400 for TID 2013). Since we cannot find all the compression ratios used in the four databases, we cannot verify if the different performances of the MCSD on the JPEG distortions in different databases are due to different compression ratios or different subjective testing methods.

One limitation of the MCSD is that it does not take advantage of the phase information, thus it is unable to catch the phase distortion accurately. The fact can be drawn from Table 5 that the proposed MCSD does not perform quite well for those distortions like transmission errors and Gaussian blur. The transmission errors like JGTE (JPEG transmission errors), J2TE (JP2K transmission errors) and fast fading (a JP2K codec passed through a Rayleigh fast fading channel to simulate packet loss) will lead to the magnitude degradation and phase distortion of the image. The Gaussian blur not only affects the luminance change and the texture shape, but also causes phase shifting and loss.

Performance comparisons across different distortion types

A good IQA model should also predict the image quality consistently across different distortion types. Thus we also show the scatter plots in Fig. 5 for some representative models (SSIM, MS-SSIM, IW-SSIM, MAD, VIF, FSIM, GMSD and MCSD) on the TID2008 database. The curves were obtained by a nonlinear fitting using Eq. [START_REF] Sheikh | An information fidelity criterion for image quality assessment using natural scene statistics[END_REF]. It can be seen that our MCSD is more concentrated across different groups of distortion types than the other competitors. It also shows that objective scores predicted by the MCSD correlate better with human judgments than the other competitors. From above analyses, we can safely reach a conclusion that the proposed model MCSD is a very effective model when compared with other state-of-the-art methods.

Computational cost

Besides the effectiveness, we will show the efficiency (in term of computation cost) of the proposed MCSD. We first analyze the computational complexity of the MCSD and the GMSD, and then give the computational cost (running time) of the competing IQA models. For each scale, the MCSD first convoluted the reference image and the distorted one with a 2×2 average filter , and then computed the quality map using Eq. (3), finally, the deviation pooling was used to yield a quality score. Suppose the total number of pixels in an image is m. The number of multiplications for step and is 8m (m×2×2×2), 21m (4m×2+m×2+4m×2+3×m) and m, respectively. For each scale, the MCSD needs 30m multiplications, thus the total number of multiplications for the MCSD is 90m (30m×3). By contrast, the multiplications for the GMSD are 8m for pre-processing (m×2×2×2), 43m for the quality map generation ((m×3×3×2+2m) ×2+3m) and m for the deviation pooling. Thus the total number of multiplications for the GMSD is 52m (8m+43m+m). In all, the computational complexity for both the MCSD and the GMSD is O (m) and the computational complexity for our MCSD is a litter higher than the GMSD. In addition, the memory complexities for our MCSD and the GMSD are all in O (m). In other words, the time and memory complexities for our MCSD and the GMSD have a linear relation with the image size. In Table 6 7 we list the amount of time (in seconds) to compute each quality measure on a color image of resolution 512×512 (taken from CSIQ database) on a 2.66 GHz Intel Core2 Quad CPU with 5 GB of RAM. Note that all the algorithms were not run through parallelization. The software platform for all algorithms was MATLAB R2013a. The computational costs for all the IQA models were done using 100 repetitions and the first 30 largest values of which were discarded to account for caching effects. The average of the remaining ones was used as the computational cost. In addition, to show how accurate the measurements are, we also list their standard deviation. To reduce the impact of the operating system's scheduler, we terminated as many background processes as possible. We can see from Table 6 7 that our MCSD, the GMSD and the SSIM are the top three efficient IQA models and run much faster than other competing IQA models. For example, the MCSD is 25 times faster than the excellent model FSIM achieving the state-of-the-art prediction performance. In a word, there is no significant difference between the proposed MCSD and the GMSD. One can select the MCSD when considering the effectiveness (correlation with human judgment), and the GMSD when he processes images with huge size (e.g. remote sensing images).

Conclusion

We have presented a highly effective and efficient FR-IQA using the contrast feature and the multi-scale representation. For each image scale, we adopt the standard deviation as the pooling strategy to score the contrast similarity map. Our consideration is that the contrast and the standard deviation have instinctive attributes to some degree; namely, they are both "range" indicators. The contrast indicates the range of image luminance, whereas the standard deviation indicates the range of distortion severity in an image. Using six databases, we compared the proposed MCSD model with other eight state-of-the-art FR-IQA models. The results showed that the MCSD performs better in terms of both accuracy and efficiency, making MCSD a good candidate for high performance IQA applications. To further ameliorate the proposed metric, we will consider how to integrate the phase information into the MCSD to make it more robust to different distortion types in our future work.

Fig. 1 .

 1 Fig.1. Structure diagram of the MCSD. Ref indicates a reference image and Dist indicates the corresponding distorted one, L: low-pass filtering; 2↓: down sampling by 2; CS1,CS2 and CS3 mean contrast similarity map1, contrast similarity map2 and contrast similarity map3, respectively; CSD1, CSD2 and CSD3 mean the standard deviation of CS1, CS2 and CS3, respectively. The general idea of the MCSD is to decompose the reference and the distorted images into three scales, then calculate their contrast similarity deviation for each scale, and get the final note via a pooling of the CSD values on the three scales.

  is the number of concordant pairs in the data set and d n is the number of discordant pairs in the data set. Let 1 a set of joint observations from two random variables (IQA scores X and subjective scores S, respectively). For any pair of observations ( , ) then they are concordant; else if i j

Fig. 2 .

 2 Fig. 2. SROCC with respect to the local window size

Fig. 3 .

 3 Fig.3.SROCC with respect to constant a

Fig. 4 .

 4 Fig.4. The SROCC of using mean and STD to evaluate on LIVE, TID2008, TID2013, CSIQ, MICT and IVC databases. STD denotes the standard deviation

Table 5

 5 SROCC values of IQA model for each type of distortions in TID2013. AGN, ANC, SCN, MN, HFN, IN, QN, GB, DEN, JPEG, JP2K, JGTE, J2TE, NEPN, Block, MS, CTC, CCS, MGN, CN, LCNI, ICQD, CHA and SSR in TID database are the distortions additive Gaussian noise, additive noise in color components, spatially correlated noise, masked noise, high frequency noise, impulse noise, quantization noise, Gaussian blur, image denoising, JPEG compression, JPEG2000 compression, JPEG transmission errors, JPEG2000 transmission errors, non-eccentricity pattern noise, local block-wise distortions of different intensity, mean shift, contrast change, change of color saturation, multiplicative Gaussian noise, comfort noise, lossy compression of noisy images, image color quantization with dither, chromatic aberrations and sparse sampling and reconstruction, respectively.

Fig. 5 .

 5 Fig.5. Scatter plots of predicted quality scores against the subjective quality scores (MOS) by representative FR-IQA models on the TID2008 database.

  1 Initialization: Given the reference image Ref and its distorted image Dist, compute the contrast maps of Ref and Dist, respectively, by using Eq. (3)

	Repeat:
	for j=1:nscales do
	1) Apply a 2×2 average filter on both Ref and Dist, and then down-sample the
	outputs by a factor of 2.
	end for
	beta=[beta1 beta2 beta3];
	MCSD=CSD(1) beta1 ×CSD(2) beta2 ×CSD(3) beta3

2) Compute the contrast maps of Ref and Dist, respectively, by using Eq. (

3

); 3) Compute the quality map of Ref and Dist by Eq. (

5

); 4) Adopt the standard deviation as the the pooling strategy to obtain the image quality through Eq. (

6

);

Table 1

 1 : It contains 1700 test images, which are derived from 25 reference images. There are 17 types of distortions for each reference image and four different scales of each type of distortion. 4) The TID2013 database[START_REF] Ponomarenko | Color image database TID2013: Peculiarities and preliminary results[END_REF]: TID2013 is an acronym for Tampere image database 2013. It is a newly arrived with very huge number of images are present among the IQA databases presented among the IQA databases. Information about the databases.

	It

Table 2

 2 Performances of the proposed metric and the other eight competing FR-IQA metrics on six benchmark databases. The top three metrics for each criterion are highlighted in boldface

			SSIM MS-SSIM IW-SSIM	VIF	MAD	FSIM GMSD MCSD
		SROCC 0.7417	0.7859	0.7779	0.6769 0.7807 0.8015 0.8044 0.8089
	TID2013	KROCC 0.5588 PLCC 0.7895	0.6047 0.8329	0.5977 0.8319	0.5147 0.6035 0.6289 0.6339 0.6385 0.7720 0.8267 0.8589 0.8553 0.8565
		RMSE	0.7608	0.6861	0.6880	0.7880 0.6975 0.6349 0.6423 0.6399
		SROCC 0.7749	0.8542	0.8559	0.7491 0.8340 0.8804 0.8907 0.8911
	TID2008	KROCC 0.5768 PLCC 0.7732	0.6568 0.8451	0.6636 0.8579	0.5861 0.6445 0.6945 0.7092 0.7133 0.8084 0.8308 0.8738 0.8788 0.8844
		RMSE	0.8511	0.7173	0.6895	0.7899 0.7468 0.6527 0.6404 0.6263
		SROCC 0.8756	0.9133	0.9213	0.9195 0.9466 0.9310 0.9570 0.9592
	CSIQ	KROCC 0.6907 PLCC 0.8613	0.7393 0.8991	0.7529 0.9144	0.7537 0.7970 0.7690 0.8122 0.8171 0.9277 0.9502 0.9192 0.9541 0.9560
		RMSE	0.1334	0.1149	0.1063	0.0980 0.0818 0.1034 0.0786 0.0770
		SROCC 0.9479	0.9513	0.9567	0.9636 0.9672 0.9634 0.9603 0.9668
	LIVE	KROCC 0.7963 PLCC 0.9449	0.8045 0.9489	0.8175 0.9522	0.8282 0.8427 0.8337 0.8268 0.8407 0.9604 0.9688 0.9597 0.9603 0.9675
		RMSE	8.9455	8.6188	8.3473	7.6137 6.7672 7.6780 7.6214 6.9079
		SROCC 0.9018	0.8847	0.9125	0.8966 0.9147 0.9262 0.9146 0.9231
	IVC	KROCC 0.7223 PLCC 0.9119	0.7012 0.8934	0.7339 0.9231	0.7165 0.7406 0.7564 0.7371 0.7522 0.9028 0.9210 0.9376 0.9235 0.9292
		RMSE	0.4999	0.5474	0.4686	0.5239 0.4746 0.4236 0.4675 0.4503
		SROCC 0.8794	0.8874	0.9202	0.9077 0.9362 0.9050 0.8528 0.8973
	MICT	KROCC 0.6939 PLCC 0.8887	0.7029 0.8927	0.7537 0.9249	0.7315 0.7823 0.7280 0.6588 0.7180 0.9138 0.9367 0.9065 0.8582 0.8971
		RMSE	0.5738	0.5639	0.4759	0.5084 0.4381 0.5283 0.6424 0.5531

Table 3

 3 Overall performance of IQA models over six databases

	IQA models	SROCC	KROCC	PLCC	RMSE
	SSIM	0.7993	0.6159	0.8186	1.6426
	MS-SSIM	0.8442	0.6637	0.8612	1.5358
	IW-SSIM	0.8443	0.6677	0.8680	1.4926
	VIF	0.7718	0.6112	0.8305	1.4788
	MAD	0.8450	0.6750	0.8656	1.3237
	FSIM	0.8631	0.6935	0.8856	1.3813
	GMSD	0.8684	0.7020	0.8874	1.3781
	MCSD	0.8729	0.7092	0.8915	1.2877

Table 4

 4 Ranking of overall performance of IQA models

	IQA models	SROCC	KROCC	PLCC	RMSE
	SSIM	7	7	8	8
	MS-SSIM	4	6	6	7
	IW-SSIM	6	5	4	6
	VIF	8	8	7	5
	MAD	5	4	5	2
	FSIM	3	3	3	4
	GMSD	2	2	2	3
	MCSD	1	1	1	1

Table 6

 6 SROCCs of IQA models for each type of distortions in TID20008, CSIQ, LIVE and MICT.The abbreviations for distortion types in TID2008 are the same with those in TID2013 as given in the caption of Table5. The AGWN, JPEG, JP2K, AGPN, GB and GCD in CSIQ database denote the distortion types of additive Gaussian white noise, JPEG compression, JPEG2000 compression, additive pink Gaussian noise, Gaussian blur and global contrast decrements, respectively. The JP2K, JPEG,

	SSIM	MS-SSIM	IW-SSIM	VIF	MAD	FSIM	GMSD MCSD

Table 6 7

 6 Running time of the competing IQA models IQA models Running time (s) standard deviation Running time(s)

	standard deviation

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grants 81272501 and by Taishan Scholars Program of Shandong Province, Chi na (Grant NO. ts20120505).

Highlights

Perceptual image quality assessment (IQA) uses a computational model to assess the image quality in consistent with human opinions. Generally, the effectiveness, namely, high correlation with the human subjective score, is the prerequisite of a good IQA model, it is of first importance to a IQA model. The efficiency (the least computation cost), however, is the second requirement of a good IQA model which become important under the premise that an IQA model meet the condition of "effectiveness". Thus the effectiveness and efficiency are two ultimate goals for the design of IQA models, but unfortunately it is hard to reach well these goals simultaneously. In this paper, we developed an effective and efficient IQA model called multiscale contrast similarity deviation (MCSD) which explores the contrast features by resorting to multiscale representation. Performances on six benchmark databases demonstrate its effectiveness and efficiency.