Tom Rohmer
email: tom.rohmer@inrae.fr

Some numerical methods with R

Generating distributions

In the following exercises, we will use the R functions:

• rnorm, rexp, rbinom, rpois to simulate random variables with respectively Gaussian, exponential, binomial and poisson distributions • pnorm, pexp, pbinom, ppois to to evaluate the c.d.f. of random variables with respectively Gaussian, exponential, binomial and poisson distributions • qnorm, qexp, qbinom, qpois to to evaluate the quantile function, i.e. the inverse of the c.d.f. of random variables with respectively Gaussian, exponential, binomial and poisson distributions • dnorm, dexp, dbinom, dpois to to evaluate the density or p.m.f. of random variables with respectively Gaussian, exponential, binomial and poisson distributions

Simulation of classical distributions

First, fix the seed by using the following command set.seed(1)

• Using the R functions rexp, rnorm, rbinom, generate i.i.d. samples X 1 , X 2 and X 3 of size n = 10000 respectively from exponential distribution with shame parameter λ = 2, from Gaussian distribution with mean value µ = 3 and standard deviation sd = 2 and from Bernoulli distribution with probability of success p = 0.25.

• Create a data frame X (R command data.frame) which contains X 1 , X 2 and X 3 . Print the top of the data frame (use head). • Using the R function apply, Calculate the mean value and the empirical variance of the samples.

Compare with the theoretical values. Evaluate the Pearson's correlation between the 3 samples and also the covariance matrix (use cov(X)).

Histogram of X1

Simulating distributions using uniform samplings

Let U a random variable distributed from the uniform distribution on [0, 1]. Then for any cumulative distribution function F , the random variable

Z = F -1 (U ) has distribution F.
Using this result, simulate the same exponential, Gaussian samples and answer the same questions as exercise 1. Use the r function runif, qexp, qnorm, qbinom

U1=runif(n) U2=runif(n) U3=runif(n) X1= qexp(U1, 2) X2= qnorm(U2,3,2) X3=qbinom(U3,

Histogram of X1

table(X3) ## X3 ## 0 1 ## 7521 2479
Let X a continuous random variable with c.d.f. F . Then the random variable Z = F (X) is uniformly distributed on [0, 1].

Using this result, from a Gaussian distribution, then from an exponential distribution, simulate a uniform sample on [0, 1].

X1=rnorm(n,2,3) X2=rexp(n,3) U1=pnorm(X1,2,3) U2=pexp(X2,3) hist(U1,breaks=30,freq=FALSE) U1 Density 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 hist(U2,breaks=30,freq=FALSE)

Histogram of U2

U2 Density 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0

Theorem Central Limit (TCL), convergence in distribution

Let X 1 , . . . , X n an i.i.d. sample with expectation E(X) = µ and variance V ar(X) = σ 2 . Then,

√ n( Xn -µ) → N (0, σ 2 ) as soons as n → ∞
We say that the mean value Xn converges in distribution to a Gaussian distribution.

Using a loop 'for', simulate M = 10 4 samples of size n = 1000 from a Poisson distribution with parameter λ = 5 and verify the theorem. Plot the histogram of the statistic with the density of the Gaussian. try now with a exponential distribution with rate parameter λ = 5. Note that for an exponential distribution the expectation is 1/λ = 1/5, and the variance is 1/λ 2 = 1/25 (so the sd is 1/5) M=10ˆ4;n=10ˆ3 Z=NULL; We say that Xn converges almost surely (a.s.) to µ.

M=10ˆ4;n=10ˆ3 Z=NULL; for(m in 1:M){ X=rpois(n,5) Z[m]=sqrt(n)*(mean(X) -5) } hist(Z,freq=FALSE) plot(function(x) dnorm(x,0,sqrt(5)),add=TRUE,col="red",xlim=c(-6,6) )

Histogram of Z

for(m in 1:M){ X=rexp(n,5) Z[m]=sqrt(n)*(mean(X) -1/5) } hist(Z,freq=FALSE) plot(function(x) dnorm(x,0,sqrt(1/25)),add=TRUE,col="red",xlim=c(-1,1) )

Histogram of Z

simulate a sample of size n = 10 4 of exponential distribution with parameter λ = 2. For k varying from 1 to n, evaluate the mean value Z k = Xk . Plot the sequence Z k and add the horizontal line y = 0.5 (use abline(h=0.5)).

set.seed(2) n=10ˆ4 Z=NULL; X=rexp(n,2) for(k in 1:n){ Z[k]=mean(X[1:k]) } plot(Z) abline(h=1/2,col="red") 0 2000
4000 6000 8000 10000 0.4 0.5 0.6 0.7 0.8 0.9 Index Z 1.5 Multivariate Gaussian distribution

Cholesky decomposition

Any symmetric definite-positive matrix Σ can to be decompose in the product of an lower triangular matrix L (composed with 0 above the diagonal) and his transposed:

Σ = LL T .
Find these two triangular matrices require specific algebra algorithm (refereed as the Cholesky decomposition).

Let

Σ = 1 0.4 0.4 1
Using the eigen R function that provides the eigen value of a matrix, check that the matrix Σ is definitepositive (Note that a symmetric matrix is positive definite if and only if all the eigen values of Σ are strictly positives). Use the R function chol to get the lower triangular matrix. Make the matricial product to check that Σ = LL T . The R matricial product is %*% and the transpose operation is t().

sigma=matrix(c(1,0.4,0.4,1),2,2) eigen(sigma)

## eigen() decomposition ## $values ## [1] 1.4 0.6 ## ## $vectors ## [,1] [,2] ## [1,] 0.7071068 -0.7071068 ## [2,] 0.7071068 0.7071068 L=t(chol(sigma)) L%*%t(L) ## [,1] [,2] ## [1,] 1.0 0.4 ## [2,] 0.4 1.0

Simulating of the multivariate Gaussian distribution

Let X = (X 1 , X 2 ) be a random vector. We say that X is multivariate Gaussian with mean value m = (m 1 , m 2 ) and covariance matrix Σ if the density of X is

f (x, m, Σ) = 1 (2π) n/2 √ det Σ exp - 1 2 (x -m) t Σ -1 (x -m) , x ∈ R 2
To simulate a multivariate Gaussian sample with mean value m = (m 1 , m 2 ) and covariance matrix Σ we can use the Cholesky decomposition. In fact, Consider Z 1 and Z 2 two independent random variable from the Gaussian distribution with mean value 0 and sd 1. Let Z = (Z 1 , Z 2 ). Then we have the property that the vector X = L T Z + m is multivariate Gaussian with covariance matrix Σ and mean value m.

• Generate a sample of size n from the multivariate Gaussian distribution with covariance Σ and mean value m = c(0, 0). To do that, use the cholesky decomposition (optional) or use the R package mvtnorm:

##or using mvtnorm library(mvtnorm) sigma=matrix(c(1,0.4,0.4,1),2,2) #matrix sigma m=c(0,0) #mean value vector m X<-rmvnorm(M,m,sigma)

• Plot the sample and the two histograms

• Calculate the mean value and the empirical variance of the two component of the samples and the empirical covariance between the two components.

• Start again with Σ 2 = 1 0.1 0.1 1 and Σ 3 = 1 0.9 0.9 1 .

• Use the following code to add the 'contour-plot' of the distribution library(rvinecopulib) ## Warning: le package rvinecopulib a été compilé avec la version R 4.1.3 plot(X[,1],X[,2],main=paste("rho=",0.4)) Cn <-bicop_dist(family = "gaussian", rotation = 0, parameters =0.4) contour(Cn,col="blue",add=TRUE)

-3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 rho= 0.4 X[, 1] X[, 2]

Other multivariate distributions

Simulate a bivariate sample (X 1 , Y 1 ), . . . , (X n , Y n ) in which X 1 , . . . , X n are i.i.d. and follow a exponential distribution with shape parameter λ = 3, Y 1 , . . . , Y n are i.i.d. and follow a Gaussian distribution white mean value µ = 2 and variance σ 2 = 1, and such that X 1 and X 2 be correlated To do that:

• Simulate a (X 1 , Y 1 ), . . . , (X n , Y n ) following a multivariate Gaussian distribution with mean value m = (2, 2) and covariance 0.4 using the R package mvtnorm and the function rmvnorm. • Transform X 1 , . . . , X n in order to obtain an uniform sample following Exercise 2.

• Transform the uniform sample in order to obtain an exponential sample following Exercise 2.

• plot Y vs X. Plot the histogram of X 1 , . . . , X n and Y 1 , . . . , Y n • print the Pearson's correlation and Spearman's correlation.

(mvtnorm) X<-rmvnorm(M,c(2,2),K) X[,1]<-pnorm(X[,1],2,1) X[,1]<-qexp(X[,1],3) hist(X[,1],freq=FALSE) plot(function(x) dexp(x,3),xlim=c(0,3),add=TRUE,col="red") Histogram of X[, 1] X[, 1] Density 0 1 2 3 4 0.0 0.5 1.0 1.5 2.0 plot(X[,1],X[,2]) 0 1 2 3 4 -2 0 2 4 X[, 1] X[, 2] cor(X[,1],X[,2]) ## [1] 0.3643721 cor(X[,1],X[,2],method="spearman") ## [1] 0.3909433
• Use the following R code to have the scatterplot of the distribution and the histograms: (vx,vy,z,col=rev(heat.colors(101))) plot(X,cex=0.2) #points(X,cex=.2) par(mar=c(0,3,1,1)) barplot(xhist$density, axes=FALSE, ylim=c(0, top), space=0) lines((density(X

xhist <-hist(X[,1], plot=FALSE) yhist <-hist(X[,2], plot=FALSE) top <-max(c(xhist$density, yhist$density)) nf <-layout(matrix(c(2,0,1,3),2,2,byrow=TRUE), c(3,1), c(1,3), TRUE) par(mar=c(4,4,1,1)) #image
[,1])$x-xhist$breaks[1])/diff(xhist$breaks)[1], dexp(density(X[,1])$x,3),col="red") par(mar=c(3,0,1,1)) barplot(yhist$density, axes=FALSE, xlim=c(0, top), space=0, horiz=TRUE) lines(dnorm(density(X[,2])$x,2,1),(density(X[,2])$x-yhist$breaks[1])/ diff(yhist$breaks)[1],col="red") log L(µ, σ 2 ; y 1 , . . . , y n ) = n i=1 log L(µ, σ 2 ; y i ) = n i=1 log 1 √ 2πσ 2 exp - (y i -µ) 2 2σ 2 = - n 2 log(2π) - n 2 log(σ 2 ) - n i=1 (y i -µ) 2 2σ 2 So ∂ ∂µ log L(µ, σ 2 ; y 1 , . . . , y n ) = 0 ⇔ ∂ ∂µ - n 2 log(2π) - n 2 log(σ 2 ) - n i=1 (y i -µ) 2 2σ 2 = 0 ⇔ ∂ ∂µ n i=1 (y i -µ) 2 2σ 2 = 0 ⇔ - 1 σ 2 n i=1 (y i -µ) = 0 ⇔ µ = ȳn ∂ ∂σ 2 log L(µ, σ 2 ; y 1 , . . . , y n ) = 0 ⇔ ∂ ∂σ 2 - n 2 log(2π) - n 2 log(σ 2 ) - n i=1 (y i -µ) 2 2σ 2 = 0 ⇔ ∂ ∂σ 2 - n 2 log(σ 2 ) - n i=1 (y i -µ) 2 2σ 2 = 0 ⇔ - n 2σ 2 + 1 2σ 4 n i=1 (y i -µ) 2 = 0 ⇔ σ 2 = 1 n n i=1 (y i -ȳn ) n=10ˆ4 Y<-rnorm(n,2,2) mu.MLE<-mean(Y) sigma2.MLE<-var(Y) mu.MLE;sigma2.MLE ## [1] 1.98438 ## [1] 4.060696
• Generate a sample (use rnorm function) of size n = 10 6 from a Gaussian distribution with mean value µ = 2 and variance σ 2 = 4. Estimate the MLE and illustrate the a.s. convergence of the estimator, that is to say the convergence of the estimator when the sample size increase

Me<-Va<-NULL for(m in 1:M){ Me[m]=mean(Y[1:m]) Va[m]=var(Y[1:m]) } plot(Me) abline(h=2,col="red")

Gamma distribution

The density of the Gamma distribution with shape parameter α and rate parameter β is defined by

f (x; α, β) = β α Γ(α) x α-1 exp(-βx), x > 0, α > 0, β > 0,
where the Gamma function Γ is defined by

Γ(z) = ∞ 0 t z-1 e -t dt.
Note that Γ(z + 1) = zΓ(z) and Γ(1) = 1. If Y follows a Gamma distribution with shape parameter α and rate parameter β,

E(Y ) = α β , V ar(Y ) = α β 2 .
• Check that for α = 1, the Gamma distribution is the exponential distribution.

• Plot the density of the Gamma distribution for many parameter α and β (use dgamma function).

plot(function(x) dgamma(x,1,3),xlim=c(0,5)) plot(function(x) dgamma(x,2,0.5),xlim=c(0,5),col="red",add=TRUE) plot(function(x) dgamma(x,4,6),xlim=c(0,5),col="blue",add=TRUE) plot(function(x) dgamma(x,0.5,6),xlim=c(0,5),col="green",add=TRUE)

  Note that pour a Poisson distribution the expectation and the variance is λ = 5, so the standard deviation is √ λ = 5.

  of large numbers (LLN), almost surely convergenceLet X 1 , . . . , X n an i.i.d. sample with expectation µ. Then P ( Xn → µ) = 1.

  (c(1,rho,rho,1),2,2) ##matrice de covariance souhaitée ##in the same way of the previous exercise L=chol(K) ##matrice triangulaire supérieur L=t(L) ##matrice triangulaire inférieur X=matrix(rep(0,M*2),M,2) ##Initialisation de la matrice for(m in 1:M) X[m,]=L%*%rnorm(2) + c(2,2) ##Remplissage de la matrice X ##or using mvtnorm library

  number of iteration to reach the solution • Compare the solution with the true value that is qnorm(0.975)• print the convergence of the algorithm maxit=1000 >=0) a<-c[m] if(abs(b-a)<eps) break; } c[m]=(a+b)/2 #print(c[m]) #print(qnorm(0.975))plot(c, type= l ,ylab="value",xlab="iteration") abline(h=qnorm(0.975),col="red",lty=2) ("we used",m,"iterations to get the solution",c[m])) ## we used 27 iterations to get the solution 1Let recall that the equation of the unit circle (circle with a radius of 1) is given by x 2 + y 2 = 1. In the following Figure, the circle equation y = √ 1 -x 2 is plotted on [0,1]. plot(function(x) sqrt(1-xˆ2),ylab="",xlab="")

•

  Evaluate π/4 then π • Illustrate the (almost surely) convergence of the Monte Carlo estimator of π. (Note that in R, you can access to the true value of π using the R command pi): Z=NULL for(k in 1:M) Z[k] =4*mean(U[1:k,1]ˆ2+U[1:k,2]ˆ2<1)plot(Z,type= l ,xlab="number of simulations",ylab="") abline(h=pi,col="red",lty=2) text(10000+0.04,pi+0.04,expression(pi),col="red") Carlo approximation of the area under the Gaussian densityIn the the same way of exercise 1, propose a Monte Carlo(MC) algorithm to evaluate the area under the Gaussian density between a = -1 and b = 5, that is to say the probability P (a ≤ X ≤ b) = P (X ≤ b) -P (X ≤ a) for X a random variable with standard Gaussian distribution.• Plot the density of the Gaussian distribution on the interval (-10, 10). • In the same graphic, add the vertical lines a = -1 and b = 5.• Add the points of the MC algorithm which are under the Gaussian curve in red and the points of the MC algorithm which are above the Gaussian curve in blueP (a ≤ X ≤ b) ≈ number of red points total number of points × (b -a)• Evaluate the value of the area under the Gaussian density between a = -1 and b = 5 and compare with the theoretical value obtained using pnorm • Illustrate the (almost surely) convergence of the Monte Carlo estimation of the area under the Gaussian curve.

#Print the head of the matrix (or dataset) X

  • Plot the histogram of the distributions of the exponential et Gaussian distribution and add the corresponding density curves (use hist(X1, freq=FALSE)) • print the table of frequency (use table(X3)/n) and compare these values with p and 1 -p

	n=10000
	X1<-rexp(n,2)
	X2<-rnorm(n,3,2)
	X3<-rbinom(n,1,0.25)
	X=data.frame(exp=X1,Gauss=X2,Bernouilli=X3) ###concatenation de X1, X2 et X3
	head(X) ###	exp	Gauss Bernouilli
	## 1 0.37759092 4.869048	0
	## 2 0.59082139 3.580626	0
	## 3 0.07285336 3.387249	0
	## 4 0.06989763 5.147031	1
	## 5 0.21803431 2.286873	0
	## 6 1.44748427 3.790633	0
	apply(X,2,mean)
	##	exp	Gauss Bernouilli
	## 0.4991806 3.0078905 0.2482000
	apply(X,2,var)
	##	exp	Gauss Bernouilli
	## 0.2578852 4.0025920 0.1866154
	cov(X)
	##			exp	Gauss	Bernouilli
	## exp		0.25788516 -0.023411310 -0.006222800
	## Gauss		-0.02341131 4.002592022 -0.005486475
	## Bernouilli -0.00622280 -0.005486475 0.186615422
	hist(X1,breaks=30,freq=FALSE)
	plot(function(x) dexp(x,2),add=TRUE,col="red",xlim=c(0,5))

##concatenation de X1, X2 et X3 head(X) ##Affiche le début de la matrice X

	## 3 0.59454820 2.5494842	0
	## 4 0.08673629 6.3367267	0
	## 5 0.91940992 2.3569812	1
	## 6 0.78985083 4.1971632	0
	apply(X,2,mean)
	##	exp	Gauss Bernouilli
	##	0.507474	3.009853	0.247900
	apply(X,2,var)
	##	exp	Gauss Bernouilli
	## 0.2578895 4.0610531 0.1864642
	cor(X)
	##		exp	Gauss	Bernouilli
	## exp	1.0000000000 -0.0009199897 -0.0121168116
	## Gauss	-0.0009199897 1.0000000000 -0.0009433278
	## Bernouilli -0.0121168116 -0.0009433278 1.0000000000
	hist(X1,breaks=30,freq=FALSE)
		1,0.25)
	X=data.frame(exp=X1,Gauss=X2,Bernouilli=X3) ### exp Gauss Bernouilli
	## 1 0.01972026 4.5880807	0
	## 2 0.18615195 0.7783434	0
					4

2 Maximum likelihood estimator

log-Likelihood

Let y 1 , . . . , y n a i.i.d. sample with density (or p.m.f.) f with parameter θ. The likelihood of the sample y 1 , . . . , y n is

where L(θ; y i ) is the function θ → f (y i , θ). Generally, Instead of the likelihood function, we consider the log-likelihood:

The Maximum Likelihood Estimation (MLE) consists to maximizing the log-likelihood of the sample, (or minimizinglog L(θ; y 1 , . . . , y n ) ). With some distributions, it can be done to solve the scoring equations, that are ∂ ∂θ log L(θ; y 1 , . . . , y n ) = 0.

for many distributions, the scoring equations do not have an explicit solution, and an optimization procedure can be down to obtain the MLE, use optim in R.

• Let y 1 , . . . , y n be a sample from a Gaussian distribution with mean value µ and variance σ 2 . Found the MLE of µ and σ 2 .

Dichotomy algorithm (bisection method)

The objective of the exercise is to find the value of x such that F (x) = P (X ≤ x) = 0.975, that is to say F -1 (0.975) where X is a random variable following a standard Gaussian distribution, using a dichotomy algorithm plot(function(x) pnorm(x),xlim=c(-6,6)) abline(h=0.975,col="red",lty=2) abline(v=qnorm(0.975),col="blue",lty=2) points(c(qnorm(0.975),0.01),pch="?",col="blue") Consider now the middle of the new interval and find a smaller interval which contains x. Reiterate the algorithm as many times as necessary until you get an interval smaller than an , chosen by user. (We will consider = 10 -7 ). The final value for the estimation of x will be the middle of the considered interval.

To do that:

• define a maximum number of iteration, maxit = 1000 and do a loop from 1 to maxit.

• Use the following command to stop the algorithm: In this example,

• U 1 = (U 1,1 , U 1,2 ) = (0.27, 0.37) is under the curve of the quarter circle:

Remember that the area of the circle is π so the area of the quarter circle is π/4. What is the probability for any simulated U = (U 1 , U 2 ) where U 1 and U 2 are uniform, to be under the curve of the quarter circle?

To estimate π/4, Monte Carlo procedure consists in simulating M random vectors U 1 = (U 1,1 , U 1,2 ), . . . , U M = (U M,1 , U M,2 ), according to uniform distributions on [0, 1], and evaluate the proportion of the points U m which are under the curve of the quarter circle, that is to say, such that U 2 m,1 + U 2 m,2 < 1. • Using M = 10 4 , simulate M such random vectors • Plot the points which are under the curve of the quarter circle in red and the points which are above the curve of the quarter circle in blue:


Yannan Zheng

Weiling Chen
email: weiling.chen@fzu.edu.cn

Rongfu Lin

Tiesong Zhao
email: t.zhao@fzu.edu.cn

UIF: An Objective Quality Assessment for Underwater Image Enhancement

Keywords: CCS CONCEPTS, Computing methodologies->Image Processing; Image Quality Assessment (IQA), Underwater Image Enhancement (UIE), underwater image processing

Due to complex and volatile lighting environment, underwater imaging can be readily impaired by light scattering, warping, and noises. To improve the visual quality, Underwater Image Enhancement (UIE) techniques have been widely studied. Recent efforts have also been contributed to evaluate and compare the UIE performances with subjective and objective methods. However, the subjective evaluation is time-consuming and uneconomic for all images, while existing objective methods have limited capabilities for the newly-developed UIE approaches based on deep learning. To fill this gap, we propose an Underwater Image Fidelity (UIF) metric for objective evaluation of enhanced underwater images. By exploiting the statistical features of these images, we present to extract naturalness-related, sharpness-related, and structure-related features. Among them, the naturalness-related and sharpness-related features evaluate visual improvement of enhanced images; the structure-related feature indicates structural similarity between images before and after UIE. Then, we employ support vector regression to fuse the above three features into a final UIF metric. In addition, we have also established a large-scale UIE database with subjective scores, namely Underwater Image Enhancement Database (UIED), which is utilized as a benchmark to compare all objective metrics. Experimental results confirm that the proposed UIF outperforms a variety of underwater and general-purpose image quality metrics.

INTRODUCTION

The underwater optical images bring additional information beyond sonar imaging. However, the complex waterbody and poor light conditions impair the visual quality of underwater images.

In practice, Underwater Image Enhancement (UIE) technique is thus necessary to transfer low-quality underwater images to highquality ones. As shown in Fig. 1, diversified enhancements are employed in original underwater images, resulting in pictures with higher visual qualities. To compare the UIE algorithms and select optimal results, it is imperative to score these enhanced images, which is still a challenging task. Until now, the visual quality of images can be evaluated by Image Quality Assessment (IQA), whose popular works include subjective and objective methods. For most of images, human is the ultimate receiver, thus the subjective evaluation is considered to be the most accurate and reliable way of IQA. Recently, ITU has promoted several methods to subjectively evaluate image quality [START_REF]Methodology for the Subjective Assessment of the Quality of Television Pictures BT Series Broadcasting Service[END_REF]. However, the subjective evaluation also has significant drawbacks: high complexity, high cost and unable to be embedded into real-world systems. As a result, the subjective methods are usually utilized to set benchmark to evaluate objective metrics, and the objective metrics with higher correlations to subjective scores are embedded into real-world systems.

Existing objective IQA can be classified into full-reference, reducedreference and no-reference algorithms, subject to the accessibility of ideally unimpaired references. In underwater imaging, an ideally unimpaired image is unable to be obtained, thus the typical reference-based IQA approaches, such as Structural Similarity Index (SSIM) [START_REF] Wang | Deep Learning for Image Super-resolution: A Survey[END_REF] and Feature Similarity Index (FSIM) [START_REF] Zhang | FSIM: A Feature Similarity Index for Image Quality Assessment[END_REF], are not applicable. On the other hand, no-reference IQA approaches have achieved significant performances to evaluate generic images [START_REF] Gu | Using Free Energy Principle For Blind Image Quality Assessment[END_REF][START_REF] Mittal | No-Reference Image Quality Assessment in the Spatial Domain[END_REF][START_REF] Mittal | Making a "Completely Blind" Image Quality Analyzer[END_REF][START_REF] Yang | Blind image quality assessment on authentically distorted images with perceptual features[END_REF][START_REF] Zhang | A Feature-Enriched Completely Blind Image Quality Evaluator[END_REF]. However, these no-reference approaches are usually designed based on Natural Scene Statistics (NSS), which performs different in underwater environment. Thus, they also fail to evaluate the quality of underwater images.

The Underwater Color Image Quality Evaluation (UCIQE) [START_REF] Yang | An Underwater Color Image Quality Evaluation Metric[END_REF] and Underwater Image Quality Measure (UIQM) [START_REF] Panetta | Human-Visual-System-Inspired Underwater Image Quality Measures[END_REF] metrics have been widely used to qualify the enhancement performance of underwater images [START_REF] Chang | Single Underwater Image Restoration Based on Depth Estimation and Transmission Compensation[END_REF][START_REF] Li | Underwater Scene Prior Inspired Deep Underwater Image and Video Enhancement[END_REF][START_REF] Li | Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior[END_REF][START_REF] Peng | Underwater Image Restoration Based on Image Blurriness and Light Absorption[END_REF]. Among them, the UCIQE metric quantifies non-uniform color casts, blurring, and noise in underwater images, and then combines them in a linear manner. The UIQM metric evaluates underwater images with color, sharpness and contrast. These two methods have been proved to achieved good performances in most underwater scenarios at that time. However, this decade has witnessed a booming of UIE algorithms, especially deep-learning-based UIE algorithms. In such case, the UIE may generate complicated color and structure changes which are unable to be evaluated by conventional metrics. From Fig. 1, a higher UCIQE/UIQM score may not represent a better visual quality. For example, some images have obvious reddish color shifts and artifacts as shown in Fig 1 . (b), (g) and (k), but these images obtain better UCIQE and UIQM scores. Therefore, it is highly desirable to design an effective IQA metric for objective UIE evaluation.

To address this issue, we exploit the characteristics of underwater images to obtain three types of features: naturalness-related, sharpness-related and structure-related features, which are demonstrated to be effective to assess the underwater image quality. The Support Vector Regression (SVR) are then utilized to fuse all features into a final metric. In addition, we also develop the largest-ever UIE quality database with subjective scores, which helps to guarantee the generalization performance of our model. It can also serve as a benchmark to compare subjective IQA metrics for UIE. In summary, the main contributions of this paper include:

1) A large-scale Underwater Image Enhancement Database (UIED), which is the first-of-its-kind database with human-labeled quality scores. The database includes different underwater scenes that are enhanced by 10 representative UIE approaches. The UIED can be utilized as a benchmark to develop and evaluate subjective methods of underwater image quality assessment. 2) Three types of image features to evaluate the underwater image fidelity in three dimensions. An optimal UIE should generate natural images with sharp textures and high structure similarity to original image. Inspired by this, we propose to extract and model the three types of features based on statistics of underwater images. The effectiveness of features has been proved in ablation study. 3) An SVR-based fusion of features to obtain our Underwater Image Fidelity (UIF) metric. We employ the popular SVR models for feature pooling and regression. To ensure the generalization performance of our UIF metric, it is trained and examined by kfold validation in UIED. Experimental results reveal the efficiency of our method.

RELATED WORK

In this section, we review related works of our paper. Among them, the UIE methods are utilized to construct our UIED database. The conventional IQA methods and Underwater IQA methods are compared in our experiments.

Underwater Image Enhancement Methods

In recent years, many UIE methods have been proposed. The existing UIE methods can be divided into three categories: Non-physicalmodel-based methods [START_REF] Ancuti | Enhancing underwater images and videos by fusion[END_REF], [START_REF] Fu | A retinex-based enhancing approach for single underwater image[END_REF], [START_REF] Fu | Twostep approach for single underwater image enhancement[END_REF], physical-model based [START_REF] Paulo | Underwater Depth Estimation and Image Restoration Based on Single Images[END_REF], [START_REF] Peng | Generalization of the Dark Channel Prior for Single Image Restoration[END_REF], [START_REF] Peng | Underwater Image Restoration Based on Image Blurriness and Light Absorption[END_REF], [START_REF] Li | Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior[END_REF] and deep-learning-based methods [START_REF] Li | Underwater Scene Prior Inspired Deep Underwater Image and Video Enhancement[END_REF], [START_REF] Li | An Underwater Image Enhancement Benchmark Dataset and Beyond[END_REF].

Non-physical-model-based methods aim to adjust input image pixel values to improve visual quality. Ancuti et al. [START_REF] Ancuti | Enhancing underwater images and videos by fusion[END_REF] proposed a UIE algorithm with multi-scale fusion strategy. Fu et al. [START_REF] Fu | A retinex-based enhancing approach for single underwater image[END_REF] proposed a retinex-based UIE method, which decomposes the reflectance and illumination of underwater images. In [START_REF] Fu | Twostep approach for single underwater image enhancement[END_REF], they also proposed a two-step enhancement procedure, which includes a color correction and a contrast enhancement.

Physical-model-based methods construct physical models for underwater degradation. Li et al. [START_REF] Li | Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior[END_REF] proposed a contrast enhancement algorithm which combined with an image dehazing model. In [START_REF] Paulo | Underwater Depth Estimation and Image Restoration Based on Single Images[END_REF], an Underwater Dark Channel Prior (UDCP) was proposed based on the fact that the information of red channel is undependable. Peng et al. [START_REF] Peng | Generalization of the Dark Channel Prior for Single Image Restoration[END_REF] proposed a Generalized Dark Channel Prior (GDCP) that incorporates adaptive color correction into an image formation model for UIE. Based on image blurriness and light absorption, they also [START_REF] Peng | Underwater Image Restoration Based on Image Blurriness and Light Absorption[END_REF] proposed a depth estimation method for underwater scenes.

Deep-learning-based methods have led a fast development and offered state-of-the-art performance in many UIE tasks. Li et al. [START_REF] Li | Underwater Scene Prior Inspired Deep Underwater Image and Video Enhancement[END_REF] proposed an end-to-end deep network, namely UWCNN, to address the UIE problem for diverse underwater images. In [START_REF] Li | An Underwater Image Enhancement Benchmark Dataset and Beyond[END_REF], they also designed a Water-Net model which is trained on paired underwater images and the corresponding reference images.

In addition, the commercial application dive+ [2] has also been released in iTunes store, with good performances of underwater enhancement.

Conventional IQA Methods

IQA plays an important role in many computer vision problems. In recent years, metrics have been used to evaluate enhancement or restoration performance for underwater images.

Most traditional no-reference IQA metrics are based on NSS regularities. Anish et al. extracted effective statistical features to evaluate image quality [START_REF] Mittal | No-Reference Image Quality Assessment in the Spatial Domain[END_REF]. The Natural Image Quality Evaluator (NIQE) model proposed by Mittal et al. [START_REF] Mittal | Making a "Completely Blind" Image Quality Analyzer[END_REF] extracted a set of local features from an image. Gu et al. [START_REF] Gu | Using Free Energy Principle For Blind Image Quality Assessment[END_REF] designed No-reference Free Energy Robust Metric (NFERM) by adding features of Human Vision System (HVS). Gu et al. [START_REF] Gu | No-reference image sharpness assessment in autoregressive parameter space[END_REF] solved the no-reference IQA for blur images using a sharpness metric in autoregressive parameter space. Xue et al. [START_REF] Xue | Blind image quality assessment using joint statistics of gradient magnitude and laplacian features[END_REF] combined gradient magnitude with laplacian of gaussian to predict image quality. Kang et al. [START_REF] Kang | Convolutional Neural Networks for No-Reference Image Quality Assessment[END_REF] investigated a Convolutional Neural Network (CNN) to jointly learn features for IQA. However, an underwater image is always degraded by light absorption and scattering, which may not be captured by these NSS models effectively. Thus, these NSS-based approaches are not effective enough for underwater images.

Meanwhile, some no-reference IQA methods have been proposed for enhanced image quality assessment. Fang et al. [START_REF] Fang | No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics[END_REF] proposed a blind contrast quality metric based on image skewness, kurtosis, and entropy. Gu et al. [START_REF] Gu | Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data[END_REF] proposed a blind metric according to 17 features through analysis of images. These methods may not perform well in underwater IQA due to lack of consideration on the underwater imaging models and image features. Liu et al. [START_REF] Liu | Enhanced image no-reference quality assessment based on color space distribution[END_REF] used synthetic data to generate the ideal reference image, thereby converting the evaluation of enhanced images into a full-reference IQA problem. However, its synthetic data only cover very limited types of underwater conditions, which may be insufficient to simulate underwater image distortions. Thus, these methods are also not reliable to apply for underwater image enhancement.

Underwater IQA Methods

Until now, there are two underwater IQA methods that are widely applied to evaluate UIE approaches. They are the UCIQE [START_REF] Yang | An Underwater Color Image Quality Evaluation Metric[END_REF] and UIQM [START_REF] Panetta | Human-Visual-System-Inspired Underwater Image Quality Measures[END_REF] metrics.

Yang et al. designed the UCIQE metric in [START_REF] Yang | An Underwater Color Image Quality Evaluation Metric[END_REF]. The UCIQE transforms an underwater image from RGB color space to CIELab color space, which is more consistent with HVS. It quantifies non-uniform color casts, blurring and noise in underwater monitor images and then linearly combines these three components. A higher UCIQE score indicates the result has a better balance among the chroma, saturation and contrast.

Panetta et al. proposed the UIQM inspired by HVS in [START_REF] Panetta | Human-Visual-System-Inspired Underwater Image Quality Measures[END_REF]. The UIQM comprises a colorfulness measure, asharpness measure, and a contrast measure for underwater images, based on the underwater image modeling presented in [START_REF] Panetta | Human-Visual-System-Inspired Underwater Image Quality Measures[END_REF]. The choice of weighted coefficients depends on the application purpose. A higher UIQM score denotes the result is more consistent with human visual perception.

As shown in Fig. 1, the two metrics also have lower correlations to human scores when the enhancement distortion is complex. However, due to the wide applications of deep learning models, the UIE technique has inevitably brought complex warping and color changes. To model the impacts of these complex distortions, we suggest to utilize a deep learning network. This also requires a large-scale database to train and test the deep-learning-based model.

PROPOSED UIED DATABASE

There is a lack of publicly available large-scale UIE database with human subjective scores. To fill this void, the UIED dataset is developed as the largest-ever subjective database of UIE images, which can serve as a benchmark to develop and evaluate objective approaches. In this section, we will elaborate the preparation of images, subjective test and post-processing to construct this database.

Preparation of Enhanced Underwater Images

We select 100 authentic underwater images from Google and related works [START_REF] Xiao | SUN database: Large-scale scene recognition from abbey to zoo[END_REF], [START_REF] Li | An Underwater Image Enhancement Benchmark Dataset and Beyond[END_REF]. These underwater images are taken from real underwater scenes, with resolutions ranged from 183 × 275 to 1350 × 1800. Typical underwater images are presented in Fig. 2. To generate more enhanced images for test, we utilize 10 representative UIE algorithms, including 3 non-physical-model-based methods (i.e. fusion-based [START_REF] Ancuti | Enhancing underwater images and videos by fusion[END_REF], retinex-based [START_REF] Fu | A retinex-based enhancing approach for single underwater image[END_REF], two-step-based [START_REF] Fu | Twostep approach for single underwater image enhancement[END_REF]), 4 physicalmodel-based methods (i.e. histogram prior [START_REF] Li | Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior[END_REF], UDCP [START_REF] Paulo | Underwater Depth Estimation and Image Restoration Based on Single Images[END_REF], UIBLA [START_REF] Peng | Underwater Image Restoration Based on Image Blurriness and Light Absorption[END_REF], GDCP [START_REF] Peng | Generalization of the Dark Channel Prior for Single Image Restoration[END_REF]), 2 deep-learning-based methods (i.e. UWCNN [START_REF] Li | Underwater Scene Prior Inspired Deep Underwater Image and Video Enhancement[END_REF], Water-Net [START_REF] Li | An Underwater Image Enhancement Benchmark Dataset and Beyond[END_REF] and 1 commercial application (i.e. dive+ [2]). With the 100 images and 10 UIE approaches, we have a total of 1,000 enhanced underwater images. All enhancement underwater images and the corresponding raw images are included in the UIED.

Subjective Test

We implement a subjective quality study to evaluate the 1,000 enhanced images. In the subjective quality evaluation, we adopt a single-stimulus strategy and a five-level quality scale to label images [START_REF]Methodology for the Subjective Assessment of the Quality of Television Pictures BT Series Broadcasting Service[END_REF]. A lower rating score represents a worse perceptual visual quality, and vice versa. A detailed description of the rating criteria is given in Table 1.

The subjective test consists of 10 sessions and in each session, 100 of the 1,000 images are evaluated. At the beginning of each session, a subject watches and evaluates 5 prescreen images to get familiar to testing environment and procedure. Then, the subject is asked to score the following 100 images in a random order. Each The target is invisible, and color severely distorts. 2

The target is invisible, and color partially distorts. 3

The target is visible, and color slightly distorts. 4

The target is visible, some flaws in visual quality. 5

The target is clearly, and visual quality is great.

image is shown for 7 seconds before next. The subject can take a 5-minute break between sessions to avoid visual fatigue.

There are totally 16 subjects including 9 males and 7 females in the subjective test. They all have prior knowledges of image processing. All test images are shown in random order with a MATLAB graphical user interface. The images are displayed in laboratory environment with normal illumination conditions. All of the above conditions are set and calibrated according to the recommendations of ITU-R [START_REF]Methodology for the Subjective Assessment of the Quality of Television Pictures BT Series Broadcasting Service[END_REF].

Data Post-Processing

To show the reliability of data, we choose to use Normalized Cross Correlation (NCC) and Euclidean distance (EUD) to evaluate the agreement of subject ratings [START_REF] Ma | Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics[END_REF]. A higher NCC value or a lower EUD indicates higher correlation between two subjective rating vectors. For the above 16 subjective ratings, the average value of NCC and EUD is 0.961 and 0.055, respectively. Therefore, the subject ratings are agreed on the perceptual qualities of these images and the testing results are reliable.

We then follow the steps in [START_REF] Min | Unified blind quality assessment of compressed natural, graphic, and screen content images[END_REF] to process the subjective ratings. Rating for an image is considered as outlier if it is outside 2 or √ 20 standard deviations of the mean rating of that image. A subject with more than 5% outlier evaluations is rejected. Both outlier ratings and outlier subjects are excluded from the following processing. The normalized ratings for an image are averaged over all valid subjects to the Mean Opinion Score (MOS). The MOS of the database is almost ranged from 20 to 80. That is, our collected subjective scores span a wide range from low to high scores.

Fig. 3 shows the MOS values corresponding to the enhancement results in Fig. 1. It can be seen that our subjective experiment rated has a high correlation to HVS. On the other hand, the popular UCIQE and UIQM show inferior performances that still have room to be further improved.

PROPOSED UIF METRIC

Based on the UIED, we are able to develop the UIF metric for a more accurate quantitative evaluation of enhanced underwater images. The design philosophy lies in three key aspects, including naturalness, sharpness, and structure of an enhanced underwater image. In this section, we present to attract features to characterize the naturalness, sharpness and structure, and further integrate them to infer the ultimate quality score. Details of our method are presented as follows.

Naturalness-Related Features

In most cases, the natural underwater images have low color richness due to light absorption and scattering. Thus, many UIE methods aim to eliminate the color attenuation from vision and improve color richness. However, overemphasis on color may also result in unreasonable color or an unnatural look. Therefore, we employ the NSS regularities to approximately capture the attributes of naturalness of underwater images. In addition, researches have shown a good correlation between underwater colors and human perception in CIELab space [START_REF] Yang | An Underwater Color Image Quality Evaluation Metric[END_REF]. Inspired by this, we also transfer the underwater images to CIELab space to calculate the naturalness-related features.

Distribution of NSS model. To detect naturalness-related features in enhanced underwater images, we first choose the NSS model which is frequently used in IQA tasks [START_REF] Li | No-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics[END_REF][START_REF] Yang | Blind image quality assessment on authentically distorted images with perceptual features[END_REF]. It is defined on the brightness of image

𝑓 (𝑥; 𝜈, 𝜎 2 ) = 𝜈 2𝛿 Γ( 1
𝜃 ) 𝑒 (-( |𝑥 | 𝜃 ) 𝜈 ) , (1)
where 𝛿 and Γ(•) are defined as:

𝜃 = 𝜎 √ Γ(1/𝜈) Γ(3/𝜈) , (2)
Γ(𝜃 ) = ∫ ∞ 0 𝑡 𝜃 -1 𝑒 -𝑡 𝑑𝑡 𝜃 > 0. (3)
The parameter 𝜈 and 𝜎 2 control the shape and variance of the distribution, respectively, which are collected to describe image naturalness.

Contrast and variance in CIELab. We transform the underwater image from RGB to CIELab, which is a uniform color space. Considering the luminance contrast is one of the most sensitive factors in poor illumination, we calculate the luminance contrast for enhanced images in turbid underwater environment. It is obtained by the ratio between highest and lowest luminance values

𝜎 CIE = 1 𝑁 𝑁 ∑ 𝑖=1 √ 𝛼 2 𝑖 + 𝛽 2 𝑖 𝐿 𝑖 , (4)
where 𝛼 and 𝛽 are the channel parameters of CIELab space, 𝐿 is value in luminance channel and 𝑁 presents the number of image pixels.

In summary, the naturalness-related features include

F naturalness = {𝜈, 𝜎 2 , 𝐶 CIE , 𝜎 (5)

Sharpness-Related Features

A high-fidelity image is usually with rich details that are characterized by sharpness of edges and pixels. In underwater imaging, forward scattering may affect image details and blurs edges. As a result, an important objective of UIE is to improve the sharpness of edges and thus present high-quality pictures with clear objects.

To characterize the sharpness of an enhanced image, we exploit the following features including Dark Channel Prior (DCP) index, contrast, edge contrast and entropy. DCP Index. The index was utilized to capture image contrast during haze removal [START_REF] He | Single Image Haze Removal Using Dark Channel Prior[END_REF]. In underwater environment with low light intensities, dark images are usually captured, which might hinder object recognition by humans or algorithms. Therefore, the UIE approaches are designed to enhance the lightness of underwater images, which changes the DCP channel pixels. Inspired by this, we compute the average value of DCP channel pixels as:

𝜇 dark = average{ min 𝑐 ∈𝑅,𝐺,𝐵 𝐼 𝑐 }, (6)
where 𝑐 ∈ 𝑅, 𝐺, 𝐵 indicates the RGB channels of enhanced image 𝐼 . Contrast. The contrast has been utilized in visual enhancement of underwater images [START_REF] Li | Underwater Image Enhancement by Dehazing With Minimum Information Loss and Histogram Distribution Prior[END_REF]. First of all, the enhanced image is divided into non-overlapped grayscale patches with size 64 × 64. Then, a patch is labeled as a textured patch if its edge density, i.e. the ratio between edge pixels and all pixels within the patch, is larger than 0.2%. Finally, the overall contrast is obtained as the sum of standard variances of all textured patches:

𝐶 = 𝑀 ∑ 𝑖=1 std_var(𝑃 𝑖,𝑗 ), (7)
where 𝑃 𝑖,𝑗 is the 𝑖, 𝑗-th patch of image and 𝑀 represents the number of textured patches.

Edge contrast. The edge information has been widely utilized to assess fidelities of images [START_REF] Li | Quality Assessment of DIBR-Synthesized Images by Measuring Local Geometric Distortions and Global Sharpness[END_REF][START_REF] Zhan | No-Reference Image Sharpness Assessment Based on Maximum Gradient and Variability of Gradients[END_REF]. In this work, we utilize a simple but efficient edge extraction for ease of calculation. First of all, the Canny edge detector is applied to all channels of image to obtain three edge maps. Then, each edge map is equally divided into 𝑚 × 𝑛 non-overlapped blocks with size 5×5. Finally, the contrasts of all blocks are calculated and averaged to obtain a final measure within edge maps [START_REF] Panetta | Choosing the Optimal Spatial Domain Measure of Enhancement for Mammogram Images[END_REF]:

𝐶 edge = 3 ∑ 𝑐=1 𝜆 𝑐 • 2 𝑚𝑛 𝑚 ∑ 𝑖=1 𝑛 ∑ 𝑗=1 log( max𝐵 𝑐,𝑖,𝑗 min𝐵 𝑐,𝑖,𝑗 ), (8)
where 𝐵 𝑐,𝑖,𝑗 represents the 𝑖, 𝑗-th edge block in channel 𝑐, and max and min are to calculate the extreme values within the edge block.

𝜆 𝑐 is a coefficient for RGB color channels.

Entropy. As a classic image measurement, the entropy characterizes diversity of image pixels. Therefore, it has a high correlation to image contrast. For the enhanced image with 𝐿 brightness levels, the entropy is defined by

𝐸 = - 𝐿-1 ∑ 𝑖=0 𝑝 𝑖 log 𝑝 𝑖 , (9)
where 𝑝 𝑖 is the histogram probability of brightness value 𝑖.

In summary, the sharpness-related features include

𝐹 sharpness = {𝜇 dark , 𝐶, 𝐶 edge , 𝐸}. ( 10
) (a) (b)
• • ••, • • , . . . , • • , • • • •, • • • ••• • ••• • •••• C • •••• • • • • • s I Figure 6:
The detailed framework of the proposed UIF method.

Structure-Related Features

The structural similarity between original and enhanced images is critical because the UIE algorithms, especially the deep-learningbased UIE, may introduce structural artifacts during the enhancement process. An example is shown in Fig. 4, where unwanted textures are introduced due to color changes. These artifacts may severely degrade visual quality of enhanced images. Therefore, we need to measure the structural similarity between original and enhanced images. It is noted that the image structural similarity was firstly introduced by SSIM. To evaluate the similarity between an image and its distorted version. Inspired by this index, we calculate our structure-related features with brightness values of original and enhanced underwater images.

Variance similarity. In most cases, over enhancement of underwater images will cause large variance of brightness, with an example shown in Fig. 5 (b). In such case, we can construct a variance similarity measure to identify these over-enhanced regions. Similar to SSIM, we define the variance similarity as:

𝑠 𝜎 = 2𝜎 𝐼 𝜎 0 + 𝑐 1 𝜎 2 𝐼 + 𝜎 2 0 + 𝑐 1 , (11)
where 𝜎 𝐼 and 𝜎 0 are the variances of enhanced and original images, respectively. They are calculated locally with a 7×7 window. 𝑐 1 is a small constant to avoid zero denominators. With this measure, we can calculate the similarity between Fig. 5 (a) and (b) to obtain the variance map shown in Fig. 5 (c). It can be readily seen that the over-enhanced regions with large brightness changes are labeled with low variance similarities.

Mean and normalized similarities. The local mean 𝜇 and normalized image Ī can describe the perceptual similarity of textured regions [START_REF] Gu | A Fast Reliable Image Quality Predictor by Fusing Micro-and Macro-Structures[END_REF], where the normalized image is calculated by (𝐼 -𝜇 (𝐼 ))/𝜎 (𝐼 ). Therefore, we also utilize the similarities between mean and normalized images as supplementary indexes:

𝑠 𝜇 = 2𝜇 𝐼 𝜇 0 + 𝑐 2 𝜇 2 𝐼 + 𝜇 2 0 + 𝑐 2 , (12)
𝑠Ī = 2 Ī𝐼 Ī0 + 𝑐 3 Ī 2 𝐼 + Ī 2 0 + 𝑐 3 , (13)
where 𝜇 𝐼 and 𝜇 0 respectively denote the local mean values of enhanced and original images, Ī𝐼 and Ī0 respectively denote the normalizations of enhanced and original images, 𝑐 2 and 𝑐 3 are small constants to avoid zero denominators. In summary, the structure-related features include

𝐹 structure = {𝑠 𝜎 , 𝑠 𝜇 , 𝑠 𝐼 }. (14)

Feature Pooling and Regression

Fig. 6 shows the detailed framework of proposed UIF metric, which mainly consists of two modules: feature extraction and feature regression. In feature extraction module, we obtain the naturalnessrelated, sharpness-related and structure-related features, as discussed above. If a feature is represented by a two-dimensional map, an average pooling is employed to reduce the dimensions. All features are then linked into a feature vector:

𝐹 = [𝜈, 𝜎 2 , 𝐶 CIE , ..., 𝑠 𝜎 , 𝑠 𝜇 , 𝑠Ī ]. (15)
In feature regression module, we select the SVR for fusion considering its success in regression tasks. As illustrated in Fig. 6, we use labeled underwater enhanced image pairs to train the regressor, which can be utilized to predict the quality of any input image. Given the features 𝐹 = [𝜈, 𝜎 2 , 𝐶 CIE , ..., 𝑠 𝜎 , 𝑠 𝜇 , 𝑠Ī ], the corresponding quality label MOS values 𝑆 𝑖 and the training image set, we can train the regressor by SVR:

regressor = TRAIN(𝐹 𝑖 , 𝑆 𝑖 ). (16)
After the training process, we can apply this regressor to yield quality scores of any testing enhanced image 𝐼 and original image

𝐼 0 : 𝑞 = PREDICT(𝐼, 𝐼 0 , regressor). (17)
LIBSVM [START_REF] Chang | LIBSVM: A library for support vector machines[END_REF] is adopted to implement SVR and has previously been applied to IQA problems [START_REF] Fang | No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics[END_REF]. In training process, we choosea Radial Basis Function (RBF) kernel. The other SVR parameters are set as: penalty coefficient=0.1, 𝜀-insensitive loss function=0.01, and RBF kernel parameters=1.

EXPERIMENTAL RESULTS

In this section, we will evaluate proposed UIF metric with experimental results. Popular IQA metrics, including general no-reference metrics, enhanced image quality metrics and UIE quality metrics are examined for comparison. We also present ablation study to show the effectiveness of all types of features.

Experiment Settings

To demonstrate the efficiency of our method, we choose a variety of IQA methods for comparison. They include 9 no-reference IQA methods for natural images (BRISQUE [START_REF] Mittal | No-Reference Image Quality Assessment in the Spatial Domain[END_REF], NIQE [START_REF] Mittal | Making a "Completely Blind" Image Quality Analyzer[END_REF], NFERM [START_REF] Gu | Using Free Energy Principle For Blind Image Quality Assessment[END_REF], IL-NIQE [START_REF] Zhang | A Feature-Enriched Completely Blind Image Quality Evaluator[END_REF], SISBLIM [START_REF] Gu | Hybrid No-Reference Quality Metric for Singly and Multiply Distorted Images[END_REF], BLIINDS-II [START_REF] Xue | Blind image quality assessment using joint statistics of gradient magnitude and laplacian features[END_REF], dipIq [START_REF] Ma | dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs[END_REF], og-iqa [START_REF] Liu | Blind image quality assessment by relative gradient statistics and adaboosting neural network[END_REF], CNN-IQA [START_REF] Kang | Convolutional Neural Networks for No-Reference Image Quality Assessment[END_REF]), 3 IQA methods for enhanced images (CPCQI [START_REF] Gu | Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data[END_REF], BIQME [START_REF] Gu | Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data[END_REF], and NR-CDIQA [START_REF] Fang | No-Reference Quality Assessment of Contrast-Distorted Images Based on Natural Scene Statistics[END_REF]) and 2 IQA methods for enhanced underwater images (UCIQE [START_REF] Yang | An Underwater Color Image Quality Evaluation Metric[END_REF] and UIQM [START_REF] Panetta | Human-Visual-System-Inspired Underwater Image Quality Measures[END_REF]). For fair comparison, we use the publicly available codes provided by authors. If a metric is obtained by machine learning and its training code is available, its parameters are further tuned for fair comparison. To examine the generalization ability of machine learning models including our UIF metric, they are examined with 𝑘-fold cross-validations, where 𝑘 is set as 4. A fold of data is examined and recorded only when it is used as testing test.

The performances of all IQA metrics are evaluated by two commonly used consistency criteria, including Spearman Rankorder Correlation Coefficient (SRCC) and Pearson Linear Correlation Coefficient (PLCC). The SRCC scores the prediction monotonousness to MOS values, while the PLCC scores the linear correlation between the IQA model's predictions and MOS values. In particular, higher PLCC or SRCC indicates an IQA metric is more consistent with subjective quality evaluations.

Comparisons and Discussions

The comparison results of all IQA metrics are summarized in Table 3, where the best results and 2nd-best results are highlighted with red bold and blue bold, respectively. As shown in the table, some no-reference IQA methods are designed for images in the air (e.g. BRISQUE, NFERM, NIQE, IL-NIQE, OG-IQA, SISBLIM, and BLIINDS-II), thus their extracted NSS characteristics are not applicable to all underwater images. The accuracy of these no-reference IQA metrics is low. Even for learning-based metrics such as dipIQ, CNN IQA, the performance in terms of SRCC and PLCC are not high. This fact implies the particularity and complexity of underwater image characteristics, which makes the IQA for UIE images a more challenging problem.

The methods designed for enhanced images (e.g. CPCQI, BIQME, and CDIQA) also show inferior performances in UIE images. This may be because they are designed to prefer images with high contrasts without consideration to the special distortions and artifacts in underwater images. As a result, the features extracted by these methods are not competitive and the experimental results are not ideal.

Besides, the performances of popular UCIQE and UIQM are still low in UIED. The possible reasons are as follows. Firstly, these methods were proposed earlier when few UIE algorithms were developed. The recent booming of UIE approaches have led to a variety of image distortions and artifacts, which cannot be considered by early methods. Secondly, they were tuned in small datasets due to lack of large-scale dataset, which could not be constructed due to lack of UIE algorithms. Thirdly, they incorporated fewer features to evaluate underwater images, which limits their performance in diverse underwater scenarios.

Finally, the proposed UIF metric achieves superior performance to other metrics. This demonstrates the effectiveness of multiple features and SVR-based fusion. More importantly, our metric is also benefited from the large-scale UIED, which is the largest-ever underwater image quality database with subjective scores.

To present an intuitive comparison between UIF and conventional metrics, we select four typical underwater images and their enhancements, as shown in Fig. 7. We also calculate the UCIQE and UIF values of all enhanced images and present them in the same figure. From this figure, the conventional UCIQE metric may focus on chroma components of images and thus tend to prefer images with rich colors. It gives high scores to the (b) (f) (l) (p), which have either red artifacts or contrast distortions. In contrast, the UIF metric has preferences for (c) (h) (j) (n). This is because the UIF considers naturalness, contrast and similarity besides of underwater image colors, which are more consistent with HVS. Therefore, our UIF metric shows the state-of-the-art performance in quality evaluation of enhanced underwater images.

Ablation Study

To evaluate the contribution of each type of features, we conduct a series of ablation experiments. Specifically, the image features merged in the following methods.

• Method1: Only naturalness-related features.

• Method2: Only sharpness-related features.

• Method3: Only structure-related features.

• Method4: Naturalness-related and sharpness related features.

• Method5: Sharpness-related and structure-related features.

• Method6: Naturalness-related and structure-related features.

• Method7: The UIF with all three types of features.

Table 4 shows the performances of all 7 methods. From the table, we can draw several conclusions. Firstly, the model achieves acceptable performances even with one type of feature. Compared with Table 3, the corresponding performance is still superior to other IQA models in underwater IQA. Secondly, with more types of features, the proposed model generally improves its prediction performance in terms of SRCC and PLCC. Thirdly, by a fusion of all types of features, our UIF metric achieves the best performance of all. This fact demonstrates the effects of all types of features in our UIF metric. With an SVR-based regression, the fused index achieves superior performance to predict subjective evaluations with an objective approach. Naturalness and structure 0.701 0.727 Method 7

All type features 0.733 0.757

Further Applications

The proposed UIF metric can be further applied in other IQA tasks of enhanced images, with acceptable correlations to subjective scores.

In this section, we test the UIF metric in three other databases, including DHQ [START_REF] Yeganeh | Objective Quality Assessment of Interpolated Natural Images[END_REF] for dehazing quality assessment, CID2013 [START_REF] Gu | Subjective and objective quality assessment for images with contrast change[END_REF] and CCID2014 [START_REF] Gu | The Analysis of Image Contrast: From Quality Assessment to Automatic Enhancement[END_REF] for contrast-enhanced quality assessments. For comparison, we keep the same SVR parameter settings and training steps. The training to testing ratio is set as 80:20. In Table 5, we present average SRCC and PLCC values for the three databases.

Although the existing IQA methods for general enhanced images cannot work well in underwater IQA, as shown in Table 3, our UIF metric can still achieve acceptable performances in for IQA of general enhanced images. This also shows the generalization of our UIF metric to a certain extent.

CONCLUSIONS

Recent efforts of UIE have greatly promoted the visual quality of underwater images. Nevertheless, the quality measures of UIE have not been updated to evaluate the enhanced images by newly developed UIE models. In this paper, we make the first attempt to build a large-scale underwater image quality database for UIE, namely UIED, which can be utilized to train and evaluate objective IQA approaches for enhanced underwater images. Based on this database, we also propose a new quality metric of underwater images, namely UIF, which shows the state-of-the-art performance in this scenario. Experimental results also demonstrate the effectiveness of features and the generalization ability of UIF. The proposed metric can evaluate the qualities of enhanced underwater images and also help to select optimal UIE approaches under different underwater environments.

Figure 1 :

 1 Figure 1: Typical underwater images (the first left column) and their scored enhancements (the other three columns), where the scores of each enhanced image are respectively given by UCIQE and UIQM. How to compare the visual quality of enhanced underwater images is still a challenging task.

Figure 2 :

 2 Figure 2: Typical underwater images in UIED, including 16 coral images, 26 marine life images, 14 seabed rock images, 12 sculpture images, 10 wreck images and 22 diver images.

  (a) Original image (b) MOS:23.94 (c) MOS:42.89 (d) MOS:52.76 (e) Original image (f) MOS:28.80 (g) MOS:37.61 (h) MOS:56.40 (i) Original image (j) MOS:33.08 (k) MOS:22.42 (l) MOS:58.77

Figure 3 :

 3 Figure 3: The MOS values of images in Fig. 1.

Figure 4 :

 4 Figure 4: An example of structural artifact introduced by UIE.

Figure 5 :

 5 Figure 5: An example of over-enhanced underwater images and variance similarity map. (a) original image, (b) over-enhanced image, (c) variance similarity map.

Figure 7 :

 7 Figure 7: Typical underwater images (the first left column) and their scored enhancements (the other three columns), where the scores of each enhanced image are respectively given by UCIQE and proposed UIF.

Table 1 :

 1 Rating Criteria for Subjective Test

	Level	Description
	1

  Table 2 lists an overview of the test methodology and conditions.

Table 2 :

 2 Detailed Setting of Subjective Experiment

	Category	Item	Detail
		Monitor	AOC 24n2h LCD
	Display	Resolution	1920 × 1080
		Platform	Matlab R2016b
		Method	Single-stimulus
	Methodology	Quality scale	5-level categorical
		Order	Random
		Groups	10
	Test settings	Subjects number 9 males / 7 females Time interval 7 seconds
		Environment	Laboratory

Table 3 :

 3 Performance Comparison of Selected IQA metrics

	Type	Methods	SRCC	PLCC
		BRISQUE	0.465	0.496
		NFERM	0.355	0.339
		NIQE	0.326	0.344
		IL-NIQE	0.393	0.347
	General IQA	OG-IQA	0.216	0.233
		SISBLIM	0.277	0.321
		BLIINDS-II	0.341	0.352
		dipIQ	0.126	0.217
		CNN IQA	0.027	0.081
		CPCQI	0.266	0.284
	IQA for enhancement	BIQME	0.205	0.259
		CDIQA	0.276	0.292
		UCIQE	0.252	0.298
	IQA for UIE	UIQM	0.276	0.268
		UIF	0.733	0.757

Table 4 :

 4 Performance of Different Feature Groups

	Feature Groups	Descriptions	SRCC PLCC
	Method 1	Only naturalness	0.663 0.689
	Method 2	Only sharpness	0.588 0.605
	Method 3	Only structure	0.618 0.645
	Method 4	Naturalness and sharpness 0.676 0.691
	Method 5	Sharpness and structure	0.707 0.713
	Method 6

Table 5 :

 5 Evaluations on Other IQA Tasks

	Databases	Type	SRCC PLCC
	CID2013	Contrast-changed Image 0.6959 0.6795
	CCID2014 Contrast-changed Image 0.7167 0.7031
	DHQ	Dehazing Image	0.6234 0.6707

ACM Multimedia '29, October 20-24, 2021" Chengdu, China Lin, et al.

ACKNOWLEDGMENTS


Dragana D Sandić-Stanković

Dragan D Kukolj
email: dragan.kukolj@rt-rk.uns.ac.rs

Patrick Le Callet
email: patrick.lecallet@univ-nantes.fr

Morphological Difference of Closings Operator for No-Reference Quality Evaluation of DIBR-Synthesized Images

Keywords: No-reference image quality assessment, structural image distortion, morphological multi-resolution multi-scale image representation, sparse representation, granulometry, difference of closings operator

Images synthesized using Depth-Image-Based Rendering (DIBR) techniques are characterized by complex structural distortion. Multi-resolution multi-scale sparse image representation generated using morphological Difference of Closings operator (DoC) is used to efficiently capture structure-related distortion of synthesized images in the noreference DoC-GRNN image quality assessment model. Nonlinear morphological Difference of Closings operator (DoC) with an array of line-shaped structuring elements of increasing length is used to extract perceptually important details of object structure at different scales and resolutions. The sparsity of DoC band is calculated as scalar feature. The extracted features are mapped to the quality score by general regression neural network (GRNN). We have explored the influence of the direction of an array of line-shaped structuring elements on the model's performances. The DoC-GRNN model shows high agreement with perceptual quality scores, comparable to the state-of-the-art metrics, when evaluated on the stereoscopic DIBR-synthesized images of MCL-3D dataset.

I. INTRODUCTION

View synthesis using depth-image-based rendering (DIBR) algorithms is important for immersive imaging technologies such as multi-view video and free-viewpoint video with application in various fields: medicine, education, remote surveillance and entertainment. Since only limited number of camera is used to capture different views, new video sequences have to be synthesized at the viewpoints, where captured video sequences are missing from captured color video and depth maps using DIBR algorithm. Since an area around object edges which are invisible from the reference view become visible from the position of the synthesized view, black holes appear in the synthesized images [START_REF] Bosc | Towards a New Quality Metric for 3-D Synthesized View Assessment[END_REF]. Although inpainting and interpolation techniques are used for hole filling, a texture distortion may appear. Geometric distortions occur locally at object contours. Distortions in the synthesized images may also appear due to distortions of color images and depth maps, which may occur during the process of acquisition, compression and transmission.

The image quality assessment (IQA) of the synthesized views is of great importance for the development of 3D video applications and for the development of the view synthesis algorithms. In multi-view video systems, reference views are not available for all viewpoints. Therefore, referenceless quality metrics are highly desirable.

It is believed that the human visual system (HVS) is complex nonlinear system which separate images into a set of "spatial frequency" channels as a first encoding of visual information. Sparse image representations that capture the salient structures in line with the human perception are reported to be a processing strategy of the nervous system [START_REF] Olshausen | Sparse coding of sensory inputs[END_REF]. In order to mimic the hierarchical structure of HVS, multi-resolution and multi-scale image representations have been used in image quality assessment models. Multiresolution and multi-scale image representations have also been used in image quality assessment models designed for DIBR-synthesized images: morphological pyramid of Laplacian type [START_REF] Sandić-Stanković | DIBR synthesized image quality assessment based on morphological multiscale approach[END_REF][START_REF] Sandić-Stanković | DIBR synthesized image quality assessment based on morphological pyramids[END_REF][START_REF] Sandić-Stanković | Multi-scale synthesized view assessment based on morphological pyramid[END_REF], Gaussian type pyramid [START_REF] Zhou | Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity[END_REF], multiresolution representation generated using only downsampling [START_REF] Gu | Using multiscale analysis for blind quality assessment of DIBR-synthesized images[END_REF], [START_REF] Gu | Multiscale Natural Scene Statistical Analysis for No-Reference Quality Evaluation of DIBR-Synthesized Views[END_REF], morphological wavelets [START_REF] Sandić-Stanković | DIBR synthesized image quality assessment based on morphological wavelets[END_REF] and linear wavelets [START_REF] Bosc | A wavelet-based image quality metric for the assessment of 3D synthesized views[END_REF], [START_REF] Wang | Blind Quality Metric of DIBR-Synthesized Images in the Discrete Wavelet Trasform Domain[END_REF]. The sparsity of morphological pyramid and wavelet bands have been used in the image quality model based on general regression neural network [START_REF] Bokan | Blind DIBRsynthesized image quality assessment based on sparsity features in morphological multiscale domain[END_REF].

The scale-space framework developed to handle the multi-scale nature of image data has been used in IQA models designed for DIBR-synthesized images. The quality of the whole view synthesis process was estimated based on the array of low-pass smoothed images from Gaussian scale space [START_REF] Zhou | Qauality assessment for view synthesis using low-level and mid-level structural representation[END_REF]. The array of band-pass detail images generated by the Difference of Gaussian operator (DoG) of Gaussian scale space, has been used to capture low-order structure (edges) in IQA models [START_REF] Sandić-Stanković | Blind DIBR-synthesized Image Quality Assessment using multi-scale DoG and GRNN[END_REF], [START_REF] Zhou | No-Reference Quality Assessment for View Synthesis Using DoG-based Edge Statistics and Texture Naturalness[END_REF]. Morphological Difference of Closings (DoC) operator has been used to extract higherorder structures (fine grained details, texture) of high curvature, which corresponds to the distortion of shapes to which the HVS is highly sensitive in the combined DoC-DoG-GRNN model [START_REF] Sandić-Stanković | Quality Assessment of DIBR-synthesized views based on Sparsity of Difference of Closings and Difference of Gaussians[END_REF]. DoC operator nonlinearly removes redundancies and extracts local structure and contrast to which HVS is highly sensitive. An array of line-shaped structuring elements of increasing sizes has been used by morphological close filter to create a set of simplified signals at different scales. In this work, we explore the influence of the direction of an array of the line-shaped structuring elements used in the generation of DoC bands on the DoC-GRNN model's performances. DoC-GRNN model can be successfully used to evaluate complex distortions of synthesized images which appear due to the rendering process and the compromised quality of color images and depth maps.

In the next section, the proposed model is described in more details. The experimental setup and the performances of the proposed model are presented in Section three.

II. IMAGE QUALITY MODEL

The framework of the DoC-GRNN model is shown on Fig. 1. In the first stage, multi-resolution and multi-scale image representation using morphological Difference of Closings (DoC) operator is generated to mimic the hierarchical property of human visual system. DoC operator nonlinearly removes redundancies and extracts local structure and contrast to which HVS is highly sensitive. DoC operator extracts structures (fine grained details, texture) of high curvature, which corresponds to the distortion of shapes to which the HVS is highly sensitive. In the second stage, Hoyer index [START_REF] Hoyer | Non-negative Matrix Factorization with Sparseness Constraints[END_REF] is used to measure the sparsity of DoC band. The extracted features and subjective scores are used to train the general regression neural network (GRNN) which is particularly advantageous with sparse data in real-time environment [START_REF] Specht | A general regression neural network[END_REF]. In the third stage, the trained network GRNN is used to predict the quality score.

)

) ) v y- u, (x- f ( max ) , ( , SE v u y x SE f Î = Å (1) ⊝ , = ) v) y u, (x (f min , + + ÎSE v u (2)
The morphological filter closing, SE f • , which locally modifies the geometric signal features, iteratively applies dilation and erosion [START_REF] Sandić-Stanković | DIBR synthesized image quality assessment based on morphological multiscale approach[END_REF]. Closing filter produces a smooth signal approximation with details removed. Visually, the closing filter smooths contours of image dark regions, tends to fuse narrow breaks, eliminates small holes and fills gaps in the contours. Morphological filter is useful for visual perception since it removes valleys from the signal, which correspond to the critical points of contour.

= • SE f ( ⊕ ) ⊖ (3)

A. Multi-resolution multi-scale image decomposition using morphological DoC operator

Image decomposition in the first stage of the framework can be explained using granulometry, a tool for the computation of size distribution in images [START_REF] Maragos | Pattern Spectrum and Multiscale Shape Representation[END_REF] = , constitutes granulometry by closing. A fast algorithm can be used for efficient computation of linear granulometries [START_REF] Vincent | Fast grayscale granulometry algorithms[END_REF].

j i j i SE s s • = 0 , , , , , ... , 1 N j = M i , ... , 1 = (4)
length of the structuring element determines the scale j. The structuring element at scale j, j SE is created using the dilation of the structuring element at the first scale, 1 SE , with itself 1 j times (5).

4 4 3 4 4 2 1 times j j SE SE SE SE 1 1 1 1 . . . - Å Å = , N j £ £ 2 (5)
We have explored the morphological filtering using an array of linear structuring elements with increasing length in three directions: 0, 45, 90 degrees. Structuring elements at scale 1, 1 SE , in three directions are shown on Fig. 2. , the initial image from the current resolution level, 0 , i s , is filtered using morphological filter closing with the structuring element R and downsampled by a factor of two ( s ) by rows and by columns [START_REF] Zhou | Reduced-reference quality assessment of DIBR-synthesized images based on multi-scale edge intensity similarity[END_REF]. In this work, we've used square 2x2 as R SE .

f s = 0 , 1 ) ( 0 , 0 , 1 R i i SE s s • = + s M i , ... , 1 = (6)
The Difference of Closing (DoC) operator has been defined as the difference between morphologically filtered images of adjacent scales using morphological filter closing [START_REF] Gu | Using multiscale analysis for blind quality assessment of DIBR-synthesized images[END_REF] [START_REF] Sandić-Stanković | Quality Assessment of DIBR-synthesized views based on Sparsity of Difference of Closings and Difference of Gaussians[END_REF]. The generation of non-negative integer valued DoC band is shown on Fig. 3.

DoC: 1 , , , - - = j i j i j i s s d , N j , ... , 1 = (7)
The DoC bands, of DoC-based image representation has low computational complexity.

Fig. 3. DoC band is created as the difference between closings of nearby scales; 0 , i s is the input image at resolution level i, j i s , is the low-pass copy and j i d , is the detail image (DoC band) at resolution level i and scale j.

The DoC bands at resolution level 4 and scale 5, created using line-shaped structuring elements in three directions 0, 45, 90 degrees from the image Kendo synthesized from the content with JPEG2000 distortion are shown on Fig. 4. The oriented lines extracted in DoC bands correspond to perceptually important building blocks of object structures.

III. EXPERIMENTAL RESULTS

The proposed metric is evaluated using publicly available dataset of stereoscopic DIBR-synthesized images MCL-3D [START_REF] Song | MCL-3D: A database for stereoscopic image quality assessment using 2D-image-plus-depth source[END_REF]. The MCL-3D dataset contains 684 synthesized stereoscopic image pairs synthesized from nine image-plusdepth sources and associated mean opinion score (MOS) values. One third of images are of resolution 1024×728 and two thirds are of resolution 1920×1080. The majority of images (648 stereo images) are synthesized using both left and right views distorted by one of six distortion types: Gaussian blur, additive white noise, down-sampling blur, JPEG and JPEG-2000 compression and transmission error. Distortions have been symmetrically applied to the left and right viewpoints to either the color images or the depth maps or both the color images and the depth maps at four distortion levels before stereoscopic image rendering using the View Synthesis Reference Software [START_REF] Tanimoto | View synthesis algorithm in view synthesis reference software 3[END_REF]. The minority of the images (36 stereo images) have been synthesized from the undistorted color images and depth maps of single view using four DIBR synthesis algorithms. They contain only the distortions caused by imperfect rendering.

In order to evaluate the performances of the proposed metric and other metrics, three evaluation criteria were used: Root Mean Squared Error (RMSE) to compute the prediction error, Pearson Linear Correlation Coefficient (PLCC) to compute prediction accuracy and Spearman Rank-order Correlation Coefficient (SROCC) for prediction monotonicity.

In the evaluation of DoC-GRNN model, the k-fold cross validation strategy is to split the dataset with D images to k=5 disjoint test subsets of similar size, each with D/k images (20% images of the dataset), and to select the train subsets with D-D/k images (80% images of the dataset) such that there not overlap between the test and the train subsets. At each iteration of the k-fold cross validation strategy, GRNN is used to predict the scores of the test subset. The median value of the performances through crossvalidation iterations and through 1000 repetitions is calculated as the final model's performances.

We have explored model's performances using three arrays of line-shaped structuring elements in three directions: 0, 45, 90 degrees, in the multi-resolution and multi-scale DoC-based decomposition. When the array of horizontal structuring elements (0 degrees) is used, the best performances are achieved using M=7 resolution levels and N=5 scales in the MR-MS-DoC decomposition and 36 are extracted. In the case when the array of structuring elements with direction 45 degrees is used, the best performances are achieved using M=7 resolution levels and N=6 scales, leading to 43 features. When the array of vertical structuring elements is used, the best performances are achieved using M=4 resolution levels and N=6 scales leading to 25 features. The only parameter of GRNN is the spread, that represents the width of radial basis function of GRNN. The PLCC of the DoC-GRNN scores using three arrays of structuring elements in different directions, 0, 45, 90, with fluctuation of the GRNN spread parameter is shown on Fig. 5. The best performances of DoC-GRNN model using three arrays of structuring elements in three directions are shown in Table I. The results show that the performance dependency from the direction of line-shaped structuring elements is small. The lowest number of features, 25, is needed when the array of vertical structuring elements is used and the performances are slightly better in that case. The performances are slightly worse when the array of structuring elements with orientation 45 degrees is used.

The performances of the proposed model are compared to the no-reference DoG-GRNN metric proposed for DIBRsynthesized images [START_REF] Sandić-Stanković | Blind DIBR-synthesized Image Quality Assessment using multi-scale DoG and GRNN[END_REF] and to 2D IQA metrics for natural images which use reference: PSNR, SSIM, MS-SSIM, IW-SSIM, Table II. The DoC-GRNN metric shows reliable agreement with human scores, PLCC=0.914, SROCC=0.909. High performances are achieved mostly thanks to the capacity of morphological series to capture higher order properties of spatial random processes. DoG-GRNN model, using 31 features calculated from DoG-based image representation in 5 resolution levels and 6 scales, achieves similar performances as the DoC-GRNN model. Among the metrics proposed for natural images, IW-SSIM achieves highest performances in the quality evaluation of MCL-3D dataset. Several more reference-based metrics designed for the evaluation of natural images can be successfully used to assess the quality of images from MCL-3D dataset [START_REF] Sandić-Stanković | Quality Assessment of DIBR-synthesized views based on Sparsity of Difference of Closings and Difference of Gaussians[END_REF]. Reference-based metrics, designed to predict the quality of DIBR-synthesized images show limited performances in the evaluation of MCL-3D dataset [START_REF] Sandić-Stanković | Quality Assessment of DIBR-synthesized views based on Sparsity of Difference of Closings and Difference of Gaussians[END_REF]. No-reference machine learning-based techniques for the quality evaluation of DIBR-synthesized images achieve the best performances in the evaluation of MCL-3D dataset [START_REF] Sandić-Stanković | Quality Assessment of DIBR-synthesized views based on Sparsity of Difference of Closings and Difference of Gaussians[END_REF].

IV. CONCLUSION

No-reference training-based DoC-GRNN model uses sparsity of Difference of Closings (DoC) bands as features. DoC bands which contain perceptually important structurerelated information at different scales and resolutions are generated by morphological filtering using an array of lineshaped structuring elements. We have analysed the performances of DoC-GRNN model using an arrays of lineshaped structuring elements in three directions: horizontal, 45 degrees, vertical. The performances are slightly higher using vertical line-shaped structuring elements and the number is the lowest in that case. The DoC-GRNN model achieves high performance in the evaluation of DIBR-synthesized images with rendering distortions and the distortion which appear during acquisition, compression and transmission of color images and depth maps. The code for feature extraction is available at https://sites.google.com/site/draganasandicstankovic/code/d oc.

Fig. 1 .

 1 Fig. 1. The framework of the DoC-GRNN model First, we review the morphological operators used in the calculation of DoC bands. Basic morphological operators, dilation, SE f Å, and erosion, ⊝ E , are calculated using operators maximum and minimum on the area of a grayscale image f, defined by the flat structuring element SE[START_REF] Haralick | Image Analysis using Mathematical Morphology[END_REF]:

Fig. 2 . 1 SE 1 s

 211 Fig. 2. The structuring element of length 2 at scale 1, 1 SE , in three directions: 0, 45, 90. At the first resolution level, the input image 0 , 1 s is the under test To create the initial image for the next resolution level, 0 , 1 + i s

  at resolution levels i=1, … M, and scales j=1, …, N constitute multi-resolution and multi-scale image representation (MR-MS-DoC) used in the DoC-GRNN model. Since the morphological operators involve only integers and only max, min and addition, the calculation

Fig. 4 .

 4 Fig. 4. Image Kendo synthesized from the content with JPEG2000 distortion (top); DoC bands at resolution level 4 and scale 5 created using line-shaped structuring elements in three directions: horizontal (bottom left), 45 degrees (bottom middle), vertical (bottom right).

Fig. 5 .

 5 Fig. 5. PLCC of the DoC-GRNN scores for different directions of an array of line-shaped structuring elements used in the DoC-based decomposition.

TABLE I .

 I PERFORMANCE OF THE DOC-GRNN MODEL USING THREE ARRAYS OF LINE-SHAPED STRUCTURING ELEMENTS IN THREE DIRECTIONS

	direction	num.	spread	PLCC	SROCC	RMSE
	(degrees)	features
	0	36	0.016	0.9084	0.9092	1.1222
	45	43	0.008	0.8943	0.8966	1.2055
	90	25	0.004	0.9147	0.9091	1.0750

TABLE II

 II

	.	PERFORMANCES OF THE METRIC AND OTHER
			METRICS
		Metric	PLCC	SROCC	RMSE
	NR	DoC-GRNN	0.9147	0.9091	1.0750
		DoG-GRNN	0.9041	0.8988	1.1397
		PSNR	0.7779	0.7900	1.6397
	FR	SSIM	0.7539	0.7741	1.7144
		MS-SSIM	0.8386	0.8518	1.4213
		IW-SSIM	0.8694	0.8878	1.2894