M Tien

Thuong Le

Keywords: Image denoising, image super-resolution, Bayesian Maximum A Posteriori, Gaussian Mixture Model, Earth Mover's Distance, Sparse models, patches distribution, example-based image restoration . . . . . . . Image Processing

Les travaux présentés dans cette thèse concernent les approches bayésiennes par patchs des problèmes d'amélioration de la qualité d'images. Notre contribution réside en le choix du dictionnaire construit grâce à un ensemble d'images de haute qualité et en la définition et l'utilisation d'un modèle à priori pour la distribution des patchs dans l'espace du dictionnaire. Nous avons montré qu'un choix attentif du dictionnaire représentant les informations locales des images permettait une amélioration de la qualité des images dégradées. Plus précisément, d'un dictionnaire construit de façon exhaustive sur les images de haute qualité nous avons sélectionné, pour chaque patch de l'image dégradée, un sous dictionnaire fait de ses voisins les plus proches. La similarité entre les patchs a été mesurée grâce à l'utilisation de la distance du cantonnier (Earth Mover's Distance) entre les distributions des intensités de ces patchs. L'algorithme de super résolution présenté a conduit à de meilleurs résultats que les algorithmes les plus connus. Pour les problèmes de débruitage d'images nous nous sommes intéressés à la distribution à priori des patchs dans l'espace du dictionnaire afin de l'utiliser comme pré requis pour régulariser le problème d'optimisation donné par le Maximum à Postériori. Dans le cas d'un dictionnaire de petite dimension, nous avons proposé une distribution constante par morceaux. Pour les dictionnaires de grande dimension, la distribution à priori a été recherchée comme un mélange de gaussiennes (GMM). Nous avons finalement justifié le nombre de gaussiennes utiles pour une bonne reconstruction apportant ainsi un nouvel éclairage sur l'utilisation des GMM.

First and foremost, I would like to express my deepest gratitude to my doctoral advisors: Professor Françoise Dibos, Assoc. Prof. Marie Luong and Assoc. Prof. Sébastien Li-Thiao-Té for their guidance and support during my thesis. I am thankful to them for listening patiently all my questions, sharing their immense knowledge to me and giving me a lot of outstanding advice to overcome scientific obstacles as well as to improve my research competence.

Next, I am truly grateful to Professor Thuong Le-Tien for his encouragement and valuable advice, not only in the scientific activities, but also in my career direction.

I would like to send to Dr. Jean-Marie Rocchisani my appreciation for providing me the medical image dataset and for his valuable comments and verification on the experimental results associated with this dataset. Moreover, I am deeply thankful to Professor Yann Gousseau and Professor Aymeric Histace, the two reviewers of my dissertation. It is a great honor for me to have my PhD thesis evaluated by these experts. I also want to express my gratitude to the rest of my thesis committee: Professors Julie Delon, Assoc. Prof. Georges Koepfler and Professors Canh Duong Pham, for their insightful comments, suggestions and interesting questions which help me to significantly improve the quality of my thesis.

I owe my warmest affection to all members of the laboratories LAGA and L2TI of the University Paris 13 for their kindness and helpfulness, especially the secretaries Isabelle and Yolande (LAGA) and Amel (L2TI).

I would like to thank all my Vietnammese friends for their encouragement and support. Together, we have shared many wonderful and unforgettable memories.

Finally, I would also express my appreciation to my family for their sacrifice and consideration. They are always supporting and encouraging me with their best effort.

Introduction

Chapter content [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] State-of-the-art Image Restoration . . . . . . . . . . . . . . . . . . .

Introduction

Digital images undoubtedly play a huge part in many applications such as medical imaging, remote sensing, surveillance, and entertainment, etc, because they can convey and store the information that reflect the objects or environment. However, the images acquired from digital imaging systems are the degraded observations of the unknown clean images. The degradation comes from various factors, e.g. noise corruption, blurring, resolution limitation, object movement, or a combination of them. Therefore, image restoration (IR) is a fundamental task that aims to recover a latent high quality image x ∈ R N from its degraded observation y ∈ R M . The IR problem can be modeled as follows:

y = Hx + η (1.1)
where H denotes the non-invertible degradation matrix and η is the Gaussian additive Image restoration is an ill-posed inverse problem because a lot of informations have been lost during the degradation which prevents a full recovery of the latent image

x. Several attempts have been made in the state-of-the-art to deal with the image restoration problem by suppressing the degradation factors and preserving as much as possible the image details. Due to the large space of image content, prior knowledge of image structure is crucial to regularize the inverse solution and obtain a reliable estimation x of x. However, learning priors and optimizing over whole image may lead to tremendous computational challenges. Instead, we select to handle the image restoration problem on small local patterns of image, which are called the image patches, in which the image priors can be learned more efficiently.

In the context of patch-based image restoration, each image is considered to be a set of overlapping image patches, and the reconstruction will be performed on each patch.

The image restoration model in (1.1) can be formulated on each patch as follows.

y i = H i x i + η i (1.2)
where y i ∈ R m is the i-th patch in the degraded image y, x i ∈ R n is the high-quality latent version of y i , H i is the degradation matrix on x i , and η i is the residual noise in

y i .
Without loss of generality, we can represent the latent image patch x i by a linear combination of a set of K basis vectors (also called atoms) {d 1 , . . . , d j , . . . , d K |d j ∈ R n } that form a dictionary of patches D = [d 1 , . . . , d K ] ∈ R n×K . Let Ω α ⊂ R K denote the vector space generated by the K atoms of the dictionary D. Then, each image patch x i can be described by a representation coefficients vector α i = [α i (1), . . . , α i (j), . . . , α i (K)] T such that x i = Dα i . Therefore, the image restoration model in (1.2) becomes:

y i = H i Dα i + η i (1.3)
From the Bayesian framework, the restoration of an image patch y i is equivalent to estimating a representation coefficients vector α i that maximizes the posteriori where p(α i ) is called the prior model of α i since it specifies a priori statistical features of image in the vector space Ω α and is independent of y i . With the assumption of Gaussian noise corruption N (0, σ 2 ), the likelihood can be described as p(y i |D, α i ) ∝ exp -1 2 ||y i -H i Dα i || 2 2 , and the Maximum A Posteriori (MAP) problem in (1.4) can be reformulated as αi = arg min

α i 1 2 ||y i -H i Dα i || 2 2 -λ log p(α i ) = arg min α i 1 2 ||y i -H i Dα i || 2 2 + λΦ(α i ) (1.5)
where Φ(α i ) ∝ -log p(x) is called the regularization term and λ is the trade-off parameter which balances the two terms of (1.5). After solving the optimization problem in (1.5), we can obtain an estimation of the latent image patch, denoted by xi , as xi = Dα i . Finally, the latent image x can be found by aggregating the overlapped regions between adjacent patches xi .

In this thesis, we investigate the image restoration in the context of image denoising and super-resolution via solving the optimization problem in (1.5). The two main issues involving to find the solution of (1.5) are the determination of the dictionary D and the formulation of the prior model p(α i ). The dictionary D contains atoms (patches)

Chapter 1. Introduction that will be directly used to recover the latent patch x i and thus has effects on the reconstruction quality. Thus it is preferable to build a dictionary that can hold the similar information in the underlying patch. Additionally, the prior model p(α i ) of the probability distribution of representation coefficients describes the manner that the latent image patch x i will be constructed from the K atoms. In this thesis, we will alternatively discuss the influence of both factors on the reconstruction of a degraded image.

The remainder of this chapter is organized as follows. Section 2 presents the motivation and organization of our thesis. In section 3, we briefly describe some image quality assessment metrics used for the evaluation of an image restoration algorithm.

Motivation and thesis organization

Patch-based image restoration under the Maximum A Posteriori (MAP) framework has proved successful in the reconstruction of a degraded image [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][2][3][4][5]. The principal concept of this thesis is built upon an interesting property of digital images that relates to the redundancy of information across different images, where the local patterns tend to repeat between multiple images. It is a fact that for a patch in an underlying image x, we can find a list of similar patches in other external good quality (clean, high resolution) images {x s }, which are often used as references and referred to as the standard images or example images.

For a better demonstration, we present on the left of Fig. 1.1 some reference patches, which are marked by rose blocks with red contours, in four example images, including two natural images of Peppers and of Boat, one CT image of Lung and one MRI image of Brain. For standard images, we collected multiple datasets, including the Berkeley Segmentation dataset [START_REF] Arbeláez | Contour Detection and Hierarchical Image Segmentation[END_REF] of natural images, the TCIA dataset [START_REF] Clark | The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository[END_REF] of CT image of Lung, and the MRI image of Brain in [START_REF] Di | The autism brain imaging data exchange: towards a largescale evaluation of the intrinsic brain architecture in autism[END_REF]. For each dataset, we randomly collected 200000 image patches to create a database of patches. For each reference patch in each image, we searched for 9 closest query patches in the database and portrayed them in blue contour blocks as shown on the right of Fig. 1.1. We can observe that the databases established from the external images contain several patches with similar structures to the local patches in underlying images. Therefore, it is expected that we can exploit the useful information in the database of patches in recovering a degraded image patch y i . In particular, we can learn a dictionary D or analyze the statistical characteristics of the distribution of image patches p(α i ) in the database and then used them in image restoration.

Motivation and thesis organization 7

Our first research is in the context of super-resolution of images affected by Poisson noise and inspired by previous work in the LAGA and L2TI labs on the SRSW method [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF].

The patches in the input image are transformed into the Anscombe domain so that the Poisson distribution is converted in approximately additive standard normal noise.

Thus we can apply the optimization framework in (1.5) to restore each patch y i in the noisy image. In this approach, we construct an exhaustive dictionary D, which is identical with the database of patches, so that we may ensure that the dictionary can cover the local information appearing in the noisy image. However, solving the problem (1.5) with a very large number of atoms in the dictionary requires tremendous computational complexity and makes the implementation challenging. To deal with this issue, a practical solution is to apply a patch-selection step on the dictionary D in order to choose the most appropriate atoms for the reconstruction. Hence, for each noisy patch y i , we exploit the Euclidean-based measurement to determine a set of similar pairs of low and high resolution patches in the database to generate a local reproducing kernel Hilbert space. The high resolution patch is recovered based on a linear regression of similar patches in the Hilbert space. We perform experiments on synthetic images and medical image dataset and demonstrate the outperforming of the proposed method comparing with some existing super-resolution methods. The details of this work will be reported in chapter 3.

Our second work is an expansion of the first study and the method SRSW [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] for super-resolution of an image corrupted by additive Gaussian noise. Similar to the above approach, we construct a large, exhaustive dictionary to cover as much as possible the local patterns in an input noisy and low-resolution image. However, we study an alternate strategy for selecting the local dictionary (a set of smaller number of atoms) for image super-resolution. Instead of taking into account the classical Euclidean distance as originally designed in SRSW and our previous work, as well as in many super-resolution methods in the literature, we consider an image patch as a distribution of grey levels and propose to exploit the Earth Mover's Distance (EMD) to measure the similarity between image patches. We show that the EMD better describes the perceptual similarity between patches than the Euclidean distance and helps to improve the searching accuracy and efficiency. Moreover, we introduce an 1 -norm-based threshold for each low-resolution noisy patch y i , which is built on the characteristic of the EMD, to facilitate the searching process and dynamically select the number of similar patches for constructing the local dictionary. Finally, the degraded patch y i is restored via solving (1.5) with the obtained local dictionary and the assumption of the sparse model of the probability distribution p(α i ). We carry out several experiments on Chapter 1. Introduction medical images with different modalities to demonstrate that the super-resolution with EMD is more efficient than the classical Euclidean-based super-resolution methods, in both case of noise-free and noise-corrupted images. We will present this work in further details in chapter 4.

In the two previous works, we have created a dictionary D by purely extracting a large number of patches in the standard images. Then to solve the restoration problem in (1.5), we first apply a patch-selection strategy to collect a local dictionary of smaller set of similar atoms for each degraded patch y i . After that, the latent patch x i can be recovered using the obtained local dictionary with the assumption of sparsity model of the representation coefficients α i . In the conventional sparse representation methods, the distribution of image patches in the vector space generated from the atoms in a local dictionary is assumed to be sparsely distributed and can be characterized in a smaller dimensional subspace. But the choice of sparse models is too arbitrary and they cannot exactly represent the true distribution of image patches. In our third work in this thesis, we will analyse the distribution of the representation coefficients p(α i ) of patches in the vector space generated by atoms of a dictionary. Moreover, to make convenient for studying the distribution of patches, we train a unique dictionary D and apply for all noisy image patches rather than using the local dictionary as in the first two researches.

By exploiting the redundancy of local information between multiple images, we assume that the distribution of representation coefficients of patches in the database and that of latent patches x i in the latent image x have similar forms. Hence, instead of adopting an arbitrary sparse prior, we propose to estimate the prior probability distribution p(α i ) from the empirical distribution of patches in the database and then apply it to regularize the denoising problem via (1.5). In the scope of this work, we adopt a simple scheme to estimate the prior probability of p(α i ) via a construction of a cell-wise constant histogram. This simple model for p(α i ) allows us to solve the optimization problem efficiently with application to image denoising. To demonstrate the potentiality of the proposed approach, we study a toy problem in three dimensions. By carrying out experiments on synthetic and natural images and making the comparison with existing sparsity models, we justify the applicable capacity of the estimation-based probability in dealing with an image denoising task. The proposed method will be discussed in detail in chapter 5.

In our last contribution in this thesis, we develop the idea introduced in chapter 5 by exploring Gaussian Mixture Models (GMMs) to represent the prior model of distribution of image patches instead of the piecewise constant model. Gaussian mixture models allow us to model the distribution of patches with more details and more regularity.

3. Image quality assessment metrics 9 However, solving the denoising problem with the whole GMM prior is inefficient because of high computational complexity. For favorable implementation in practice, the mixture model is trained with all its components, but only one prominent component is used for reconstructing each noisy patch. To our knowledge, justification for this approach is lacking in the literature. Therefore, we attempt to verify this strategy on several image datasets by evaluating the number of Gaussian components required for recovering patches. Our contribution is a comprehensive assessment of the number of useful components in the GMM for patch-based image denoising. We perform extensive simulations for a combination of two dictionary choices, including an identity and a K-SVD-based [2] matrices, and two model complexities of GMM, e.g. a small model and a large one. We conduct several experiments on 8 image datasets, with diversity of modalities and image structures, and study denoising with increasing number of components, which is in range {1, 5, 10, 15, 20}. We show that all patches in the degraded image can be recovered by only one prominent component with little loss of performance.

The interesting outcomes of our studies make strong evidence-based justification for the current practical use of GMM in the literature and drastically reduce computational cost. The details of this work is introduced in chapter 6.

The structure of this thesis is organized as follows. In chapter 2, we will present the concepts of image restoration methods, especially the existing works that inspired our researches during my thesis. Chapter 3 introduces the super-resolution method for images corrupted by Poisson noise under an exhaustive dictionary of patches and reproducing kernel Hilbert space. In chapter 4, we propose to exploit the Earth Mover's Distance in the pre-filtering step for similar image patches selection in the large dictionary to facilitate the super-resolution process. In chapter 5 we introduce an estimation-based framework for learning the prior distribution of image patches from the standard images and study its applications in image denoising. In chapter 6, we investigate the study of Gaussian mixture model for estimating the image prior in image denoising by analyzing the number of useful Gaussian components in the reconstruction of a noisy image patch.

We conclude this thesis and provide some future perspectives in chapter 7.

Image quality assessment metrics

Typically, we can not recover the exact unknown original image X ∈ R N On the other hand, the objective validations are computer-based methods that can automatically predict the perceived image quality. Moreover, according to whether the reference image is available or not, the objective assessment methods can be classified into full-reference metrics or no-reference metrics. In the full-reference approaches, the original good quality image X, which is referred to as reference image, is assumed to be known a priori. After that, we obtain a simulated degraded image Y through the degradation process and apply a restoration method to get the reconstruction image X. In our works, we adopt two objective full-reference metrics, namely the peak signal-to-noise ratio (PSNR) and the structural similarity index (SSIM) [START_REF] Wang | Image Quality Assessment: From Error Visibility to Structural Similarity[END_REF], to measure the quality of the recovered image X by quantifying the distortion between X and the reference image X. The rest of this section will make a summary on these two metrics.

Peak signal-to-noise ratio (PSNR)

The PSNR is the most commonly used image quality assessment metric for many image restoration task including denoising and super-resolution. The PSNR are mostly defined via the mean square error (MSE) between the reference image X and the reconstructed image X, which is determined by the 2 -distance:

M SE(X, X) = 1 N 1 N 2 N 1 i=1 N 2 j=1 X(i, j) -X(i, j) 2 (1.6)
where X(i, j) denotes the pixel at i-th row and j-th column of the image X. The definition of PSNR can be formulated as follows:

P SN R(X, X) = 10 log 10 M AX 2 X M SE(X, X) = 20 log 10   M AX X M SE(X, X)   (1.7)
3. Image quality assessment metrics 11

where M AX X is the maximum possible pixel value of the image, i.e. M AX X = 2 8 -1 = 255 for 8-bit images. The PSNR is a simple yet effective metric to describe the gray-level differences between two images. However, its fails to consider the structure distortions of images, which relate to the human visual perception because higher PSNR does not mean higher visual structural similarity between two images. Hence, many researchers have made efforts on finding alternative and better IQA metrics.

Structural similarity index (SSIM)

In [START_REF] Wang | Image Quality Assessment: From Error Visibility to Structural Similarity[END_REF], Wang et al. proposed the Structural SIMilarity (SSIM) metric based on the assumption that the human visual system is more sensitive to structural information.

The SSIM metric is very effective in evaluation the perceptual distortion between two images and has been adopting by the image processing community since the time it was introduced.

The SSIM value between two images is calculated by averaging the similarity index between two local image patches x i , y i ∈ R n taken from the same location of two images that are being compared. The local SSIM index measures the similarities of three elements of the image patches: luminance, contrast and structure as indicated in (1.8), (1.9) and (1.10).

(x i , y i ) = 2µ x µ y + c 1 µ 2 x + µ 2 y + c 1 (1.8) c(x i , y i ) = 2σ x σ y + c 2 σ 2 x + σ 2 y + c 2 (1.9) s(x i , y i ) = 2σ xy + c 3 σ x σ y + c 3 (1.10)
where µ x , µ y are the means of the patches x i and y i , respectively; σ x , σ y the standard deviations of x i and y i , and σ(x i ,

y i ) = 1 n-1 n j=1 (x i (j) -µ x )(y i (j) -µ y )
the covariance of the two image patches x i and y i . c 1 , c 2 , c 3 are positive constant to stabilize the division with weak denominator. Then, the SSIM index between two image patches was defined as a weighted combination of three comparative measures.

SSIM (x i , y i ) = (x i , y i ) α • c(x i , y i ) β • s(x i , y i ) γ (1.11)
where α, β, γ are positive constant. In this thesis, we follow the configuration proposed in [START_REF] Wang | Image Quality Assessment: From Error Visibility to Structural Similarity[END_REF] by setting In this chapter, we first make a comprehensive resume of the development of the image restoration methods in the state-of-the-art in section 1. After that, we will present some background information and preliminary knowledge of direct relevance to our researches in this thesis in section 2.

α = β = γ = 1, c 1 = (k 1 L) 2
SSIM (x i , y i ) = (2µ x µ y + c 1 )(2σ xy + c 2 ) (µ 2 x + µ 2 y + c 1 )(σ 2 x + σ 2 y + c 2 ) (1.

State-of-the-art Image Restoration

The history of image restoration began very early when image quality enhancement started to attract much attention in many applications of satellite imagery and remote sensing in the years 1970s. Several contributions have been made in the literature in solving the image restoration problem in (1.1), which aim to estimate the latent image

x from its degraded observation y such that the original information like edges, image details, etc., can be preserved.

Image restoration techniques can be categorized into three main groups: the local filtering-based methods, the model-based methods and the discriminative learning-based methods. In this chapter, we make a comprehensive overview on the state-of-the-art of image restoration.

Filtering-based image restoration

An early and simple image restoration approach is known as the local filtering-based methods because they estimate the true value of each pixel in the latent image x by using a weighted combination of its around neighbor pixels found in a local windows in the degraded image y. The weights are often specified by the coefficients of a spatial filter kernel, whose values are dependent on the spatial distance between two pixels, or by an interpolation function.

In the last few decades, many authors developed various simple local kernels for image restoration applications such as Gaussian filter [START_REF] Shapiro | Computer Vision[END_REF] in image denoising, fixed-function kernels like nearest, bilinear and bicubic interpolation [START_REF] Keys | Cubic convolution interpolation for digital image processing[END_REF][START_REF] Hou | Cubic splines for image interpolation and digital filtering[END_REF] in image super-resolution, etc. Later, several researchers investigated structure adaptive filters with regularized weights and sizes such as bilateral filter [START_REF] Tomasi | Bilateral filtering for gray and color images[END_REF], anisotropic filtering [START_REF] Perona | Scale-space and edge detection using anisotropic diffusion[END_REF], SUSAN filter [START_REF] Smith | SUSAN-A New Approach to Low Level Image Processing[END_REF], least-mean-square adaptive filter [START_REF] Haykin | Least-mean-square adaptive filters[END_REF], steering kernel regression [START_REF] Takeda | Kernel Regression for Image Processing and Reconstruction[END_REF], and edge-guided interpolation kernels [START_REF] Li | New edge-directed interpolation[END_REF][START_REF] Zhang | An edge-guided image interpolation algorithm via directional filtering and data fusion[END_REF] to better preserve image structures such as edge, corner, etc., and thus enhance the reconstruction results.

In spite of the low complexity and easy implementation, the major drawback of local filtering methods is that they are very sensitive to heavily degraded factors such Chapter 2. State-of-the-art image restoration as high noise level or large upscaling factor because the correlation between neighbor pixels has been severely corrupted and thus reduce the accuracy of the estimation of pixels in latent image.

Model-based image restoration

Exploiting only the information of pixels in a local window of the degraded image y, as proposed in the local filtering-based methods, is not enough to produce a reliable estimation of the pixels in the latent image x. In order to achieve a better reconstruction, we need to be provided with some prior knowledge about some properties of the latent image such as the statistical distribution in image space to regularize the restoration process.

Under the Bayesian MAP framework and the assumption of additive Gaussian noise corruption on image, the image restoration problem is described as:

x = arg max x p(x|y) = arg min x 1 2 ||y -Hx|| 2 2 -λ log (p(x)) (2.1)
p(x) is called the prior model of x because it characterizes the distribution of the latent image in a specific space and is independent of the observation y. Several efficient image priors have been proposed in the literature for solving the image restoration problem in (2.1). Many researchers investigated the prior models of the whole image by studying some properties of images in a specific domain.

One characteristic of the images is that their total variations (TV), which measure how much the image intensities change between the adjacent pixels and can be defined as the sum of absolute of the image gradient, are often small with good quality images.

When an image is degraded, e.g. corrupted by noise, its TV will dramatically increase.

A representation class of image restoration method, named TV-based, exploits the TV of images as a prior model of p(x) to regularize the TV of the degraded image y, with the aim of matching the TV of reconstructed image close to that of latent image x [START_REF] Rudin | Nonlinear Total Variation Based Noise Removal Algorithms[END_REF].

Another approach is to study the distribution of images in the wavelet domain which is generated by a set of fixed basis. An image will be represented by the wavelet coefficients in the wavelet domain. Several studies [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF][START_REF] Chang | Adaptive wavelet thresholding for image denoising and compression[END_REF][START_REF] Portilla | Image denoising using scale mixtures of Gaussians in the wavelet domain[END_REF][START_REF] Sendur | Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency[END_REF] have been proposed in the literature to learn a prior model from the distribution of wavelet coefficients of the images.

Learning image priors and performing restoration over the whole image may require tremendous computational effort. Instead, a considerable number of researchers attempted to work with small patterns in the images, which are known as image patches.

By observing the properties and distribution of patches in several images, numerous priors of image patches have been proposed in the state-of-the-art. For example, the nonlocal self-similarity and the low-rank approximation are based on the repetition of patches in different places within an image. The sparse representation and Gaussian mixture model are built to represent the distribution of patches in a vector space generated by a set of fixed atoms. Additionally, there are many endeavors to combine available priors to benefit their advantages and lead to more effective image restoration performances.

Total variation

Total variation (TV) is a well-known, early developed class of methods for image restoration. The TV of an image x relates to the gradient of image and can be formulated as:

T V (x) = Dx 1 (2.2)
where D denotes the gradient operator.

In the pioneering work of TV restoration, Rudin et al. [START_REF] Rudin | Nonlinear Total Variation Based Noise Removal Algorithms[END_REF] exploited the TV as a prior model p(x) of the image to regularize the denoising process and minimize the TV of the reconstructed image. More specifically, the problem in (2.1) was written as

x = arg min x 1 2 ||y -Hx|| 2 2 + λ||Dx|| 1 (2.3)
Many algorithms have been developed for solving the problem in (2.3), including dual formation [START_REF] Chambolle | An Algorithm for Total Variation Minimization and Applications[END_REF], Newton-based method [START_REF] Michael | On Semismooth Newton's Methods for Total Variation Minimization[END_REF], alternating direction method [START_REF] Michael | Solving Constrained Totalvariation Image Restoration and Reconstruction Problems via Alternating Direction Methods[END_REF], split

Bregman algorithm [START_REF] Getreuer | Rudin-Osher-Fatemi Total Variation Denoising using Split Bregman[END_REF], etc.

One of the shortcoming of the TV methods is the appearance of staircasing artifacts in slanted regions. Many researchers [START_REF] Scherzer | Denoising with higher order derivatives of bounded variation and an application to parameter estimation[END_REF][START_REF] Chan | High-Order Total Variation-Based Image Restoration[END_REF][START_REF] Bredies | Total Generalized Variation[END_REF][START_REF] Knoll | Second Order Total Generalized Variation (TGV) for MRI[END_REF] modified the TV model by adding variety of higher-order derivatives of image to reduce the staircasing effects and produced more pleasant reconstruction results. Another drawback of the TV is that it does not consider the orientation of image gradients and thus is not very suitable for images with rich textures where the edges and contours exhibit dominant directions. To overcome this issue, Bayram and Kamasak [START_REF] Bayram | Directional Total Variation[END_REF] proposed the directional total-variation (DTV) by introducing the weights in the image gradient coefficients depending on their directions.

The strategy of Bayram significantly improved denoising performance on nature texture images with dominant direction. Later, Wang et al. [START_REF] Wang | Single-image super-resolution using directional total variation regularization and alternating direction method of multiplier solver[END_REF] exploited the DTV in the image super-resolution and obtained promising results.

The TV regularization showed its effectiveness for restoring of piecewise-smooth Chapter 2. State-of-the-art image restoration regions while having the capacity of preserving the sharp edges in the image. However, due to the minimization of TV, the small details in degraded image can also be considered as noise and thus will be remove during the reconstruction. In order to better preserve the image details, a combination of the variational approach and another prior model such as the nonlocal self-similarity [START_REF] Li | Single image super-resolution using combined total variation regularization by split Bregman Iteration[END_REF][START_REF] Chao Ren | Single Image Super-Resolution via Adaptive High-Dimensional Non-Local Total Variation and Adaptive Geometric Feature[END_REF] has been proposed in recent years and achieved promising performances in image restoration. H is an identity matrix, as y = x + η. Applying the wavelet transform on these images, we have:

Wy = Wx + Wη = α w + v (2.4)
where Wy ∈ R N is the wavelet coefficients of the noisy image, α w and v are the wavelet coefficients of the unknown clean image x and the residual noise, respectively. The objective of wavelet-based denoising is to estimate αw from the wavelet transform Wy of the noisy image. Finally, the latent image x can be recovered by applying the inverse transform x = W -1 αw . Under the Bayesian perspective and MAP framework, we have:

αw = arg max α w {p(α w |Wy, v)} = arg max α w {p(Wy|α w , v) + p(α w )} (2.5)
Determining a priori model p(α w ) on the statistic properties of wavelet coefficients is crucial for solving the problem in (2.5). Several attempts [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF][START_REF] Portilla | Image denoising using scale mixtures of Gaussians in the wavelet domain[END_REF][START_REF] Sendur | Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency[END_REF][START_REF] Chan | High-Order Total Variation-Based Image Restoration[END_REF] have been made

in the literature to learn the prior model of p(α w ) based on the observation of wavelet coefficients of multiple standard images. A fascinating characteristic of the wavelet transform is that the wavelet coefficients α w of a clean image, in the high-frequency sub-bands, often exhibit non-Gaussian behavior with a sharp peak centered around zero and heavy-tailed distribution, which is close to sparse models. That means, only a small portion of wavelet coefficients of image having high-magnitude (at the tails of the distribution) which accounts for most energy of the image, while a large ratio of coefficients (around center of distribution) are very small or close to zero. In practice, the low-magnitude wavelet coefficients relate to noise and small details in image. Therefore, by performing a shrinkage to eliminate small wavelet coefficients, we can remove the noise (and certainly, the small details) from the image.

Many studies have been carried out on exploiting mathematical models to represent the sparse distribution of wavelet coefficients such as the Laplacian model [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF][START_REF] Donoho | De-noising by Soft-thresholding[END_REF],

the generalized Gaussian distribution [START_REF] Chang | Adaptive wavelet thresholding for image denoising and compression[END_REF], the Gaussian scale mixture model [START_REF] Portilla | Image denoising using scale mixtures of Gaussians in the wavelet domain[END_REF] or bivariate distributions [START_REF] Sendur | Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency[END_REF]. Based on the proposed sparsity models, numerous shrinkage strategies were also introduced for noise reduction, e.g. VisuShrink [START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF], SureShrink [START_REF] Donoho | De-noising by Soft-thresholding[END_REF],

BayesShrink [START_REF] Chang | Adaptive wavelet thresholding for image denoising and compression[END_REF], BiShrink [START_REF] Sendur | Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency[END_REF].

The wavelet transform is also exploited in image super-resolution [START_REF] Chan | Wavelet Algorithms for High-Resolution Image Reconstruction[END_REF][START_REF] Hui | Robust Wavelet-Based Super-Resolution Reconstruction: Theory and Algorithm[END_REF][START_REF] Do | Super-Resolution Image Reconstruction Using Wavelet Based Patch and Discrete Wavelet Transform[END_REF][START_REF] Ayas | Single image super resolution based on sparse representation using discrete wavelet transform[END_REF][START_REF] Lu | Wavelet-based single image super-resolution with an overall enhancement procedure[END_REF]. A common feature of wavelet-based super-resolution reconstruction is the assumption, without loss of generality, that the input low-resolution image y is considered as the low-frequency sub-band of the wavelet decomposition of the latent high-resolution image x. Therefore, the main purpose is to reconstruct the other missing high-frequency spectra of the latent image x and then apply the inverse wavelet transform (W -1 ) to obtain the estimation of

x. One representative approach is to deploy the patch-based representation methods such as the sparse representation or external nonlocal self-similarity, which will be discussed in the next sections, to reconstruct the wavelet coefficients in the high-frequency sub-bands.

The wavelet-based methods have shown their efficiency for image restoration, in particularly for denoising and super-resolution. However, when using shrinkage methods to reject the small wavelet coefficients, we not only separate the noise from the image, but also discard the small details in the image. As a result, the reconstructed image can be oversmoothed or smeared out the details.

Sparsity-based learning image restoration

Developing an effective prior over the whole image and then solving the restoration problem is a challenge because of high computation complexity. Another high-regarded Chapter 2. State-of-the-art image restoration class of method is to study the image restoration problem on image patches, from which the learning of image priors, as well as the optimization process can be more easily performed. In the last decade, the patch-based image restoration methods has been received remarkable research interest. In the context of this thesis, we also concentrate on studying the image restoration problem using the patch-based approaches.

In these methods, an image is partitioned into a set of overlapping image patches,

where each patch can be considered as an independent signal or in a correlation with other similar patches in the image. By this way, the reconstruction will be performed on each patch or on collaborative neighborhood patches. Let denote

{y i |y i ∈ R m ; i = 1, . . . , M }
is the set of M overlapping patches in the degraded image y. Our aim is to find a latent version x i ∈ R n of each patch y i that satisfies:

y i = H i x i + η i (2.6)
where η i is the additive noise in the patch y i , and H i denotes the degradation matrix on x i .

Without loss of generality, the motivation of patch-based methods is to represent each latent image patch x i by a linear combination of K basis vectors, which are also called atoms, {d 1 , . . . , d j . . . , d K |d j ∈ R n } of a dictionary of patches D ∈ R n×K . This means x i = Dα i , where α i ∈ R K is called the representation coefficients vector of x i .

The problem in (2.6) can be rewritten as

y i = H i Dα i + η i (2.7)
The restoration of a degraded image patch y i is equivalent to estimate a coefficients vector α i that satisfies the degradation model. Under the MAP framework and Gaussian

noise assumption, we have: αi = arg min

α i {||y i -H i Dα i || 2 2 -λ log (p(α i ))} = arg min α i {||y i -H i Dα i || 2 2 + λΦ(α i )} (2.8)
In the last ten years, the sparse representation is one of the most highly regarded class of methods for tackling patch-based restoration of images. The principal concept of this approach is based on an assumption that each image patch x i can be described as a linear combination of few atoms in the dictionary [2,[START_REF] Mallat | Matching pursuits with time-frequency dictionaries[END_REF][START_REF] Elad | Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries[END_REF][START_REF] Zeyde | On Single Image Scale-Up Using Sparse-Representations[END_REF]. This leads to the prior model of α i in (2.8) which can be formulated as Φ(α i ) = ||α i || 0 , where the pseudo norm 0 counts the number of non-zero elements of the vector. Another well-known approach is to assume that the distribution of representation coefficients α i in the vector space of dictionary D exhibits the heavy-tailed forms and thus can be fitted by some well-known mathematical distributions such as the Laplacian model [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]3,[START_REF] Yang | Alternating Direction Algorithms for 1 -Problems in Compressive Sensing[END_REF][START_REF] Wang | Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis[END_REF][START_REF] Yang | Fast 1 -Minimization Algorithms for Robust Face Recognition[END_REF][START_REF] Huang | Coupled Dictionary and Feature Space Learning with Applications to Cross-Domain Image Synthesis and Recognition[END_REF][START_REF] Trinh | An effective example-based learning method for denoising of medical images corrupted by heavy Gaussian noise and poisson noise[END_REF], with Φ(α i ) ∝ λ||α i || 1 , or the hyper-Laplacian model [START_REF] Xu | L 1/2 Regularization: A Thresholding Representation Theory and a Fast Solver[END_REF][START_REF] Rakotomamonjy | pq Penalty for Sparse Linear and Sparse Multiple Kernel Multitask Learning[END_REF][START_REF] Lyu | A Comparison of Typical p Minimization Algorithms[END_REF][START_REF] Guo | Enhancing sparsity via p (0 < p < 1) minimization for robust face recognition[END_REF][START_REF] Jia | Image denoising using hyper-Laplacian priors and gradient histogram preservation model[END_REF][START_REF] Cao | Image Super-Resolution via Adaptive p (0 < p < 1) Regularization and Sparse Representation[END_REF][START_REF] Kong | Generalized p -regularized representation for visual tracking[END_REF], which leads to Φ(α i ) ∝ λ||α i || p with 0 < p < 1. Likewise, the collaborative representation with 2 -norm regularized least squares, Φ(α i ) ∝ λ||α i || 2 , has been studied in many image processing task such as recognition [START_REF] Zhang | Sparse representation or collaborative representation: Which helps face recognition?[END_REF], super-resolution [START_REF] Timofte | Anchored Neighborhood Regression for Fast Example-Based Super-Resolution[END_REF][START_REF] Timofte | A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution[END_REF][START_REF] Zhang | CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution[END_REF][START_REF] Yang | Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries[END_REF] and achieve competitive performances.

Though being useful and popular, 0 or 1 based sparsity priors have several limitations, e.g., they fail to capture complex sparse structures. As a result, the elastic net has been proposed [START_REF] Zou | Regularization and variable selection via the Elastic Net[END_REF], in which the regularization constraint can be written as 2 ) encourages group selection. The elastic net is also convex, which can also be efficiently solved by many solvers, e.g., LARS-EN [START_REF] Zou | Sparse Principal Component Analysis[END_REF].

R(α i ) = λ 2 ||α i || 2 2 + λ 1 ||α i || 1 ,
Despite the impressive performances in image restoration applications, one shortcoming of the sparse models is that they may fail to describe the true distribution of image patch in the vector space generated by atoms of the dictionary D. Therefore, exploring a prior model which can more accurately present the true distribution of image patches can help to improve the quality of image recovering. In chapter 5 and 6, we investigate mathematical models of piece-wise constant function and Gaussian mixture model to precisely characterize the statistical distribution of patches and apply to restore a degraded image.

Nonlocal self-similarity

One of the most remarkable priors for patch-based image restoration is the so-called nonlocal self-similarity (NSS), which is motivated by the rich redundancy of local informations within an image. Based on the fact that for each patch, we can find many similar versions across the whole image, Buades et al. [START_REF] Buades | A non-local algorithm for image denoising[END_REF][START_REF] Buades | A Review of Image Denoising Algorithms, with a New One[END_REF] proposed a pioneer work named nonlocal means (NLM) to gather repeating structures in a given image and perform a weighted filtering to suppress the noise. The basic idea of NLM method is that each pixel can be restored by a weighted average of other pixels over a large window in the degraded image, where the weights are determined by the similarity between the local patches surrounding these pixels. The NLM filter is able to remove additive white noise while preserving sharp edges and fine texture details and has attracted significant attention in the image processing community at the time of its introduction. Recently, several variants of NLM have been proposed [START_REF] Kervrann | Local Adaptivity to Variable Smoothness for Exemplar-Based Image Regularization and Representation[END_REF][START_REF] Deledalle | Non-local Methods with Shape-Adaptive Patches (NLM-SAP)[END_REF][START_REF] Chatterjee | Patch-Based Near-Optimal Image Denoising[END_REF][START_REF] Zhang | Two-Direction Nonlocal Model for Image Denoising[END_REF] to improve the adaptivity of the nonlocal filter.

Image restoration methods with NSS priors have achieved competing performance Chapter 2. State-of-the-art image restoration in the literature. However, they have some drawbacks. First, the recovering of each degraded patch is built on the weighted graph based on the similarity between image patches, which can be disturbed in many cases, e.g. heavy noise corruption. Thus it can produce inaccurate weights and lead to inexactness in reconstruction. Another shortcoming of this approach is that the performance highly depends on the number of matched patches used for reconstruction. To overcome these weakness, many researchers [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]4,[START_REF] Zhang | CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution[END_REF][START_REF] Yang | Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries[END_REF][START_REF] Dong | Sparsity-based image denoising via dictionary learning and structural clustering[END_REF][START_REF] Dong | Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[END_REF][START_REF] Zhang | Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution[END_REF] have investigated a combination of nonlocal self-similarity and sparsity priors in the same framework and attained impressive results. We will have a brief discussion on these approaches in section 1.2.7.

A brilliant variant of the nonlocal self-similarity is called the external similarity, which is based on the fact that the similar local information can be found in different images, as demonstrated in chapter 1. Several studies have been made in the literature to exploit the redundancy across images for image restoration. In [START_REF] Chang | Super-resolution through neighbor embedding[END_REF], Chang et al.

prepared a database of low-and high-resolution image patches from a set of good quality images (called standard images) for image super-resolution. Each patch in the low-resolution image can be approximated by a weighted average of its matched patches retrieved from database. After that, these weights are used to estimate the latent high-resolution patch. The concept of the external similarity is also exploited in recent researches in the image super-resolution [START_REF] Zhang | CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution[END_REF][START_REF] Yang | Consistent Coding Scheme for Single-Image Super-Resolution Via Independent Dictionaries[END_REF][START_REF] Zhang | Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution[END_REF] where the database of patches are divided into groups and the mappings between low-and high-resolution feature spaces are constructed on each group with the helps of sparse models. Another remarkable approach is to learn a prior model or study a property (e.g. the statistical distribution) of image patches in a database extracted from the standard images. Due to the redundancy of image contents, we can expected that every patch in the reconstructed image is likely followed the prior of patches in the database. Therefore, we can use the learned prior to regularize the image restoration process and recover a degraded image. A glorious contribution in this approach is to take advantage of the mixture model to simulate the probability distribution of image patches, and will be discuss in subsection 1.2.6.

In chapters 3 and 4 of this thesis, inspired from the framework in [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][START_REF] Chang | Super-resolution through neighbor embedding[END_REF], we exploit the external nonlocal similarity to establish a large dictionary of patches randomly extracted from a set of standard images and then select a subset of similar patches from the dictionary to reconstruct each patch in the degraded image. After that, we investigate the use of some mathematical models such as the piece-wise constant function

Low-rank approximation

Another remarkable image prior derived from the nonlocal self-similarity property of images, is known as the low-rank approximation, which aims to recover an underlying low rank matrix from its degraded observation. The low rank terminology means that the observed data can be represented in a lower dimensional subspace with some different outliers corruption sources.

Recently, patch-based low-rank minimization methods for image/video denoising have achieved great success [START_REF] Ji | Robust video denoising using low rank matrix completion[END_REF][START_REF] Ji | Robust Video Restoration by Joint Sparse and Low Rank Matrix Approximation[END_REF][START_REF] Gu | Weighted Nuclear Norm Minimization with Application to Image Denoising[END_REF][START_REF] Guo | An Efficient SVD-Based Method for Image Denoising[END_REF][START_REF] Jia | Rank Constrained Nuclear Norm Minimization with Application to Image Denoising[END_REF][START_REF] Xie | Weighted Schatten p-Norm Minimization for Image Denoising and Background Subtraction[END_REF][START_REF] Gu | Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision[END_REF][START_REF] Zha | Analyzing the group sparsity based on the rank minimization methods[END_REF]. For each patch y i in the underlying image y, a set of its nonlocal similar patches across the image is first retrieved. Then these similar patches (including y i ) are stacking into a matrix Y i , where each column of Y i corresponds to an image patch. The degraded model is described on each group of patches as:

Y i = X i + V i (2.9)
where X i is a matrix containing clean versions of patches (columns) in Y i and N i is the residual noise in these patches. It is intuitive that the matched patches in Y i have close image structures, thus the noiseless version X i should lie in a low dimensional subspace. The denoising problem is performed on each of group of patches Y i under the constraint of minimization of the rank of X i as: Xi = arg min

X i {rank(X i )} subject to ||Y i -X i || 2 2 ≤ (2.10)
where is a positive constant related to the noise level in the observed image y. Since direct rank minimization in (2.10) is NP hard, non-convex and difficult to solve, the problem is generally relaxed by substitutively minimizing the nuclear norm of the estimated matrix X i . The nuclear norm of a matrix X i , denoted by ||X i || * , is defined as the sum of its singular values, e.g.

||X i || * = k |σ k (X i )|, where σ k (X i ) is the k-th
singular value of X i . Using the Lagrange multiplier, the denoising is described as:

Xi = arg min X i {||Y i -X i || 2 2 + λ||X i || * } (2.11)
The optimization of (2.11) can be obtained by off-the-shell algorithms such as the hard thresholding filter with principle component analysis (PCA) or singular value decomposion (SVD) [START_REF] Ji | Robust video denoising using low rank matrix completion[END_REF][START_REF] Ji | Robust Video Restoration by Joint Sparse and Low Rank Matrix Approximation[END_REF][START_REF] Wright | Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization[END_REF][START_REF] Cai | A Singular Value Thresholding Algorithm for Matrix Completion[END_REF]. The nuclear norm minimization (NNM) approach has been attracting significant attention due to its rapid development in both theory and implementation. In recent years, many researchers investigate to improve the flexibility Chapter 2. State-of-the-art image restoration of the original nuclear norm, by proposing the weighted nuclear norm [START_REF] Gu | Weighted Nuclear Norm Minimization with Application to Image Denoising[END_REF][START_REF] Gu | Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision[END_REF][START_REF] Zha | Analyzing the group sparsity based on the rank minimization methods[END_REF] or Schatten p-norm minimization [START_REF] Xie | Weighted Schatten p-Norm Minimization for Image Denoising and Background Subtraction[END_REF], and achieve competitive denoising performances in the state-of-the-art.

Gaussian mixture model (GMM)

Under the Bayesian MAP framework (1.5), learning good image priors is of most importance for the success of an image restoration method. In the literature, mixture models have attracted considerable attention due to their adaptability and flexibility in describing the characteristic of signals by assuming that the signals are generated by a mixture of probability distributions. Among various contributions, Gaussian Mixture Models (GMMs) have shown their powerful ability in many applications such as image classification and segmentation [START_REF] Celeux | Gaussian parsimonious clustering models[END_REF][START_REF] Permuter | Gaussian mixture models of texture and colour for image database retrieval[END_REF][START_REF] Permuter | A Study of Gaussian Mixture Models of Color and Texture Features for Image Classification and Segmentation[END_REF][START_REF] Fu | Color Image Segmentation Using Gaussian Mixture Model and EM Algorithm[END_REF]. Moreover, several studies investigated to exploit GMM as an image prior p(x) to regularize the inverse optimization problem in (1.5).

Portilla et al. [START_REF] Portilla | Image denoising using scale mixtures of Gaussians in the wavelet domain[END_REF] have attained impressive image denoising results by using the Gaussian scale mixture, which is derived from the GMM by assuming different scale factors in the mixture of Gaussians, to model the distribution of wavelet coefficients of an image.

Recently, by considering that natural images are non-Gaussian and image patches are regarded as samples of a multivariate random variable, many researchers [5,[START_REF] Zoran | Natural Images, Gaussian Mixtures and Dead Leaves[END_REF][START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Wang | SURE Guided Gaussian Mixture Image Denoising[END_REF][START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF][START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF][START_REF] Lu | Image-Specific Prior Adaptation for Denoising[END_REF][START_REF] Luo | Adaptive Image Denoising by Mixture Adaptation[END_REF][START_REF] Teodoro | Single-frame Image Denoising and Inpainting Using Gaussian Mixtures[END_REF][START_REF] Teodoro | Image restoration with locally selected class-adapted models[END_REF] used Gaussian mixture models (GMMs) to characterize the statistical distribution of image patches and obtain state-of-the-art denoising and image restoration results.

In [5], Zoran et al. modeled the distribution of patches in the database by a mixture of K Gaussian components as:

p(x i ) = K k=1 π k N (x i |µ k , Σ k ) (2.12)
where π k is the mixing weight with K k=1 π k = 1, µ k , Σ k are the mean and covariance matrix of the k-th Gaussian component. The generic GMM prior in (2.12) is then exploited to regularize the denoising problem under the Bayesian MAP framework.

In [START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF], Xu et al. combined the nonlocal self-similarity prior and GMM in a unified scheme by considering that a group of similar patches in an image should belong to the same Gaussian component and proposed a collaborative patch-based denoising algorithm called patch-group-based GMM to learn nonlocal self-similarity prior from standard images and then adopted them to recover an underlying image.

Another noticeable GMM-based denoising approach is to learn a GMM from the noisy observation [START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF] by clustering patches in the degraded image into multiple groups via nonlocal self-similarity and learning a Gaussian distribution for each cluster. An iterative MAP expectation-maximization algorithm was also proposed to alternatively update the reconstructed image and parameters of the GMM.

Because of the large space of image patches, the generic GMM learned from standard images may not be able to model every image patch well. Some image-specific patches of a given image will be outliers in the view of a generic GMM prior and thus can not be well reconstructed using that GMM. Many attempts have been made in order to deal with this issue [START_REF] Lu | Image-Specific Prior Adaptation for Denoising[END_REF][START_REF] Luo | Adaptive Image Denoising by Mixture Adaptation[END_REF][START_REF] Teodoro | Image restoration with locally selected class-adapted models[END_REF] by unifying internal and external GMM image patch priors. A generic GMM model is first trained from a collection of patches randomly sampled from standard images and then is adapted to the degraded image by simultaneously adding additional components and refining the component parameters. Although it is very effective for restoring images, a shortcoming of the GMM is the high runtime complexity making them ill-suited for most practical applications. In [START_REF] Parameswaran | Accelerating GMM-based patch priors for image restoration: Three ingredients for a 100× speed-up[END_REF], Parameswaran et al.

proposed an approximation algorithm to dramatically speed-up the implementation of the GMM in image denoising and deblurring, while incurring a negligible drop in the quality of reconstructed image.

Multiple variants of the Gaussian mixture models have been proposed in the literature for image restoration, especially for image denoising. An important characteristic of the GMM-based restoration is that a complete GMM is learn from the sample of image patches. In the reconstruction process, only one prominent Gaussian component is selected to represent the distribution of each patch in the degraded image. However, to our knowledge, it lacks clear evidences on the use of GMM in the reconstruction of an image. In this thesis, we made a contribution on justification of the number of useful Gaussian components exploited for recovering patches in the degraded image.

The details of our proposal will be presented in chapter 6.

Combination of multiple image prior models

Different image priors characterize varying and complementary aspects of natural image statistics, and thus it is possible to combine multiple priors to improve the image restoration performance. In the last few years, numerous studies [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]4,[START_REF] Zhang | CCR: Clustering and Collaborative Representation for Fast Single Image Super-Resolution[END_REF][START_REF] Dong | Sparsity-based image denoising via dictionary learning and structural clustering[END_REF][START_REF] Dong | Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[END_REF][START_REF] Zhang | Learning Multiple Linear Mappings for Efficient Single Image Super-Resolution[END_REF][START_REF] Dabov | Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering[END_REF][START_REF]Non-local sparse models for image restoration[END_REF][START_REF] Bengio | Group Sparse Coding[END_REF][START_REF] Chatterjee | Clustering-Based Denoising With Locally Learned Dictionaries[END_REF][START_REF] Zhang | Group-Based Sparse Representation for Image Restoration[END_REF] have investigated the combination of nonlocal self-similarity (NSS) and sparsity priors in a unified framework for image restoration. For example, Dabov et al. [START_REF] Dabov | Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering[END_REF] proposed a denoising method called BM3D by exploiting the NSS to enhance the sparse representation in transform-domain and achieved state-of-the-art denoising performance.

By constructing 3D data from an orthogonal transform of groups of similar image patches and carrying out a 3D collaborative filtering via thresholding and Wiener filter, the residual noise in image patches can be effectively removed. Later, in [4], Dong Many researchers [START_REF]Non-local sparse models for image restoration[END_REF][START_REF] Bengio | Group Sparse Coding[END_REF] offers a powerful mechanism of combining local sparsity and nonlocal self-similarity of images simultaneously in a unified framework which is the so-called group sparse coding. By partitioning the similar patches in the degraded image into multiple groups and assuming that the patches in the same group are encouraged to be sparsely represented by a similar set of the dictionary atoms, we can ensure that the reconstruction of matches patches should have the same image structure, and thus reduce the artifacts in the recovered image.

Another effective approaches [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][START_REF] Trinh | An effective example-based learning method for denoising of medical images corrupted by heavy Gaussian noise and poisson noise[END_REF] joined the external similarity and sparsity in a same scheme for image restoration. In [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF], Trinh et al. exploited the external similarity to construct a database of patches from the high quality images (called standard images).

After that, for each patch y i in the degraded image, the authors retrieved a local dictionary consists of most similar patches in the database, from which the sparse representation was performed to obtain an estimate of the latent patch x i . A part of our work in this thesis (chapter 4) was inspired from the method [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] in which we proposed a more efficient patch-selection step for searching the similar patches and thus improve the reconstruction performances.

Discriminative learning-based image restoration

Recently, the discriminative learning methods for image restoration has been attracting considerable attentions due to the blooming of artificial intelligence in computer vision.

The concept of these methods is to learn a compact inference or a mapping function from a training set of degraded-latent image pairs, which then is used to reconstruct an underlying image. The general model of learning process can be written as as the convolutional neural network (CNN) [START_REF] Jain | Natural Image Denoising with Convolutional Networks[END_REF][START_REF] Dong | Image Super-Resolution Using Deep Convolutional Networks[END_REF][START_REF] Kim | Accurate Image Super-Resolution Using Very Deep Convolutional Networks[END_REF][START_REF] Kim | Deeply-Recursive Convolutional Network for Image Super-Resolution[END_REF][START_REF] Shi | Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[END_REF][START_REF] Zhang | Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising[END_REF][START_REF] Zhang | Learning Deep CNN Denoiser Prior for Image Restoration[END_REF][START_REF] Lefkimmiatis | Non-local Color Image Denoising with Convolutional Neural Networks[END_REF][START_REF] Divakar | Image Denoising via CNNs: An Adversarial Approach[END_REF], multi-layer perceptron [START_REF] Burger | Image denoising: Can plain neural networks compete with BM3D?[END_REF],

arg min Θ L(x, x) s.t. x = F(y, H; Θ) (2.
stacked sparse denoising autoencoders [START_REF] Xie | Image Denoising and Inpainting with Deep Neural Networks[END_REF][START_REF] Agostinelli | Adaptive Multi-Column Deep Neural Networks with Application to Robust Image Denoising[END_REF]. After training process and obtaining the parameter Θ of the neuron network, we can recover the unknown latent image x from its degraded observation y by x = F(y, H; Θ). In the last decade, the discriminative learning methods using the deep convolutional neural network have achieved great success and led the state-of-the-art performances in various image restoration applications.

In a seminal work on deep learning SR, Dong et al. [START_REF] Dong | Image Super-Resolution Using Deep Convolutional Networks[END_REF][START_REF] Dong | Learning a Deep Convolutional Network for Image Super-Resolution[END_REF] first initialized the low-resolution (LR) image to a high-resolution (HR) image using a single filter, commonly bicubic interpolation, and then learns a CNN to predict the residual between the initialized HR image and the ground-truth image. In the super-resolution step, an underlying LR image is upscaled with bicubic interpolation and followed up by adding the residual estimated from the CNN model. Later, Kim et al. [START_REF] Kim | Accurate Image Super-Resolution Using Very Deep Convolutional Networks[END_REF] proposed to increase the number of hidden layer in the convolutional neuron network and cascaded small filters many times in this deep network structure to exploit contextual information over large image regions and noticeably improve the accuracy and visual quality of reconstructed image. Another very effective approach is to establish a deep recursive convolutional network [START_REF] Kim | Deeply-Recursive Convolutional Network for Image Super-Resolution[END_REF][START_REF] Tai | Image Super-Resolution via Deep Recursive Residual Network[END_REF], in which the same convolutional layer is repeatedly applied multiple times, to efficiently reuse weight parameters while exploiting a large image context. Another efficient deep learning approach is to directly predict the missing HR pixels from the LR image, as proposed in [START_REF] Shi | Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[END_REF], in which the handcrafted bicubic filter in the SR pipeline is replaced by more complex upscaling filters specifically trained on each feature map in the LR space via a sub-pixel convolution layer.

Recently, neuron network has also been successfully used in image denoising. In [START_REF] Jain | Natural Image Denoising with Convolutional Networks[END_REF],

a convolutional neuron network is learned from a training set of noisy-clean image pairs to map a degraded image to the reconstructed image. Chen and Pock [START_REF] Chen | Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration[END_REF] proposed a

Trainable Nonlinear Reaction Diffusion model to train filters and influence functions in feed-forward deep network by unfolding a fixed number of gradient descent inference steps. Instead of considering the whole image, mapping functions between noisy patches and their noise-free versions extracted from standard images can be learned via multi layer perceptrons [START_REF] Burger | Image denoising: Can plain neural networks compete with BM3D?[END_REF] or Stacked Sparse Denoising Auto-encoder model [START_REF] Xie | Image Denoising and Inpainting with Deep Neural Networks[END_REF] and applied to image denoising.

Despite the very competitive performance of the discriminative model learning for image restoration, its shortcoming is that the parameters Θ and mapping functions F(. . .) are trained for each specific degradation process (e.g. a determined noise level).

Thus, this can restrict the flexibility of application capacity of the trained models for Chapter 2. State-of-the-art image restoration different image restoration tasks.

Methods related to our work

In this section, we present significant preliminary knowledges and background information of some related works and directly relevant approaches that will be utilized or examined in the thesis. We commence with a quick description of the super-resolution method developed by our colleagues in the LAGA and L2TI labs named SRSW [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF],

which inspires our researches proposed in chapter 3 and chapter 4. Afterwards, we briefly summarize the principal concepts of two well-known competing super-resolution methods called Neighbor Embedding (NE [START_REF] Chang | Super-resolution through neighbor embedding[END_REF]) and Super-Resolution Via Sparse Representation (ScSR [3]), which are used to evaluate the performance and effectiveness of our proposed algorithms in chapter 3 and 4. Then, we present a résumé of the image restoration method using Expected Patch Log Likelihood (EPLL [5]), which coincides with one of our specific test cases proposed in chapter 6. After that, we will follow up with a presentation of the celebrated K-SVD algorithm [2] used for training a dictionary of patches, as well as a comprehensive comparison for dictionary selection mentioned in our proposed methods discussed in chapter 5 and chapter 6. We finish this section with the description of the Earth Mover's Distance, which is used as a metric for measuring the similarity between two image patches in chapter 4 of this thesis.

Image Super-Resolution by Sparse Weight (SRSW)

In [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF], Trinh et al. proposed a patch-based super-resolution method, named Super-Resolution by Sparse Weight (SRSW), for recovering each high-resolution image patch

x i ∈ R n from its degraded low-resolution version y i ∈ R m , which is assumed to be generated by the following model (derived from (1.2)):

y i = S i B i x i + η i . (2.14)
where B i and S i are the blurring and downsampling operations on image patch, respectively. η i ∼ N(0, σ 2 ) is the residual noise in each patch y i . The SRSW method consists of two main phases: Construction of dictionaries and Super-resolution reconstruction.

Dictionary construction

The authors constructed a couple of exhaustive high-resolution dictionary D h and low-resolution dictionary D l from a list of available standard (clean, high resolution) images {x s } as: ii. Generate the low-resolution database of patches {y s k |y s k ∈ R m ; k = 1, . . . , P } by downsampling the high-resolution database with the degrading factors:

y s k = S i B i x s k .
iii. Obtain the couple of high-and low-resolution dictionaries by normalizing the databases of patches:

D h = {d h j |d h j = x s j x s j 2 , j = 1, . . . , P } D l = {d l j |d l j = y s j y s j 2 , j = 1, . . . , P } (2.15)

Super-resolution

The SRSW method is developed based on a core idea that the distributions of high-resolution patches and their corresponding low-resolution version in two vector spaces generated by high-resolution dictionary D h and low-resolution dictionary D l are identical. That means, a high-resolution patch and its corresponding low-resolution version share the same representation coefficients vector. Thus, for each low-resolution patch y i , the authors found the representation coefficients vector αi under the Bayesian MAP framework αi = arg max

α i p(y i |D l , α i )p(α i ) (2.16)
and then used it to decode the high-resolution dictionary D h to get an estimation of the latent high-resolution image patch as xi = D h αi .

In [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF], Trinh et al. adopt the sparse Laplacian distribution as a prior model of p(α i ). Moreover, they proposed a weighting vector w i , which measured to the similarity between y i and the atoms d l j of the dictionary, to weight the sparse code α i . Hence, the MAP problem in (2.16) is formulated as: αi = arg min

α i y i -D l α i 2 2 + w T i α i 1 (2.17)
However, solving the problem (2.17) with the whole dictionary D l with very large number of P atoms is a really computational challenge. The authors introduced a patch-filtering step to select a smaller set of most similar atoms for recovering the underlying image patch y i . Hence, for each low-resolution patch y i , a couple of local Chapter 2. State-of-the-art image restoration dictionaries D h i and D l i were collected from the large dictionaries D h and D l :

D l i = {d l j ∈ D l |d(d l j , y i ) < r i } D h i = {d h j ∈ D h |d(d l j , y i ) < r i } (2.18)
where r i is a positive threshold chosen for each patch y i in such a way that the first K

atoms with smallest distances d(d l j , y i ) are selected (D l i ∈ R m×K , D h i ∈ R n×K ). d(d l j
, y i ) is the distance between the atom d l j and y i , and is defined in equation ( 26) in the paper [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] as:

d(d l j , y i ) = y i -µ il d l j 2 2 + E(y i -µ il d l j ) + Var(y i -µ il d l j ) -σ 2 (2.19) with µ il = E(y i )/E(d l j
). Thus, the problem (2.17) is rewritten on local dictionary D l i as: αi = arg min

α i y i -D l i α i 2 2 + w T i α i 1 (2.20)
In addition, the weighting vector

w i in (2.20) is now determined on the local dictionary D l i , w i = [w i (1), . . . , w i (j), . . . , w i (K)] T , in term of distances d(d l j , y i ) as: w i (j) =      e d(d l j ,y i ) if d(d l j , y i ) > γmσ 2 d(d l j , y i ) otherwise (2.21)
where γ is a positive constant.

After solving (2.20) with multiplicative updates algorithm to achieve the optimal value of αi , the latent high-resolution image patch can be estimated by xi = D h i αi .

The above restoration process is repeated for every patches y i in the degraded image y. After that, the super-resolved patches xi are aggregated to obtain a coarse estimate of the high-resolution image x. The authors finally apply a back-projection process to enhance the quality of the reconstructed image.

Image Super-Resolution through Neighbor Embedding (NE)

Chang et al. proposed in [START_REF] Chang | Super-resolution through neighbor embedding[END_REF] a patch-based super-resolution method called Neighbor Embedding (NE) to recover each patch y i in the low-resolution image y with a support of standard images {x s }. Generally, this method can be divided into two phases: construction of dictionaries and super-resolution.

Construction of dictionaries

From the list of standard high-resolution images {x s }, a couple of exhaustive dictionaries, which is identical to the database of patches, is created as follows:

i. Generate a low-resolution image y s for each standard image x s by blurring (with operator B) and then downsampling (using operator S) as y s = SBx s .

ii. Randomly collect P couples of low-resolution patch y s k and its high-resolution version x s k from the set of low-and high-resolution images {y s , x s } to form the couple of low-resolution dictionary D l and high-resolution dictionary D h .

D l = {d l j |d l j ≡ y s j , j = 1, . . . , P } D h = {d h j |d h j ≡ x s j , j =
D l i = {d l j ∈ D l | d l j -y i 2 < r i } D h i = {d h j ∈ D h | d l j -y i 2 < r i } (2.23)
The positive constant r i was set for each patch y i such that only first K atoms with smallest Euclidean distance to y i is selected.

The NE method was developed based on two assumptions:

• The image patches form a manifold, and a set of similar patches lie on or close to a locally linear part of the manifold. Thus, each image patch y i (or x i ) can be expressed by a linear combination of its neighbors in the local dictionary D l i (or

D h i ), e.g. y i = D l i α i with α i ∈ R K is a weighting vector.
• The corresponding patches in low-and high-resolution manifolds share the same weighting vector α i . Thus the authors can find α i from the relationship between y i and D l i and then apply it to estimate the latent high-resolution patch

x i = D h i α i .
For each low-resolution patch y i , Chang found the weighting vector

α i = [α i (1), . . . , α i (j), . . . , α i (K)] T
by solving the least mean square reconstruction error: αi = arg min

α i y i -D l i α i 2 2 s.t. K j=1 α i (j) = 1 (2.24)
Hence, the latent high-resolution image patch can be estimated as xi = D h i αi . The super-resolution was performed for every patch y i in the degraded image, then Chang obtained the super-resolved image x by aggregating the overlapping regions of the attained patches xi .

Chapter 2. State-of-the-art image restoration

Image Super-Resolution Via Sparse Representation (ScSR)

In [3], Yang et al. proposed a patch-based dictionary learning method, named ScSR, for enhancing the resolution of each low-resolution patch y i . This method is composed of two main phases: dictionary learning and super-resolution.

Dictionary learning

Instead of constructing a large couple of dictionaries from the database of patches,

D l = arg min D l ,{α s k }    P k=1 y s k -D l α s k 2 2 + λ P k=1 α s k 1    D h = arg min D h ,{α s k }    P k=1 x s k -D h α s k 2 2 + λ P k=1 α s k 1    (2.25) Let Y s = [y s 1 , . . . ,
{D l , D h } = arg min D l ,D h ,A s 1 m Y s -D l A s 2 2 + 1 n X s -D h A s 2 2 + λ 1 m + 1 n A s 1 (2.26)
2. Methods related to our work 33 By defining

X c =     1 √ m Y s 1 √ n X s     , D c =     1 √ m D l 1 √ n D h     , λ = λ 1 m + 1 n (2.27)
then, the problem in (2.26) can be rewritten as:

D c = arg min D c ,A s X c -D c A s 2 2 + λ A s 1 (2.28)
Therefore, we can use the same learning strategy in the single dictionary case for training the two dictionaries D l and D h . In [3], the authors alternatively updated the "pseudo" dictionary D c and sparse code A s using the sparse coding algorithm introduced in [START_REF] Lee | Efficient Sparse Coding Algorithms[END_REF].

Super-resolution

The super-resolution on degraded image y was performed on each low-resolution patch y i using the couple of dictionaries D l and D h jointly trained from the database of patches. Firstly, Yang found the sparse code α i of y i in the vector space generated by low-resolution dictionary D l by solving the Bayesian MAP problem: αi = arg min

α i y i -D l α i + λ α i 1 (2.29)
Then the estimation of the latent high-resolution image patch x i was obtained as xi = D h αi . The high-resolution patches xi were put into the proper locations in the high-resolution grid and Yang calculated the average of overlapping regions to get the super-resolved image. An iterative back-projection process was applied on the initial super-resolution image to enhance the reconstruction quality.

Image restoration using Expected Patch Log Likelihood (EPLL)

Zoran and Weiss [5] proposed an effective method called Expected Patch Log Likelihood (EPLL), which learned prior models from standard images to regularize the image restoration problem that modeled in (1.1). The key idea of this method is to develop an algorithm such that the patches in reconstructed image are likely to follow the designed prior, while keeping the reconstructed image still close to the corrupted image.

The proposed method EPLL can be interpreted in the point view of Bayesian MAP, which maximizes the posteriori probability of the latent image x ∈ R N given the observation y:

x = arg max x p(x|y) = arg max x {p(y|x)p(x)} (2.30)
With the assumption of Gaussian noise corruption, the likelihood is described as p(y|x) ∝ expy -Hx 2 2 /2σ 2 . Moreover, Zoran and Weiss construct prior on image patches {x i ∈ R n |i = 1, . . . , N } of the image. For more convenience, they denote an image patch as R i x, where R i is a matrix which extracts the i-th patch from the image. The prior statistical model of the whole image can be inferred by assuming that each image patch is independently draw from a prior model p.

Thus p(x) = N i=1 p(x i ) = N i=1 p(R i x).
The problem in (2.30) can be rewritten in the logarithm form as:

x = arg min x    λ 2 y -Hx 2 2 - N i=1 log(p(R i x))    (2.31)
Direct optimization of the cost function in (2.31) may be very hard, depending on the prior used. Zoran and Weiss presented an alternative optimization method called "Half Quadratic Splitting", in which a set of image patches {z i ∈ R n |i = 1, . . . , N } are introduced for the overlapping patches R i x in the latent image x.

x = arg min

x,{z i }    λ 2 y -Hx 2 2 - N i=1 β 2 z i -R i x 2 2 -log(p(z i ))    (2.32)
where β is a positive fixed value. The authors adopted an iterative manner for solving (2.32) by alternatively updating the variables as the following framework.

1. Fix x, update the new value for each image patch z i by solving ẑi = arg min

z i β 2 z i -R i x 2 2 -log(p(z i )) (2.33) 2. Fix {z i }, update x by x =   λH T H + β N j=1 R T j R j   -1   λH T y + β N j=1 R T j z j   (2.34)
3. Repeat step (1) and ( 2) until reaching the stopping condition.

At an iteration, the current reconstructed image x is fixed and each patch i in the image x is updated via the optimizing z i in (2.33). After achieving new values of image patches, the current reconstructed image x will be updated by aggregating the overlapping regions on image patches and then matching with the degraded image y using (2.34).

Finding the solution of (2.33) depends on the determination of the prior model of distribution of image patches in the image. In [5], Zoran and Weiss exploited the Gaussian Mixture Model (GMM) to characterize the probability distribution of patches in a database and expected that the patches in the underlying image x will likely follow the same distribution. More specifically, they randomly collected a set of P patches {x s k |k = 1, . . . , P } from a list of standard images {x s } and assumed that each patch x s k was drawn from a mixture of finite

M Gaussian components {π m , µ m , Σ m } as p(α s k ) = M m=1 π m N (α s k |µ m , Σ m ).
After learning the parameters of the GMM model, Zoran and Weiss supposed the prior model of image patches in the reconstructed image x can be described as p

(z i ) = M m=1 π m N (z i |µ m , Σ m ).
However, solving the problem in (2.33) is difficult. To tackle this issue, the authors proposed to select only one Gaussian component from the GMM to model the prior probability of p(z i ) and introduced a scheme for optimizing (2.33) as follows.

i. Given each patch R i x in the current reconstructed image x, calculate the conditional mixing weight γ im

γ im = π m N (R i x|µ m , Σ m + σ 2 i I) M l=1 π l N (R i x|µ l , Σ l + σ 2 i I) (2.35)
with σ i is the standard deviation of residual noise in i-th patch in the current reconstructed image x, I is an identity matrix.

ii. Select one Gaussian component which has the highest conditional mixing m max = max m γ im . Thus the probability distribution of p(z i ) in (2.33) is described as

p(z i ) ∝ N (z i |µ m max , Σ m max ).
iii. Solve the problem(2.33) to obtain the new value of

z i : ẑi = (βΣ m max + I) -1 (βΣ m max R i x + µ m max ) (2.36)
In chapter 6 of this thesis, we will analyze the use of GMM as a prior model of the distribution of representation coefficients of image patches, to regularize the denoising process. We inherit the same iteration optimization process in (2.33) and (2.34) proposed in the EPLL method [5]. At each iteration, we first update each patch in the current reconstructed image and then aggregate the obtained patches and match the reconstructed image to the corrupted image to ensure the coherent of image contents between the recovering image and the degraded image.

K-SVD: An over-complete dictionary learning for sparse representation

In [2]

D = arg min D,{α s k }    P k=1 x s k -Dα s k    subject to ∀k, α s k 0 ≤ T (2.37)
where T is a positive constant that constrains the sparsity (number of nonzero entries) of the representation coefficient vector α s k .

The authors use an iterative process to alternatively optimize the expression in (2.37).

First, they keep D unchanged and find the sparse code α s k for each patch x s k in the training set using the Orthogonal Matching Pursuit (OMP) algorithm [START_REF] Tropp | Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit[END_REF] as present in algorithm 2.1. After that, the sparse representation coefficients vectors {α s k } are fixed and the dictionary is updated atom by atom, with the support of the singular value decomposition (SVD) on the representation error corresponding to each atom. The summary of K-SVD algorithm is presented as follows: i. Initialization: Set a initial value of dictionary D (0) ∈ R n×K with 2 normalized columns. Set iterator J = 1.

ii. Repeat until convergence:

• Sparse coding stage: Fix D (J-1) , compute sparse representation coefficients vector α s k for each patch x s k in the training set by solving (2.38) using the OMP introduced in algorithm 2.1.

αs k = arg min α s k { x s k -D (J-1) α s k 2 2 } s.t. α s k ≤ T 0 (2.38)
• Dictionary update stage: Update each column (atom)

d i , i = 1, . . . , K in D (J-1) by
-Define a group of patches that use d i in representation:

ω i = {j|1 ≤ j ≤ N, α s k (j) 0}.
-Compute the representation error for all patches in ω i when removing the d i from the dictionary:

E ω i = (X s i ) ω i - j i α s k (j)d j (2.39)
-Apply SVD decomposition E ω i = U∆V T . Update the atom d i to be the first column of U (d i = U(:, 1)). Similarity, update the corresponding representation coefficients vector α s i = ∆(1, 1) * V(:, 1) • Increase the iterator J = J + 1.

Algorithm 2.1: The orthogonal matching pursuit (OMP)

Input : Image patch x s k , dictionary D, sparse constraint T 0 , representation error

Let ω be the index set of the selected dictionary atoms and initialize as ω = ∅

while not converged do

Compute the correlation of x s k to each dictionary atom d j which is not in ω, as

(x s k ) T d j x s k 2 d j 2
.

Pick the atom d j which has largest correlation and include in into the set

ω = ω ∪ {i}. Update the representation coefficient α s k,ω = (D T ω D ω ) -1 (D T ω x s k )
, where D ω ⊂ D contains all the atoms belonging to the set ω, α s k,ω consists of non-zero entries of α s k .

Compute the signal residual by subtracting the selected atoms

x s k = x s k -D ω α k,ω .
Check stop criterion (|ω| > T 0 or x s k 2 < ) end Output : Sparse representation coefficient vector α s k

Dictionary selection

In chapter 5 and 6, we demonstrate that the sparse models may fail to characterize the true distribution of representation coefficients of image patches in a vector space generated by atoms in a dictionary. Therefore, we investigate to estimate the probability distribution of patches in a database extracted from standard images as a prior to enhance the quality of a noisy image. However, analyzing the distribution of image patches in different vector space created by local dictionary for each patch y is difficult.

To this end, we construct a unique vector space from one global dictionary D learned from a training set of patches to recover all degraded patches in an underlying image.

In this subsection, we will discuss on our selection of a global dictionary for image restoration.

Determining a dictionary D which can convey the local information of the training set of patches is crucial for the success of an image restoration method. In the literature, Chapter 2. State-of-the-art image restoration there are two main categories of the dictionary: the analytical dictionary and the data-adaptive dictionary. One simple and representative approach was the use of analytic dictionary where the fixed transform bases such as wavelet [START_REF] Mallat | A Wavelet Tour of Signal Processing[END_REF], discrete cosine transform (DCT) [START_REF] Yu | DCT Image Denoising: a Simple and Effective Image Denoising Algorithm[END_REF], etc, are taken into account to build the dictionary D. However, off-the-shelf bases dictionaries have limitation in their ability to represent different types of image structures. Elad and Aharon [START_REF] Elad | Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries[END_REF] indicated that the dictionary learned from the image patches produced superior performances for image denoising than the analytic dictionary such as the DCT.

On the other hand, several efforts have been made in the literature to design 2. Methods related to our work 39 data-adaptive dictionaries including principal component analysis (PCA) [START_REF] Jolliffe | Principal component analysis[END_REF][START_REF] Vidal | Generalized principal component analysis (GPCA)[END_REF][START_REF] Bacchelli | Image denoising using principal component analysis in the wavelet domain[END_REF][START_REF] Liu | Adaptive sparse coding on PCA dictionary for image denoising[END_REF],

sparsity learning [2,3,[START_REF] Zeyde | On Single Image Scale-Up Using Sparse-Representations[END_REF][START_REF] Wang | Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis[END_REF][START_REF] Mairal | Online Dictionary Learning for Sparse Coding[END_REF][START_REF] Rubinstein | Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation[END_REF][START_REF] Mairal | Task-Driven Dictionary Learning[END_REF], or PCA-sparsity combination [4,[START_REF] Dong | Sparsity-based image denoising via dictionary learning and structural clustering[END_REF][START_REF] Dong | Image Deblurring and Super-Resolution by Adaptive Sparse Domain Selection and Adaptive Regularization[END_REF]. In the PCA-based dictionary learning, the patches in the training set X (which may be collected from standard images or in the degraded image itself) is first partition into multiple groups {X k } and each dictionary is trained for each group via performing the principal component analysis on the covariance matrices. In the stage of restoration, each degraded patch y i in the underlying image y will be assigned to one group and thus the corresponding dictionary is exploited to restore y i . However, in our researches in chapter 5 and 6, we need to prepare a global dictionary for every patches in an image to study the distribution of patches in a unique vector space. Therefore, we do not examine the choice of PCA-based dictionary here. Instead, we investigate the sparsity-based methods for the dictionary learning. In the literature, there are no full comprehensive comparison between different sparse learning dictionary methods, and the choice of a sparse model to learn the dictionary is an arbitrary decision. In our studies, we adopt the seminal work of Aharon et al. [2] known as the K-SVD algorithm, for training a data-adaptive dictionary. In the rest of this subsection, we make a comparison of image reconstruction performances between the K-SVD dictionary and the other analytic dictionaries.

For demonstration, we conducted the denoising on 20 widely used natural test images as shown in Fig.

α i α i 0 s.t. y i -Dα i 2 2 ≤ (2.40)
where is a positive constant relates to the noise level. The optimization of (2.40) can be found using the orthogonal matching pursuit method (OMP) in algorithm 2.1. We prepared three types of analytic dictionaries of K atoms, with K ∈ {64, 128, 256, 512}, including the DCT dictionary, the Haar wavelet dictionary and the Daubechies D4 (Db4) dictionary. Note that the analytic dictionaries were fixed for all images and different noise levels. In addition, when K > 64, we constructed overcomplete dictionaries from the orthogonal bases by shifting each basis (column) along vertical direction. For the data-adaptive dictionary, we randomly collected P = 100000 patches from the noisy image to train a dictionary using the K-SVD algorithm. 2. Methods related to our work 43 K-SVD dictionary trained from the image of Barbara in case of noise level σ = 30. We can observe that the K-SVD dictionary consists of multiple local structures appearing in the test image (Barbara) like diagonal textures, which can not be found in the analytic dictionaries. Fig. 2.3 presents the denoising results with 4 different dictionaries, which demonstrates that the reconstruction using K-SVD dictionary can better preserve the details of the image, as well as produce less artifacts than the others.

Moreover, Fig. that the use of an overcomplete K-SVD dictionary can improve the performance of a denoising method.

We have indicated that a dictionary trained with K-SVD algorithm can promote the reconstruction of a degraded image, comparing to the analytic dictionaries. Therefore, in our studies in chapter 5 and 6, we exploit the K-SVD algorithm to learn a global dictionary for image restoration.

The Earth Mover's Distance (EMD)

In chapter 3 of this thesis, we present a super-resolution method that take advantage of the redundancy of local information between different images of the same modality to generate a large, exhaustive dictionary that is identical to the database of patches, to recover a degraded low-resolution image y. Hence, for each patch y i in the underlying image y, we need to apply a patch-filtering step to create a local dictionary that contains only most similar patches in the database and use it in reconstruction of y i . Selecting a good local dictionary is the key issue of the image reconstruction algorithm.

In chapter 4, rather than using the conventional Euclidean distance to measure the similarity between two image patches, we consider an image patch y i ∈ R m as a distribution of gray levels at the pixels' locations and thus we can explore a metric that compares two histograms for image patches comparison. The commonly used metrics in practice such as Minkowski ( 1 , p , 2 distances), χ 2 , etc., are the bin-to-bin distances which assume that two histograms, e.g. h 1 and h 2 , are already aligned and compute the bin-wise differences between them. However this assumption can be violated through degradation of image, shape deformation, or image translation, etc. Thus, a cross-bin distance is preferred to address the alignment problem.

Among several contribution on cross-bin histogram comparison, the Earth Mover's Distance (EMD) has shown its effectiveness in perceptual evaluation the similarity between two distributions in the literature. The remainder of this subsection will briefly describe the concepts of the EMD, and in particularly one of its improvement version in both accuracy and computational complexity called FastEMD and denoted as EM D.

The Earth Mover's Distance is based on the minimal cost that must to be paid to transform one distribution into the other. It has been used in many applications such as image retrieval [130][START_REF] Lv | Image similarity search with compact data structures[END_REF][START_REF] Pele | A Linear Time Histogram Metric for Improved SIFT Matching[END_REF][START_REF] Pele | Fast and robust Earth Mover's Distances[END_REF][START_REF] Yu | On the Earth Mover'S Distance as a Histogram Similarity Metric for Image Retrieval[END_REF][START_REF] Jang | Accurate Approximation of the Earth Mover's Distance in Linear Time[END_REF], texture and color classification [START_REF] Zhang | Local Features and Kernels for Classification of Texture and Object Categories: A Comprehensive Study[END_REF], object recognition and matching [START_REF] Grauman | Fast contour matching using approximate Earth mover's distance[END_REF][START_REF] Li | A Novel Earth Mover's Distance Methodology for Image Matching with Gaussian Mixture Models[END_REF] and image segmentation [START_REF] Mendoza | Multi-dimensional earth mover's distance active contours[END_REF][START_REF] Li | Variational Earth Mover's Distance for Image Segmentation[END_REF]. The basic concept of EMD was first introduced by Peleg et al. in [START_REF] Peleg | A unified approach to the change of resolution: space and gray-level[END_REF] to measure perceptual similarity between two gray images. Each L-gray-level image is considered as a distribution of "pebbles" corresponding to its intensity, placed on the plane only at pixels' locations.

Peleg measured the distance between two gray images by proposed a minimization of a linear cost matching function to transfer the pebbles from one image to another. Later, Rubner et al. [130] introduced the original EMD to measure the similarity between two normalized histograms. The EMD is a natural and intuitive metric between two histograms if we think one of them as piles of masses and the other as holes sitting on the ground. Each pile of masses or a hole is a bin of the histogram. To quantify the difference between two histograms, we can measure how many masses we should get from one pile and how far to move them so that the holes are exactly filled in by the masses from the piles. The moving distance from a pile to a hole is called the ground distance, and the amount of masses is named the flow. Hence, EMD is the minimal total ground distance traveled weighted by the amount of masses moved.

For more convenience, we denote

h 1 = [h 1 (1), . . . , h 1 (i), . . . , h 1 (m)] T ∈ R m to
EM D(h 1 , h 2 ) = min f ij i,j f ij d ij i,j f ij subject to (2.42) (2.41)
f ij ≥ 0, j f ij ≤ h 1 (i), i f ij ≤ h 2 (j), i,j f ij = min   i h 1 (i), j h 2 (j)   (2.42)
where 1 ≤ i ≤ m, 1 ≤ j ≤ n, {f ij } represents the amount of masses moved from the bin i-th of h 1 to the bin j-th of h 2 . d ij denotes the ground distance between bin i and bin j of two histograms. The constraints in(2.42) can be intuitively understood as the amount of moving masses from a pile to a hole must be a positive value, and can not exceed the available total masses of the pile, or can not be larger than the capacity of the hole. There are two main problems with the original EMD of Rubner. First, it is only designed for normalized histograms and can not be used to evaluate two histograms with different masses ( i h 1 (i) j h 2 (j)). Second, for a general ground distance (e.g. Euclidean distance), it has a high computational time cost.

Based on the proposed model of Rubner [130], many authors developed different EMD distance for image retrieval [START_REF] Yu | On the Earth Mover'S Distance as a Histogram Similarity Metric for Image Retrieval[END_REF][START_REF] Ling | An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison[END_REF][START_REF] Shirdhonkar | Approximate earth mover's distance in linear time[END_REF] such as interest point matching using feature descriptors [START_REF] Ling | An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison[END_REF], contour matching by embedding the EMD into a normal space [START_REF] Grauman | Fast contour matching using approximate Earth mover's distance[END_REF] or in a Wavelet domain [START_REF] Shirdhonkar | Approximate earth mover's distance in linear time[END_REF]. Their objective is to improve the performances and reduce the time consumption of the EMD in many different ways, such as constructing a tree-based graph [START_REF] Lv | Image similarity search with compact data structures[END_REF][START_REF] Ling | An Efficient Earth Mover's Distance Algorithm for Robust Histogram Comparison[END_REF] or proposed different type of ground distance [START_REF] Yu | On the Earth Mover'S Distance as a Histogram Similarity Metric for Image Retrieval[END_REF]. However, like the original EMD of Rubner, all of these models are only used for distributions with equal total mass.

In [START_REF] Pele | A Linear Time Histogram Metric for Improved SIFT Matching[END_REF], Pele et al. proposed a variant of the original EMD model, called EM D to deal with unequal total mass distributions by adding into 2.41 a term of the total mass difference between them, as described in Thereafter, Pele [START_REF] Pele | Fast and robust Earth Mover's Distances[END_REF] presented a robust EMD algorithm, called F astEM D, expanded the model from [START_REF] Pele | A Linear Time Histogram Metric for Improved SIFT Matching[END_REF] by using the thresholded ground distances. That means

EM D α (h 1 , h 2 ) =   min f ij i,j f ij d ij   + i h 1 (i) - j h 2 (j) ×α max i,j d ij subject to (2.
d ij =      d ij if d ij ≤ d max d max if d ij > d max (2.44)
As demonstrated in [START_REF] Pele | Fast and robust Earth Mover's Distances[END_REF], adopting thresholded distances into EM D helps to reduce the number of edges in the flow-network of the EM D and thus it can accelerate the speed of EM D. Compare to the original EMD distance of Rubner [130], the metric EM D in [START_REF] Pele | Fast and robust Earth Mover's Distances[END_REF] improves both accuracy and speed and could be capable to apply on large histograms and databases. The implementation of the EM D can be found at http://www.ariel.ac.il/sites/ofirpele/fastemd/code/.

Chapter 2. State-of-the-art image restoration

In the context of this thesis, we exploit the EM D as a metric to measure the similarity between image patches and apply it to select a local dictionary for recovering each degraded patch y i . The details of this process will be described in chapter 4.

Lorem

Introduction

The work presented in this chapter is inspired apart from my master's internship at the L2TI and LAGA labs, where I studies the performance of an available super-resolution method 2 developed by the colleagues of the laboratories. This method is the SRSW, which was proposed for recovering a degraded low-resolution image corrupted by additive Gaussian noise, with the support of a set of standard images (same modality, highresolution, good quality images). In this chapter, we investigate a novel super-resolution method based on regression while inspiring the similar idea of using the standard images for enhancing the spatial resolution of a degraded image but for the case of Poisson noise. Our aim is to estimate a latent high-resolution image x ∈ R N from its degraded observation y ∈ R M , which is assumed to be generated by the model:

y = ξP(SBx/ξ) (3.1)
where P(•) is the Poisson distribution law, S is the downsampling operator with the magnification factor s, B is the blur operator and ξ is a constant using to control the Poisson noise level (the higher value of ξ, the heavier the noise affects the image).

In this thesis, we investigate the patch-based approach for recovering a degraded image y, in which the image is considered as a set of overlapping image patches

{y i |y i ∈ R m ; i = 1, . . . , M }.
The model in (3.1) can be formulated on each image patch as:

y i = ξP(S i B i x i /ξ) (3.2)
where S i and B i denote the downsampling and blurring operators on each image patch,

x

Proposed SRRH method for image super-resolution

The proposed method consists of two phases: construction of dictionaries and superresolution.

Construction of the dictionaries

In the first stage, we construct a couple of high-and low-resolution dictionaries {D h , D l } from the available standard images {x s }. We randomly extract a set of large P highresolution patches {x s k |x s k ∈ R n ; k = 1, . . . , P } from the standard images. After that, these high-resolution patches are blurred and downsampled to obtain their corresponding lowresolution versions {y s k |y s k ∈ R m ; k = 1, . . . , P }. The couple of high-and low-resolution dictionaries D h is determined to be identical to the database of high-and low-resolution patches.

D h = d h k |d h k ≡ x s k ; k = 1, . . . , P D l = d l k |d l k ≡ y s k ; k = 1, . . . , P (3.4

Image super-resolution

For each patch y in the degraded low-resolution image y, we first select a couple of local dictionaries {D h i ∈ R n×K , D l i ∈ R m×K , K P } of K most similar atoms from the large dictionaries {D h , D l } using the following steps [START_REF] Trinh | An effective example-based learning method for denoising of medical images corrupted by heavy Gaussian noise and poisson noise[END_REF].

i. Coarse search: We find a subset of L couples of closest patches from the dictionaries with the help of the Euclidean distance.

D l * = d l j ∈ D l | y i -d l j 2 < r i D h * = d h j ∈ D h | y i -d l j 2 < r i (3.5)
where r i is a positive constant is chosen for each patch y i such that only the first

L
V j = E T (y i ) -T (d l j ) + V ar T (y i ) -T (d l j ) -1 to create the local dictionaries. D l i = d l j ∈ D l * |V j < t i D h i = d h j ∈ D h * |V j < t i (3.6)
where t i is a set such that only the first K most matched atoms are selected.

In the next stage, we construct a linear regression mapping function between the low-resolution space of image patch and their corresponding high-resolution space, denoted as f SR (•), from the low-and high-resolution atoms in the local dictionaries.

Hence, the estimation of the high-resolution latent patch x i can be found from its degraded version y i by using the mapping function as

x i = f SR (y i ).
The linear regression function f SR (•) is built in a reproducing kernel Hilbert space H K generated from the vector space of low-resolution atoms in local dictionary D l i , which is defined in [START_REF] Wu | Multi-kernel regularized classifiers[END_REF] as:

H K =    f SR (z i ) | f SR (z i ) = K j=1 κ z i , d l j α j ; z i ∈ R m ; α j ∈ R n ; j = 1, . . . , K    (3.7)
where κ(z i ,

d l j ) = exp -z i -d l j 2 2 /2h
f SR ∈H K    K j=1 d h j -f SR (d l j ) 2 2 + λ f SR 2 H K    (3.8)
where λ is a regularization parameter.

Replacing f SR in(3.8) by its definition from (3.7), the problem in (3.8) is equivalent to finding a set of basic vector {α j ∈ R n ; j = 1, . . . , K} such that:

{ αj } = arg min α j      K j=1 d h j - K k=1 κ d l j , d l k α k 2 2 + λ K j=1 α j 2 2      (3.9) If we denote R = {R(j, k)|R(j, k) = κ(d l j , d l k ); 1 ≤ j, k ≤ K; d l j , d l k ∈ D l i }
to be a K ×K matrix of regression coefficients, and A i = {α j |α j ∈ R n ; j = 1, . . . , K} ∈ R n×K to be the matrix of basic vectors {α j }. The optimization problem in (3.9) can be written as: Âi = arg min

A i D h i -A i R 2 2 + λ A i 2 2 (3.

Performance evaluation

In this section, we compare the performance of the proposed method with some other existing super-resolution methods, including the Bicubic interpolation, the Neighbor Embedding super-resolution (NE [START_REF] Chang | Super-resolution through neighbor embedding[END_REF]) and the sparsity super-resolution method (ScSR [3]).

(a) & (f) synthetic images, (b) & (g) PET images of thorax (pet1), (c) & (h) PET images of abdomen in the area of the kidney (pet2), (d) & (i) PET images of abdomen in the area of the kidney (pet3), (e) & (j) PET images

of thorax through the lungs and heart (pet4). The summaries of NE and ScSR methods can be found in section 2 of chapter 2. The two image quality assessment metrics PSNR and SSIM are used for objective evaluation. The regularization parameter in (3.8) is set to λ = 10 -5 , and the decay parameter is chosen as h = 1000.

We carry out experiments on one synthetic image and four Positron Emission

For the ScSR method, the value of regularization parameter λ is set to 0.8, and the size of dictionary is 1024. The number of nearest neighbor of the NE method is K = 5.

Table 3.1 presents the results of competing super-resolution methods, where the best values of PSNR and SSIM are in bold red numbers. We can observe that the proposed method achieve superior performances for the PET images. Fig. 3.2 introduces the super-resolution results on the synthetic image with noise level ξ = 3. For the proposed method, we shows two results with the low-resolution patch size is 5 × 5 and 7 × 7 in Fig. 3.2(f) and (g). We can see the competing method (bicubic, NE, ScSR) recover the image with many artifacts. With the default patch size 5 × 5, our method produces some artifacts at the central of the image. When we increase the patch size to 7 × 7, the proposed method achieves better super-resolution performance and generates very pleasant result.

We also show the super-resolution results on PET images in Fig. 3.3 -Fig. 3.5. It can be seen that the proposed method yields better reconstructed image with much preserved details and closer to the original images that the competing methods.

Empirical Study on Parameters

In this section, we will discover the effects of some parameters on the performance of the proposed method, including the number of atoms K in the couple of local dictionaries (which is also the nearest neighbors of patches in the dictionary for each noisy image y i ) and the size of patches. Note that the magnification factor is always set to s = 2.

Effect of number of atoms K in local dictionaries

We will study the variation of super-resolution performance in term of PSNR of five test images in Fig. 3.1(f)-(j) with respect to the number nearest neighbors K. We present in Fig. 3.6 experiments on these test images for several values of K in [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]5,[START_REF] Shapiro | Computer Vision[END_REF][START_REF] Smith | SUSAN-A New Approach to Low Level Image Processing[END_REF][START_REF] Rudin | Nonlinear Total Variation Based Noise Removal Algorithms[END_REF][START_REF] Chan | High-Order Total Variation-Based Image Restoration[END_REF][START_REF] Trinh | An effective example-based learning method for denoising of medical images corrupted by heavy Gaussian noise and poisson noise[END_REF][START_REF]Non-local sparse models for image restoration[END_REF], and at two noise levels ξ = 1 and ξ = 5. We can see that the choice of K depends on the image content and noise level. In overall, the range of K yielding good quality results of super-resolution is around 5 to 10, for both cases of noise.

Effect of patch size

We also examine the super-resolution performance of the proposed method in the variation of size of low-resolution patch √ m × √ m. image yielding good SR results is 7×7 pixels. With this patch size, the proposed method obtains higher PSRN and SSIM values (PSNR = 34.58, SSIM = 0.972 for ξ = 1; PSNR = 32.30, SSIM = 0.937 for ξ = 3 and PSNR = 30.77, SSIM = 0.918 for ξ = 5) than the results of 5 × 5 patch size in Table 3.1.

Conclusion

We present in this chapter an effective patch-based method that takes into account the repetition of image patches across multiple images of the same modality to improve the spatial resolution of a given noisy and low-resolution image. By selecting only a small set of K nearest neighbors from the large dictionary, which is identical to the database of patches, the super-resolution is performed via learning a regression function in the reproducing kernel Hilbert space. Comparing to other state-of-the-art super-resolution algorithms, the proposed method achieves better results, especially in case of heavy Poisson noise.

Introduction

This chapter handles the super-resolution problem to reconstruct a latent high-resolution image x ∈ R N from its degraded observation y ∈ R M , which is assumed to be generated by the following model (derived from (1.1)).

y = SBx + η (4.1)
where S is the downsampling operator with magnification factor s, B is the blur operator and η ∼ N (0, σ 2 ) is the additive Gaussian noise of zero mean and standard deviation σ.

In the context of our thesis, we investigate the patch-based approach in which an image y is considered as a set of overlapping patches {y i |y i ∈ R m , i = 1, . . . , M } and the super-resolution is performed on each patch y i by finding its latent high-resolution version x i . Without loss of generality, we can represent x i in a vector space created by the atoms of a dictionary of patches D h , denoted as x i = D h α i , with α i is called the representation coefficients vector. Under the Bayesian MAP perspective as shown in (1.5), the restoration of y i is corresponding to find the representation vector α i which satisfies: αi = arg min

α i 1 2 ||y i -S i B i D h α i || 2 2 -λ log p(α i ) (4.2)
where S i , B i denote the degradation factors on image patch y i .

In order to find the solution of (4.2), we need to know the prior model of the probability distribution p(α i ). In this work, we adopt the sparse model where p(α i ) is assumed to obey the Laplacian distribution, described as p(α i ) ∝ exp(-τ α i 1 ). In addition, we define D l = S i B i D h to be the dictionary of low-resolution patches, the problem in (4.2) is formulated as: αi = arg min

α i 1 2 ||y i -D l α i || 2 2 + λ α i 1 (4.3)
An important issue in our approach is to determine the couple of high-and lowresolution dictionaries {D h , D l }. In this chapter, we adopt the similar idea of chapter 3 by exploiting the redundancy of information in multiple images to construct a couple of of high-and low-resolution dictionaries identical to the large database of patches collected from a list of standard images. In addition, we acquire a large number of atoms (patches) in dictionaries to ensure that they can cover all the local structure of the degraded image y. However, one shortcoming of this approach is the computational challenge when solving (4.3) with a huge dictionary D l . To overcome this issue, the conventional solution is to apply a patch-filtering step to select a couple of local dictionaries {D h i , D l i } for each degraded patch y i , on which we can perform the patch recovering. Therefore, retrieving the local dictionaries is crucial in the success of our super-resolution method.

In the existing methods [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][START_REF] Chang | Super-resolution through neighbor embedding[END_REF], the common Euclidean distance is adopt for image patch selection.

As indicated in section 2.7 of chapter 2, the Earth Mover's Distance (EMD) developed by Pele et al. [START_REF] Pele | Fast and robust Earth Mover's Distances[END_REF], denoted as EM D, is an efficient cross-bin metric for measure the distance between two distributions or histograms. Hence, we consider that each image patch to be a histogram of gray levels placed at pixels' locations, where each pixel can be interpreted as a bin, and its gray level is the mass of this bin in the histogram. Therefore, in the statistical perspective, comparison between two image patches is equivalent to quantify the distance between two distributions and thus, can be calculated with the

EM D metric.
To demonstrate the effectiveness of the EM D for patch selection, we present in For further illustration, we report in Fig 4 .2 another example with natural image images The rest of this paper is organized as follows. Section 2 describes details of the proposed super-resolution method, referred to as SREMD, which makes use of the EM D for patch similarity measurement. The experimental results and comparison with some existing methods are reported in section 3. The conclusion and discussion are presented in section 4.

(a) (b) (c) (d) (e) (f) (g) (h)

Super-Resolution using Earth Movers Distance (SREMD)

The proposed super-resolution method (SREMD) consists of two main stages, including the construction of dictionary and the super-resolution.

Dictionary construction

In the first stage, we randomly extract a set of P high-resolution patches {x s k |x s k ∈ R n , k = 1, . . . , P } from a list of available standard images {x s }. The high-resolution dictionary is determined to be identical with the database of high-resolution patches

as D h = {d h k |d h k ≡ x s k , k = 1, . . . , P } ∈ R n×P .
After that, we degrade each atom in the high-resolution dictionary D h to obtain the corresponding low-resolution dictionary

D l = SBD h = {d l k |d l k ∈ R m , k = 1, . . . , P } ∈ R m×P .

Super-resolution

In the super-resolution stage, the degraded low-resolution image y ∈ R M is partitioned into a set of overlapping patches {y i |y i ∈ R m , i = 1, . . . , M } and the super-resolution is performed on each patch y i using (4.3). However, solving the sparse representation of the whole dictionary of large atoms is challenged. Therefore, we collect a couple

of local dictionaries {D h i ∈ R n×K , D l i ∈ R m×K } of K most similar atoms (K P )
retrieved form the large dictionaries for recovering each degraded patch y i . Hence, the problem (4.3) is rewritten as: αi = arg min

α i 1 2 ||y i -D l i α i || 2 2 + λ α i 1 (4.4)
With the solution of αi obtained from (4.4), we can estimate the latent highresolution image patches as:

xi = D h i αi (4.5)
A common approach for patch-filtering is to make use of the conventional bin-tobin distances such as the Euclidean metric ( 2 ) as proposed in many methods [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][START_REF] Chang | Super-resolution through neighbor embedding[END_REF] and in chapter 3 of this thesis. As indicated in previous section, the EM D suggests a remarkable solution in dealing with image patch selection, which can return query patches with higher perceptual similarity to the referenced patch than using the Euclidean measurement. Moreover, because the EM D is a cross-bin metric, it can address to the alignment and translation between image patches better than the 2 distance by providing the candidates with less translation to the referenced patch, and thus leads to improve the reconstruction of degraded image patch. images

In this work, instead of fixing the number of most similar patches K in the local dictionaries {D h i , D l i } as in chapter 3, we want to construct a couple of local dictionaries with dynamic number of atoms, such that the value of K depends on the local information of each patch y i , as well as the redundancy of y i in the large dictionary D l . Therefore, in the rest of this section, we introduce a threshold of the EM D between the underlying image patch y i and the atoms d l k in the dictionary D l , for automatically selecting the closest patches.

For each given low-resolution patch y i , we may not find atoms in the dictionary D l that exactly match to it, but we may retrieve its translated or rotated version. In the scope of this work, we just consider the case of subpixel shift. That means for each given patch y i in the observed image y, we expect to select the local dictionary D l i that contains the subpixel shifted versions of y i from the dictionary D l with the help of the EM D metric.

Let y i = [y i (1), . . . , y i (j), . . . , y i (m)] T be the low-resolution patch of m pixels, and

y sh i = y sh i ( 1
               m l=1 f jl ≤ y i (j), m j=1 f jl ≤ y sh i (l), m j=1 m l=1 f jl = min   m j=1 y i (j), m l=1 y sh i (l)   , f jl ≥ 0 (4.7)
where d jl = min (||j -l|| 2 , d max ) is the thresholded ground distance in 2 norm between the j-th pixel of y i and the l-th pixel of y sh i , and d max is a threshold of ground distance.

In order to dynamically select a couple of local dictionaries {D h i , D l i } for the degraded patch y i , we compute an upper bound (denoted as δ i sup ) of the EM D between y i and y sh i in (4.6). Thereby, any low-resolution atoms

d l k ∈ R m in the low-resolution dictionary D l such that EM D y i , d l k ≤ δ i sup is acquired.
An upper bound of the first term in (4.6) can be inferred in the perspective of transportation problem and is demonstrated in Fig. 4.3. For simplicity, we assume that the intensity y i (j) of the j-th pixel of y i corresponds to y i (j) units of masses. For a pixel j in the patch y i , suppose that y i (j) ≥ y sh i (j), we will move y sh i (j) units from pixel j of y i to the same location in y sh i , and the remaining (y i (j) -y sh i (j)) units will be moved by d M (with d M ≥ d max ) pixels away from j. The cost of this work is

y i (j) -y sh i (j) × d max .
As a result, the first term in (4.6) is bounded by: min

{f jl } m j=1 m l=1 f jl d jl ≤ m j=1 y i (j) -y sh i (j) × d max (4.8)
Now it is easy to get a margin for value of the EM D in (4.6):

EM D y i , y sh i ≤ 2 × m j=1 y i (j) -y sh i (j) × d max (4.9)
The upper bound δ i sup of the EM D is determined in the right side of (4.9) as:

δ i sup = 2 × m j=1 y i (j) -y sh i (j) × d max (4.10)
Consequently, we apply δ i sup as a threshold of the EM D to retrieve the local dictionaries {D h i , D l i } for each patch y i from the large couple of dictionaries {D h , D l } as

D l i = d l k |d l k ∈ D l ; k = 1, . . . , P ; EM D(y i , d l k ) ≤ δ i sup D h i = d h k |d h k ∈ D h ; k = 1, . . . , P ; EM D(y i , d l k ) ≤ δ i sup (4.11)
The couple of local dictionaries is now exploited to recover the degraded image patch y i by first solving the sparse representation problem in (4.4). In the scope of this work, we adopt the multiplicative updates algorithm proposed by Sha et al. [START_REF] Sha | Multiplicative Updates for Nonnegative Quadratic Programming in Support Vector Machines[END_REF] to optimize (4.4). With the optimal value of the representation coefficient vector αi , we images can estimate the high-resolution latent patch of y i , as in (4.5), xi = D h i αi . Moreover, the denoised version of y i can be obtained by ŷi = D l i αi .

{y i |y i ∈ R m , i = 1, . . . , M }.
2 foreach y i in image y do 3

Determine its shifted versions {y sh i } of p s pixel in 8 main directions.

4

Calculate the threshold δ i sup using (4.10) for each of 8 shifted patches y sh i and choose the maximum value of δ i sup .

5

Calculate the distance EM D(y i , d l k ) (k = 1, . . . , P ) to all P atoms of D l and select the couple of local dictionaries {D h i , D l i } according to (4.11).

6

Optimize (4.4) to get the representation coefficients vector αi . We use the iterative back-projection method [START_REF] Irani | Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency[END_REF] to optimize (4.12) as acquired 5 other images to generate the standard images. The SREMD is compared to above competing methods without the presence of Gaussian noise (σ is set to 0 in all test). For three methods NE, ScSR and SRSW, we use the same standard images as the proposed SREMD method to create the database of patches. Fig. 4.6 shows the evolution of PSNR with respect to the number of test images which proves the out-performance of our proposed method over the competing algorithms for the noiseless images.

X t+1 = X t + Ŷ -SBX t ↑ s * b (4.13) images (a) (b) (c) (d) (e) (f) (g) (h) (i) (j)

Effect of the regularization parameter λ

To demonstrate the effects of the parameter λ on the super-resolution performance of the proposed method, we conducted the experiments on CT image of chest (Fig. 4.4(i))

with magnification factor s = 2 for different noise levels (σ = 0, 10, 20). The values of λ is set in 0, 10, 50, 100, 250, 500, 1000, 1500, 2000, 5000, 10 4 , 5 • 10 4 , 10 5 . Fig. 4.7 presents the behavior of PSNR and SSIM on the variation of λ. We can observe that the range of λ that brings the best performances of the SREMD methods is around 1000, with all different noise levels. Therefore, the default value of λ is chosen as 1000 in the experiments in this paper. [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF](2)(3)(4)(5)[START_REF] Arbeláez | Contour Detection and Hierarchical Image Segmentation[END_REF][START_REF] Clark | The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository[END_REF][START_REF] Di | The autism brain imaging data exchange: towards a largescale evaluation of the intrinsic brain architecture in autism[END_REF][START_REF] Wang | Image Quality Assessment: From Error Visibility to Structural Similarity[END_REF][START_REF] Shapiro | Computer Vision[END_REF][START_REF] Keys | Cubic convolution interpolation for digital image processing[END_REF], CT images of abdomen [START_REF] Hou | Cubic splines for image interpolation and digital filtering[END_REF][START_REF] Tomasi | Bilateral filtering for gray and color images[END_REF], CT images of chest [START_REF] Perona | Scale-space and edge detection using anisotropic diffusion[END_REF][START_REF] Smith | SUSAN-A New Approach to Low Level Image Processing[END_REF][START_REF] Haykin | Least-mean-square adaptive filters[END_REF][START_REF] Takeda | Kernel Regression for Image Processing and Reconstruction[END_REF][START_REF] Li | New edge-directed interpolation[END_REF][START_REF] Zhang | An edge-guided image interpolation algorithm via directional filtering and data fusion[END_REF][START_REF] Rudin | Nonlinear Total Variation Based Noise Removal Algorithms[END_REF][START_REF] Donoho | Ideal spatial adaptation by wavelet shrinkage[END_REF][START_REF] Chang | Adaptive wavelet thresholding for image denoising and compression[END_REF] and CT images of lung [START_REF] Portilla | Image denoising using scale mixtures of Gaussians in the wavelet domain[END_REF][START_REF] Sendur | Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency[END_REF][START_REF] Chambolle | An Algorithm for Total Variation Minimization and Applications[END_REF][START_REF] Michael | On Semismooth Newton's Methods for Total Variation Minimization[END_REF][START_REF] Michael | Solving Constrained Totalvariation Image Restoration and Reconstruction Problems via Alternating Direction Methods[END_REF][START_REF] Getreuer | Rudin-Osher-Fatemi Total Variation Denoising using Split Bregman[END_REF][START_REF] Scherzer | Denoising with higher order derivatives of bounded variation and an application to parameter estimation[END_REF][START_REF] Chan | High-Order Total Variation-Based Image Restoration[END_REF].

Conclusion

In this chapter, we present an effective example-based super-resolution method which enhances the spatial resolution while robustly reducing the noise in a degraded lowresolution image. By exploiting the redundancy of local contents in multiple images, we construct a large couple of high-and low-resolution dictionaries which are identical to the database of patches collected from the standard images.

We have demonstrated that the cross-bin EM D metric is more efficient than the Determining a good prior model that can well characterize the statistical distribution of image in the vector space generated by the atoms of a dictionary is essential in recovering a degraded patch under the Bayesian maximum a posteriori perspective. In the sparsity approach, the prior model is often assumed to obey an arbitrarily chosen distribution and sometimes may fail to describe the true distribution of image patches. In this chapter, we motivate from the redundancy of local information between multiple images to assume that, in a vector space, the latent patches of an unknown image x share the same distribution with the patches in a database extracted from a list of standard images. Therefore, our aim is to justify that we can estimate a probability distribution function from the empirical distribution of image patches in the database and then use it as a prior to regularize the image restoration process. We introduce a simple histogram estimation scheme to represent the distribution of image patches in a low-dimensional vector space and apply it for image denoising. We demonstrate that using the estimated probability distribution as an image prior is more efficient than the arbitrary sparsity models

Introduction

In two previous chapters, we have exploited the external similarity and redundancy between multiple images in solving the restoration of a degraded image y in the patchbased framework. By building an exhaustive dictionary of image patches which is identical to the large database of patches, the reconstruction was performed on each patch y i of the underlying image y by firstly applying a patch-filtering step on the dictionary for selecting a smaller set of similar candidates (also called a local dictionary).

In the next stage, the latent image patch x i (high quality version of y i ) was recovered in the hypothesis of sparse representation in which the distribution of image patches is assumed to follow a prior mathematical model such as the Laplacian distribution.

There are some shortcomings in the aforementioned approach. Firstly, utilizing a large dictionary requires a pre-filtering step for selecting a local dictionary that can slow down the reconstruction process. Secondly, the choice of sparse model in restoring an image patch under the given local dictionary is an arbitrary decision and may not reflect the true distribution of image patches.

In this chapter, we propose to use a unique global dictionary D for recovering all patches y i in the degraded image y, which can be learned from the image itself or from the database of patches (collected from the standard images). Additionally, we concentrate on investigating the distribution of image patches in the vector space generated by the dictionary D. In the next section, we will demonstrate that the sparsity models may not exactly represent the true distribution of image patches. Additionally, the main objective of this chapter is to justify that the probability distribution function learned from the database of patches can be employed as a prior to regularize the optimization process, leading to improvements in image reconstruction comparing to an arbitrary selection of sparse models. For persuasive verification, especially in the scope of this chapter, we study the application of estimation of probability distribution for solving the image denoising problem by conducting several experiments on different image modalities

The remainder of this chapter is organized as follows. Section 2 briefly describes the context of image denoising and introduces our motivation of estimation of probability distribution from the database of patches. In section 3, we expose how to employ the estimated probability function in solving the image denoising problem. After that, we present some denoising applications of the proposed method, which is referred to as ProbaEst, including the case of synthetic images, binary images, and popular natural test images. This chapter will be ended with some discussion and perspective presented

Problem Statement and Motivation

Image denoising is a fundamental task in low level vision. In this work, we tackle a simple and widely used degradation model (derived from (1.1) with H is the identity matrix), where an image x ∈ R N is assumed to be corrupted by additive Gaussian noise η ∼ N (0, σ 2 ) of mean zero and standard deviation σ as shown in (5.1)

y = x + η (5.1)
In the context of this thesis, we focus on the patch-based image reconstruction in which each image is considered as a set of overlapping patches, and the denoising will be performed on each patch. We can formulate the problem in (5.1) as:

y i = x i + η i (5.2)
where

y i ∈ R n is the i-th (1 ≤ i ≤ N ) patch in the noisy image y, x i ∈ R n is the latent
clean version of y i and η i is the residual noise in y i .

Among various contribution on image patch-based restoration, the dictionary learning approaches [2,3,[START_REF] Mallat | Matching pursuits with time-frequency dictionaries[END_REF][START_REF] Elad | Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries[END_REF][START_REF] Zeyde | On Single Image Scale-Up Using Sparse-Representations[END_REF] have been receiving noticeable attention in the last decade.

The aim of these methods is to seek a set of K basis vectors (also referred to as atoms) {d 1 , . . . , d j , . . . , d K |d j ∈ R n , 1 ≤ j ≤ K} to form a dictionary D ∈ R n×K such that the clean image patch x i can be described by a linear combination of atoms {d j } in the dictionary. That means, x i = Dα i , where α i ∈ R K is called the representation coefficients vector. Let Ω α ⊂ R K be a vector space generated by K atoms of the dictionary D. Then each image patch x i can be represented by the vector of coefficients α i in Ω α . Therefore, rather than modeling the distribution of image patches, we can analyze the distribution of representation coefficients in the vector space Ω α to extract useful prior information to regularize the image denoising problem.

Under the Bayesian MAP perspective, the denoising task is equivalent to find a representation vector αi which maximizes the posterior conditional probability: αi = arg max

α i p(α i |D, y i ) = arg max α i {p(y i |D, α i )p(α i )} = arg min α i ||y i -Dα i || 2 2 -λ log p(α i ) = arg min α i ||y i -Dα i || 2 2 + λΦ(α i ) (5.3)
Where p(α i ) is called the prior model of image patches since it specifies a priori statistical features of representation coefficients of patches in the vector space Ω α and is

p(α i ) in Ω α .
As mentioned in section 2.6 in chapter 2, a data-adaptive dictionary, e.g. the K-SVD, has demonstrated its out-performances comparing to fixed analytic dictionaries in restoring a noisy image. In this chapter, we construct a dictionary D such that it contains similar local patterns in the images by observing the structure of images or adopting an existing dictionary training method such as the K-SVD.

With a given dictionary D, determining a good prior model of p(α i ) in the vector space Ω α , which is expected to be identical to the real distribution of representation coefficients of patches {x i } in the latent image x, is the key issue in the success of an image denoising algorithm. A conventional approach in the literature is the sparsity model [2,[START_REF] Elad | Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries[END_REF][START_REF] Kreutz-Delgado | Dictionary Learning Algorithms for Sparse Representation[END_REF][START_REF] Olshausen | Sparse coding with an overcomplete basis set: A strategy employed by V1?[END_REF]. The principal idea is to assume that the representation coefficients of image patches in the vector space Ω α to be sparsely distributed and can be characterized in a smaller dimensional subspace, e.g. p(α i ) ∝ exp(-λ||α i || 0 ), with the 0 pseudo-norm counts the number of non-zero elements in α i . Many authors [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF][START_REF] Mallat | Matching pursuits with time-frequency dictionaries[END_REF][START_REF] Xu | L 1/2 Regularization: A Thresholding Representation Theory and a Fast Solver[END_REF][START_REF] Jia | Image denoising using hyper-Laplacian priors and gradient histogram preservation model[END_REF] observed that the distribution of image patches in the vector space Ω α exhibit heavy-tailed forms and proposed some mathematical models such as Laplacian (Φ(α i ) ∝ ||α i || 1 ), hyper-Laplacian (Φ(α i ) ∝ ||α i || p , with 0 < p < 1) to approximately describe the distributions of patches.

There are two shortcomings of the sparsity priors. Firstly, the choice of a sparse model in a denoising algorithm is an arbitrary decision, which can be either non-convex models such as 0 -norm, p -norm (0 < p < 1) or a convex relaxation 1 model. Secondly, the selected sparse model may not well portray the true distribution of image patches in the vector space Ω α generated by K atoms of the dictionary D. It is straightforwardly recognized that a prior model which can accurately express the true distribution p(α i ) of image patches may enhance the reconstruction of the degraded image. However, the true distribution p(α i ) of representation coefficients of patches in the latent clean image x is underdetermined in practice. Therefore, introducing a prior model that can accurately approximate the true distribution of p(α i ) is essential to improve the denoising performance.

On the purpose of studying a more consistent and reliable prior model involving to the distribution of representation coefficients of image patches in the vector space Ω α , we explored the external similarity property of images to learn the model of the probability distribution p(α i ) from the standard clean images. More specifically, due to For better comprehensive understanding the statistical property of distribution of representation coefficients of patches in the vector space Ω α , we carry out the following experiment on image patches of the database extracted from standard images {x s }. The dictionary D is prepared as follows. We use the image of Boat shown in Fig. 2.1(f) as an example. First, we add the Gaussian noise of mean zero and standard deviration σ = 20 to the image. Then, we randomly extract a set of 100000 image patches of size 8 × 8 pixels (n = 64) from the noisy image and adopt the K-SVD algorithm, as described in subsection 2.5 of chapter 2, to learn an overcomplete dictionary D ∈ R 64×256 of K = 256 atoms.

To create the list of standard images {x s }, we collected 24 high-quality noise free images from the Kodak photoCD dataset (see Fig.

α s k ||x s k -Dα s k || 2 2 = (D T D) -1 D T x s k = P D x s k ( 5.4)
where P D = (D T D) -1 D T is the projection matrix. Hence, with a database of patches in the image domain, we obtain a distribution p(α s k ) of representation coefficients vectors in Ω α .

Let A = [α s 1 , . . . , α s k , . . . , α s 200000 ] ∈ R 256×200000
denote the ensemble of all representation vectors. Thus, the j-th row of A coincides with the representation coefficients {α s k (j)} of image patches in the database on the j-th atom d j of the dictionary D. In order to facilitate the observation of statistical property of the real distribution p(α s k ) of image patches in the database, we plot (in log domain) the distribution of the 128-th row of matrix A in green curve in Fig. 5 Histogram is a simple yet effective tool that gives a rough sense of the density of an underlying distribution of the data and can be used to estimate its probability distribution function. In our work, we propose a median-based binning approach to three-dimensional (3D) histogram for estimating the empirical distribution of patches.

However, building a histogram for the distribution p(α s k ) in the whole vector space Ω α is a complex problem and can be broken down into smaller and simpler steps by determining 3 one-dimensional (1D) histograms according to 3 atoms in the dictionary. Hence, the distribution of patches along the j-th coordinate, denoted as p(α s k (j)), of the vector space Ω α is roughly approximated by a piecewise constant function of a 1D histogram. Finally, the estimated 3D histogram p(α i ) of the real probability distribution p(α s k ) becomes a cube-wise constant function whose values are generated from the three 1D histograms by computing the density of patches simultaneously falling into each cube made of three 1D bins. As a results, the empirical distribution p(α s k ) of representation coefficients of patches in the database is approximated by a grid of cubes {Ω c α } such that every patches α i stored in the same cube Ω c α share an identical probability of occurrence, as defined in (5.5).

p(α i ) =        1 Volume of Ω c α × Number of patches in Ω c α Number of patches in Ω α = 1 V c α × P c P , if α i ∈ Ω c α 0, otherwise (5.5)
where P c is the number of patches falls inside the cube Ω c α which contains α i , V c α is the volume of Ω c α (as Ω α is the 3D vector space), P is the total number of patches in the database.

A major question in estimating the histogram p(α i ) is how to determine the size of each cube Ω c α such that p(α i ) can accurately represent the empirical distribution p(α s k ). To this end, we consider to establish a 1D histogram for each dimension j of the space Ω α , which coincides with the representation coefficients α s k (j) on the j-th atom of the dictionary, by dividing the entire range of values of α s k (j) into a series of B non-overlapping intervals (also known as bins) and counting how many values falling into each bin. Practically, the choice of bin width, which is commonly inversely proportional to the number of bin B over the range, is often a little arbitrary. In a simply conventional proposal, the range limited by the minimum and maximum values of α s k (j) in the j-th dimension is split into B equal-width intervals. Despite the straightforwardness, a shortcoming of equal-size bin division is the inflexibility in describing the density of a distribution. Moreover, to precisely characterize a true distribution, we need to divide the range into a large number of bins B, which causes an increase in the complexity of estimation algorithm.

To cope with this issue, we proposed a median-based division strategy in which each axis of Ω α is split in such a way that each interval contains the same number of patches. Hence, the width of each bin can adapt to the density of the values of representation coefficients α s k (j). That means, for an interval of the j-th coordinate of Ω α with high density of patches, its length will be set to small, and so on. In the experiments, each dimension of the vector space Ω α is separated into B intervals by B -1 median points using a recursive division, so that each bin contains P/B points.

As a result, we obtain a grid of (B -1) 3 quantiles G α = {α q l |l = 1, . . . , (B -1) 3 } in the R 3 space Ω α . In Fig. 5.4, we demonstrate an example of distribution of points in Ω α which is partitioned in 4 × 4 × 4 (with B = 4, K = 3) cubes.

Using the proposed approach, we can produce a good estimation of the histogram p(α i ) to portray the true empirical distribution p(α s k ) of representation coefficients of patches in the database with only a small number of bins in each dimension of the vector space Ω α . To illustration, we consider an example on the image of Boat. We generate a noisy image by adding Gaussian noise of mean zero and standard deviation of 20. After that, we randomly extract a set of 100000 patches of size 3×3 pixels to train a dictionary of K = 3 atoms, thanks to the K-SVD algorithm. We also collect P = 200000 patches in the 24 standard images of the Kodak dataset and project them into the vector space Ω α of the dictionary. Fig. 5.5 shows (in green curve) the empirical distribution p(α s k (1)) of the representation coefficients of patches in the database corresponding to the first atom of the dictionary, as well as two estimated histograms using equal-width bin division (in red dash lines) and median-based split (in black dash lines). The number of intervals (bins) in each dimension of Ω α is set to B = 32 for two methods. It can be observed that in the median-based bin division, we only need to partition the coordinate into B = 32 bins to get a histogram that is very approximate to the true distribution of patches. On the other hand, the histogram obtained by the conventional equal-bin division method with the same number of bin B fails to match the distribution of patches. Therefore, in

Solving the optimization problem

In this section, we will present how to recover a clean image patch x i from its noisy version y i , given the same dictionary D ∈ R n×3 in section 3.1 and the prior knowledge of probability distribution p(α i ) estimated from the database of patches.

The denoised patch can be generated as a linear combination of atoms in the dictionary, xi = D αi . Our objective is equivalent to find the value of αi which satisfies the MAP optimization problem in (5.3): αi = arg min

α i ∈Ω α ||y i -Dα i || 2 2 -λ log p(α i ) = arg min α i ∈Ω α J(α i ) (5.6)
Where J(α i ) = ||y i -Dα i || 2 2 -λ log p(α i ) is called the cost function, λ > 0 is a regularization parameter.

As indicated in section 3.1, we have divided the vector space Ω α into disjoint cubes Sparsity Prior {Ω c α }, and the value of p(α i ) is constant in each cube Ω c α and is determined as in (5.5). Consequently, the cost function J(α i ) in (5.6) becomes a convex piecewise quadratic function. A simple way to handle the problem (5.6) is the exhaustive solution that minimizes J(α i ) on each 3D cube Ω c α and then select the best value αi from which we achieve the smallest cost. Therefore, we can rewrite (5.6) as: αi = arg min

Ω c α ⊂Ω α arg min α i ∈Ω c α J(α i ) = arg min Ω c α ⊂Ω α J c (α i ; Ω c α ) (5.7)
As defined in (5.5), the value of p(α i ) is equal to zero when α i is outside a cube Ω c α , and receive a constant value for every point α i ∈ Ω c α . Thus, the minimum cost function on Ω c α , denoted as J c (α i ; Ω c α ), becomes:

J c (α i ; Ω c α ) = arg min α i ∈Ω c α J(α i ) = arg min α i ∈Ω c α ||y i -Dα i || 2 2 -λ log P c V c α P (5.8)
J c (α i ; Ω c α ) turns into the minimization of a convex quadratic function J(α i ) on a subspace Ω c α . As a result, the optimized value of J c (α i ; Ω c α ) occurs at either a point inside or one of the eight corner of Ω c α . We will prove as follows.

Suppose that there exists a cube

Ω d α such that J c (α i ; Ω d α ) = arg min α i ∈Ω d α ||y i - Dα i || 2 2 -λ log N d V d
α N gets the minimum at α * i = (D T D) -1 D T y i , and

α * i ∈ Ω d α .
Hence, for all other cubes Ω c α Ω d α , the minimum of cost function on Ω c α is determined as:

J c (α i ; Ω c α ) = arg min α i ∈Ω c α ,α i Ω d α ||y i -Dα i || 2 2 -λ log P c V c α P
(5.9)

Because the root of the first derivative ∂J c (α i ;Ω c α ) ∂α i only occurs at α * i = (D T D) -1 D T y i Ω c α , we always have:

         ∂J c (α i ; Ω c α ) ∂α i - → 0 ∀α i ∈ Ω c α ∂ 2 J c (α i ; Ω c α ) ∂α 2 i = D T D > 0 : is a positive definite matrix (5.10)
Therefore, the value of the cost function J(α i ) in a cube Ω c α always increase as we go from one corner to another corner of Ω c α . Consequently, the optimized value of (5.9) is only obtained at one of the corners of Ω c α . As a result, the restoration of each noisy image patch y i is attained via minimizing the cost function J(α i ) in (5.6) under the constraint of cube-wise constant function of the probability distribution p(α i ).

Moreover, we have partitioned the vector space Ω α into B 3 rectangular bins by grid of (B -1) 3 quantiles G α . Therefore, these points becomes the corners of cubes Ω c α , and

Denoising of vertical structure images

We first evaluate the denoising performance of our proposed method ProbaEst with two sparse models OPM and LARS on simple vertical structure images. We generate In our experiments, we observe that the results of denoising are highly dependent on the choice of dictionary D. In regard to our task, where all images are composed of vertical stripes, a dictionary with vertical structural atoms can be more adapted to the image structure. Therefore, we select a dictionary with the vertical structure as exposed in Fig. 5.6(f)-(h). Tables 5.1 and 5.2 provide the results of denoising in terms of PSNR and SSIM, where the best values are in bold red numbers. We can observe that for small noise corruption (σ = 10), the LARS algorithm (5.14) yields better values of PSNR, but our proposed method ProbaEst achieves higher SSIM performances. For qualitative illustration, we present in Fig. 5.7 the denoising results of image test with noise level σ = 10. Although the LARS algorithm attains the highest PSNR value, the proposed method restores a more pleasant visual image with less vertical artifacts than the LARS and OMP method. Moreover, it can be seen that with heavier noise levels (σ = 20, 30), the proposed method is superior to the sparse models, with the improvements are 0.4 ∼ 1.4 dB of PSNR and 0.05 ∼ 0.12 of SSIM. For visual assessment, Figures 5.8

Denoising of binary images

In this part, we will not only compare the performance of the proposed method with the sparse patch-based algorithms OMP and LARS, but also with others denoising methods developed for binary images, called Iterated Conditional Mode (ICM) [START_REF] Besag | On the statistical analysis of dirty pictures[END_REF] and

Graph Cuts [START_REF] Kolmogorov | What energy functions can be minimized via graph cuts?[END_REF]. The ICM method is based on the maximization of local conditional probabilities, which begins from some initial assignment of states to pixels (0 or 1), then cycles through the pixels, greedily maximizes the potential of each pixel given its neighbors until no further local improvements are possible. While in the latter method, we construct a graph of nodes (sources and sinks) from input image and optimize the energy function by maximizing amount of flow passing from the source to the sink.

In the experiments, all images are converted into binary versions using a simple 100 Chapter 5. Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior thresholding method as a pre-processing step. We used all the 24 images in the Kodak PhotoCD Dataset (Fig. 5.1) as standard images and 5 natural images of Baby, House, Lena, Monarch and Peppers shown in Fig. 2.1 are adopted for the evaluation. The dictionary is chosen as the same as Fig. For further illustration, Fig. 5.11, Fig. 5.12 and Fig. 5.13 show the denoising results of competitive methods. In Fig. 5.11, the ICM, Graph Cuts and proposed method ProbaEst achieve higher noise-reduction than the sparsity ones, but we have lost some small details on the beanie. The results of ICM and ProbaEst are very close and similar with the original image, while the Graph Cuts discards some black regions around the eyes in the image. In the contrary, the OMP and LARS visually conserve more details on the beanie, but they produce much more noise on the face, as well as the vertical artifacts within the image.

In addition, Fig. 5.12 and Fig. 5.13 display the denoising results of binary image of monarch and peppers, respectively. As can be observed, proposed method is more efficient in noise reduction than the sparsity (OMP, LARS) and the ICM. More particularly, the OMP and LARS generate images with vertical artifacts, while the Graph In reality, with a simple dictionary of three atoms, the denoising results will not be good enough for visual quality requirement. But we have to emphasize again that the objective of our research is to demonstrate that using the estimated probability distribution as the image prior is more efficient than a sparsity model for noise removal.

As illustrated in Tables 5.5 and 5.6, the proposed method yields better values of PSNR and SSIM than the sparsity algorithms in most of cases of high noise levels (σ = 20, 30). Particularly, with σ = 30, the improvements are around 0.3 to 1.2 dB on average in term of PSNR.

For visual assessment, Fig. 5.14, 5.15, 5.16 show the denoising results of competing methods on image of Peppers, Airplane and Baby, respectively. Globally, the three methods produce acceptably visual results. For better evaluation, we display the zooming of some regions in the images. It can be observed that the proposed method ProbaEst demonstrates its effectiveness in reconstruction of degraded images with less artifacts than the sparsity models.

Conclusion

This chapter gives some significant remarks on the use of sparse model to describe the priori knowledge of distribution of representation coefficients of latent image patches, in the perspective of Bayesian MAP. First, the choice of an available sparse model in a We have demonstrated that with a small dimensional vector space (or a dictionary contains up to 3 atoms), we can produce an accurate estimation of the true distribution of representation coefficients of image patches in the database by constructing a histogram with piecewise constant functions. An efficient median-based division approach has been proposed to precisely model the distribution of image patches with small number of bins of histogram in each dimension of the vector space. We also introduce a framework that takes into account the estimation-based probability distribution in reconstruction of a noisy image. By exploring experiments on multiple type of images, we have proved that using estimated prior from the distribution of patches in the database can improve the performance of a denoising algorithm, rather than adopting an arbitrary sparse model.

However, the proposed approach with piecewise constant estimation of probability distribution is inefficient for high dimensional vector space, which corresponds to a large dictionary in practice, due to the curse of dimensionality and the high computational complexity. To deal with this problem, we can use a kernel density estimator such as the Gaussian Mixture Model (GMM) to learn the prior model of probability distribution function from the distribution of external patches in high-dimensional vector space. We will expand the proposed method, as well as discuss the use of GMM in solving the denoising optimization in the next chapter.

6

C h a p t e r

Number of Useful Components in Gaussian Mixture

Models for Patch-based Image Denoising

In the previous work, we showed that estimating the actual probability distribution of image patches improves image reconstruction. However our estimation method is limited to low dimensional dictionaries, so we now consider Gaussian Mixture Models (GMMs) for better representing the prior. However, when using GMMs as a prior for image denoising under the Bayesian maximum a posteriori (MAP) perspective, only a single prominent Gaussian component is usually selected to recover a noisy image patch, which leads to computationally efficient implementations. In this chapter, we attempt to justify this on several image datasets by evaluating the number of Gaussian components required for recovering patches. We show that even patches without a prominent component in the prior can be recovered with little loss of performance. Comparisons between two dictionary choices and between small and large models suggest that large gains are attainable, but only one component is required for reconstruction. A summary of our work has been submitted in the 2018 International Conference on Image and Signal Processing (ICISP1 ).

Abstract 1 Introduction

The initial results of the proposed method in chapter 5 have validated the successfulness of using the estimation of empirical distribution of patches in the database to recover a degraded image, rather than adopting an arbitrarily available sparse model. However, the developed framework using histogram to estimate the distribution p(α i ) of patches has limitation in dealing with high-dimensional vector space Ω α ⊂ R K , due to the curse of dimensionality where a large portion of number of hypercube bins may probably be empty. Moreover, with a large dictionary K 3, the complexity of histogram estimation algorithm dramatically increases and makes the implementation impossible. Therefore, in denoising applications presented in chapter 5, the number of atoms in the dictionary D is chosen really small with K = 3, which corresponds to a three-dimensional vector space Ω α . However, in practical applications, a complete dictionary (K = n) or an over-complete dictionary (K > n) has demonstrated its out-performance for image restoration comparing to a small dictionary.

One representative solution for dealing with estimation of probability in highdimensional vector space Ω α is the use of parametric kernel density estimators. Among various contributions, the Gaussian Mixture Model (GMM) has been commonly exploited in representing the distribution of data in a high-dimensional space and successfully employed in image restoration [5,[START_REF] Zoran | Natural Images, Gaussian Mixtures and Dead Leaves[END_REF][START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF][START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF][START_REF] Teodoro | Image restoration with locally selected class-adapted models[END_REF]. The fundamental of these approaches is to estimate the empirical distribution of patches in the database by a linear combination (often known as a mixture) of finite number of M Gaussian distributions. However, when employing the GMM as a prior model for solving image denoising under the Bayesian MAP perspective in (5.3), only a single prominent Gaussian component is usually selected to represent the probability distribution p(α i ), which leads to computationally efficient implementations.

In this chapter, we explore the Gaussian mixture model to estimate the distribution p(α s k ) of the representation coefficients of patches in the database and use it as an image prior to represent the probability distribution p(α i ) of patches in the latent image. A questionable issue is how many useful components in GMM should be used to describe the probability distribution p(α i ) and thus reconstruct a noisy patch. Can we apply all M Gaussian components or only one prominent component is enough? To our knowledge, justification for this approach is lacking in the literature. Therefore, earlier than making comparison the denoising performance of the proposed method with other competing sparsity models as in chapter 5, we concentrate in this chapter to verify this scientific question on several image datasets by evaluating the number of The experimental results on a combination of two dictionary choices and two model complexities of the GMM are shown in section 5. This chapter will be ended with some discussion and perspective in section 6.

Problem Statement and Motivation

In this work, we consider the Gaussian Mixture Model (GMM) for distribution estimation.

The fundamental concept is to suppose that each representation coefficients vector α s k of patches in the database is drawn from a mixture of M Gaussian distributions of unknown parameters {µ m , Σ m } as described in (6.1)

p(α s k ) = M m=1 π m N (α s k |µ m , Σ m ) (6.1)
Where {π m |m = 1, . . . , M } are the mixing weights with M m=1 π m = 1, µ m ∈ R K and Σ m ∈ R K×K are the mean and covariance matrix of the m-th Gaussian component, respectively. N (α s k |µ m , Σ m ) is the Gaussian distribution defined in (6.2)

N (α s k |µ m , Σ m ) = 1 2π|Σ m | exp - 1 2 (α s k -µ m ) T Σ -1 m (α s k -µ m ) (6.2)
where |Σ m | is the determinant of Σ m .

In order to demonstrate the benefit of using GMM in approximately represent the distribution of patches, we study some simulations on the image of Boat shown in estimation as in (5.4). We examined the distribution of representation coefficients α s k [START_REF] Rubinstein | Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation[END_REF] along the 128-th dimension of the vector space Ω α , which corresponds to the weight of the 128-th atom of the dictionary. From 200000 values of {α s k (128)|k = 1, . . . , 200000}, we learned a GMM model that approximates the distribution p(α s k (128)). Optimization of (6.4) is a very time-consuming process and pose a huge challenge in implementation of denoising algorithm.

Using GMM is not a new approach in image restoration with several publications have been reported in the literature [5,[START_REF] Zoran | Natural Images, Gaussian Mixtures and Dead Leaves[END_REF][START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Wang | SURE Guided Gaussian Mixture Image Denoising[END_REF][START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF][START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF][START_REF] Lu | Image-Specific Prior Adaptation for Denoising[END_REF]. Typically, a GMM model was learned from the distribution of image patches in the database. However, a really difficult obstacle that the researchers have to face is how to solve the optimization with the given GMM model under the assumption of Bayesian MAP. A conventional solution proposed in the existing works [5,[START_REF] Zoran | Natural Images, Gaussian Mixtures and Dead Leaves[END_REF][START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Wang | SURE Guided Gaussian Mixture Image Denoising[END_REF][START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF] is to select only one prominent Gaussian among M distribution of the GMM to represent the prior model of p(α i ) of each noisy patch y i . Thus, the optimization turns into a convex quadratic problem and can be easily solved by a close-form formula. In [START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF], the author first divided the degraded image y into M groups of similar patches, and for each group, a Gaussian distribution was estimated to model the distribution of image patches.

To our knowledge, existing GMM-based denoising methods only used one prominent component of the GMM for representing the probability for each image patch, which leads to computationally efficient implementations. However, a questionable issue is using only one dominant component from M components of the GMM is enough to produce a good estimation of the latent image patch x i ? In other words, may we improve the reconstruction performance when exploiting more Gaussian components to represent the prior distribution of latent patches? Additionally, what is the useful number of Gaussian components should be used for denoising an image patch y i ? Unfortunately, the justification for these issues is lacking in the state-of-the-art.

The proposed denoising model in this work is the development of our framework introduced in chapter 5 by investigating the GMM for estimating distribution of representation coefficients of image patches in case of high-dimensional vector space Ω α (which corresponds to a large dictionary K 3). Optimization of denoising problem (6.4) with GMM prior is an important step in our proposed method. In this chapter, our motivation is to evaluate the useful number of Gaussian components using to recover 3. Datasets 115 a noisy image patch. To this end, we divide the patches y i in an input image into a set N 1 of simple patches with a prominent component and a set N 2 of the remaining patches. We focus on the set N 2 and conduct multiple experiments to show that only marginal gains can be obtained by considering the full GMM in denoising. We explore different choices of dictionary (identity matrix and K-SVD based) and two choices of GMM complexity on PSNR and reconstruction error and discuss the type of images that are difficult to reconstruct.

Datasets

We conduct the experiments on denoising on 8 different datasets with different image types and structures. Fig. 6.2 shows some example images of these datasets. For each dataset, we collect two samples of images, one is used for training the GMM models, and the other is adopted for the validation. We briefly present these datasets as follows:

Cartoon [START_REF] Shahbaz Khan | Color attributes for object detection[END_REF] contains 590 images of popular cartoon characters. We choose 45 images to train a GMM and 80 images for evaluation.

Urban

Image denoising with a Gaussian mixture model

In this section, we present the GMM-based image denoising method (referred to as GPID) that takes into account the GMM model learned from the representation coefficients of external patches as an image prior to regularize the denoising problem. We will demonstrate how to learn the GMM from a set of standard images {x s } in section 4.1.

After that, we introduce a framework that evaluates the number of Gaussian components used in solving the optimization problem in section 4.2.

Training the GMM on a patch database

From the training set of good quality noise-free images x s , we randomly extract P The problem in (6.6) can be solved using the iterative Expectation-Maximization (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] via two alternative steps. In the E-step, with given values of Θ, a hidden variable γ km (also called the "membership probability") which relates to the probability of α s k belonging to the m-th (1 ≤ m ≤ M ) Gaussian components is computed as in (6.7).

γ km = π m N (α s k |µ m , Σ m ) M l=1 π l N (α s k |µ l , Σ l ) (6.7)
In the M-step, the parameters Θ of the GMM are updated using (6.

Denoising algorithm

As indicated in section 2, we can adopt the GMM model learned from distribution of external images as an image prior to characterize the probability distribution of representation coefficients p(α i ) of a latent clean image x i . That means p(α i = M m=1 π m N (α i |µ m , Σ m )). After mean substraction, denoising a patch y i is equivalent to finding the optimal representation coefficients vector αi in (6.4) such that the clean latent patch can be estimated xi = D αi .

However, solving problem (6.4) with the whole GMM of p(α i ) is a very timeconsuming process. To overcome this issue, existing studies [5,[START_REF] Zoran | Natural Images, Gaussian Mixtures and Dead Leaves[END_REF][START_REF] Yu | Solving Inverse Problems With Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity[END_REF][START_REF] Niknejad | Image Restoration Using Gaussian Mixture Models With Spatially Constrained Patch Clustering[END_REF][START_REF] Xu | Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising[END_REF] proposed to assign the noisy patch y i to a single Gaussian component. The selection can be done according to maximum of posterior probability γ im that y i belongs to the m-th Gaussian component. To determine the probability γ im , we recall that the noise model is formulated on each image patch as y i = Dα i + η i , with η i ∼ N (0, σ 2 ). Thus if we suppose that α i is drawn from the m-th Gaussian distribution with mean µ m and covariance matrix Σ m , as well as α i and η i are two independent variables, then y i will belong to another Gaussian distribution with the corresponding mean Dµ m and covariance matrix DΣ m D T + σ 2 I, with I ∈ R K×K is an identity matrix. Therefore, the posteriori probability γ im is: Regularize the denoised image: X (t) = (ηY (t-1) + βX t )/(η + β).

12 end

Complexity Analysis

The denoising method GPID consists of two parts: off-line training and denoising.

In the training phase, a GMM of M components are learned from a set of P representation coefficients vector {α s k |α s k ∈ R K , k = 1, . . . , P }. In the E-step, computing the "membership probability" in (6.7) requires O(K 3 P M ) operators. In the M-step, the total complexity of updating the parameters of the GMM is O(K 2 P M ). Therefore, the overall complexity of training the GMM is O(K 3 P M ).

In the denoising process, N 1 patches in the noisy image are restored via (6.

Dictionary choice and model complexity

Using the denoising results from the 8 datasets with L = 1, we compare the two dictionaries and GMM sizes in Figure 6.6 (see also the examples in Figure 6.3). We observe that increasing the GMM model complexity is nearly always beneficial, sometimes up to 2dB PSNR gains, and that the K-SVD dictionary tends to benefit more from M = 200. The K-SVD dictionary yields slightly better PSNR especially for large GMM models overall, but the results are variable, which implies that dictionary choice is largely image-specific.

Conclusion

In a high-dimensional vector space generated by a large given dictionary, we can learn a GMM to approximately represent the distribution of patches in the database and then apply it as a prior model to denoise a degraded image.

However, rather than focus on comparing the performance of the proposed method with other arbitrarily chosen sparse models, our main objective, in this chapter, is to error (gray-level differences) in all datasets when using more than one component.

This verifies the current practice and drastically reduces computational cost. We also demonstrated that much larger improvements can be obtained with a suitable dictionary and GMM model, but reconstruction only requires a single component.

Prior construction. Our intuition suggests that the prior should be constructed from standard images related to the input image, but it is unclear how specific this needs to be. Current approaches in deep learning and transfer learning suggest that images share common properties and that these are reflected in the prior distribution. In the near future, we would like to compare the dictionaries constructed for different types of images as in Chapter 6 to evaluate if there exists a universal prior or which kind of modifications are required to adapt to different contexts.

Locality. In Chapters 3 and 4, we used filtering to construct locally-adapted subdictionaries. The results in Chapter 6 also suggest that the prior distribution is locally simple. Following the insight from Nonlocal Self Similarity methods, would the input image itself provide enough information for estimating a local patch prior? Can this information be combined with a generic prior estimated from an image database?

  noise of mean zero and standard deviation σ. Depending on different settings of H, the model (1.1) can represent various IR problems. For example, in image denoising, H is an identity matrix. For image deblurring, the matrix multiplication operation Hx is equivalent to a two-dimensional convolution b ⊗ x between a blur kernel b and the latent image x. By selecting a diagonal matrix of H with the entries are either 0 or 1 where the zero values denote the missing pixels in the image x, (1.1) becomes the image inpainting issue. Another popular configuration is the super-resolution (SR) problem where H = SB is a composition of a blurring B and a downsampling S operators.

  probability p(α i |y i , D) as shown in αi = arg max α i p(α i |y i , D) = arg max α i {p(y i |D, α i )p(α i )} (1.4)

( a )

 a Image of Peppers (b) Image of Boat

Figure 1 . 1 :

 11 Figure 1.1: Illustration of external similarity between multiple images. Referenced patches in images are marked by the rose blocks with red contours, and a few of their matched patches selected from standard images are denoted by blue contour blocks.

Figure 1 . 1 :

 11 Figure 1.1: Illustration of external similarity between multiple images. Referenced patches in images are marked by the rose blocks with red contours, and a few of their matched patches selected from standard images are denoted by blue contour blocks.

  , and c 2 = 2c 3 = (k 2 L) 2 , with L denote the dynamic range of the pixel values (L = 255 for 8-bit images), k 1 = 0.01 and k 2 = 0.03 are two constants determined by the authors to avoid instability in homogeneous regions. Finally, Wang et al. get the specific form of the SSIM between two corresponding patches in two images as:

(chapter 5 )

 5 or the Gaussian mixture model (chapter 6) to estimate a prior image model from the database of patches and then adopt them to restore a degraded image. The 1. State-of-the-art Image Restoration 23 external nonlocal similarity and sparsity are the core ideas of our work.

Chapter 2 .

 2 State-of-the-art image restoration et al. proposed an interesting image restoration model named nonlocally centralized sparse representation (NCSR). Based on the NSS prior, Dong suggested a scheme to estimate the sparse coefficients of the original image and defined a sparse coding noise regularization (in 1 -form), which is the difference between those estimates and the sparse coefficients of the observed image, to improve the performance of sparsity-based image restoration.

  [START_REF] Tomasi | Bilateral filtering for gray and color images[END_REF] where F(•) is the inference or mapping function with parameter set Θ, and L(•) is the loss function to measure the similarity between the output image x and the ground-truth image x of the training set. Several attempts have been made in the literature which typically exploit the deep neural networks to represent the mapping function F(•) such

2. Methods related to our work 29 i.

 29 Create a high-resolution database {x s k |x s k ∈ R n ; k = 1, . . . , P } by randomly extract a set of P high-resolution patches from standard images.

  in which we have to perform a patch-filtering step to select a couple of local dictionaries for each low-resolution patch y i as in the NE or SRSW method, Yang et al. proposed to jointly train two smaller low-and high-resolution dictionaries with the support of sparse model. From the list of standard high-resolution images {x s }, Yang et al. generated the corresponding low-resolution images {y s } by blurring and downsampling as y s = SBx s . After that, a set of P couples of low-and high-resolution patches {y s k ∈ R m , x s k ∈ R n |k = 1, . . . , P } were randomly collected from the training images {y s , x s }. From the database of patches, the authors learned a couple of low-resolution dictionary D l ∈ R m×K and high-resolution dictionary D h ∈ R n×K in the Bayesian MAP perspective and the Laplacian prior model of the probability distribution of image patches. The fundamental of the ScSR method was to assume that the representation of each low-resolution patch y s k and its corresponding high-resolution patch x s k under the low-resolution and high-resolution dictionaries share the same sparse code α s k . That means, they can formulate the problem of determining the dictionaries as

  (a) Airfield (b) Airplane (c) Baboon (d) Baby (e) Barbara (f) Boat (g) Bridge (h) Cameraman (i) Couple (j) Fruits (k) Boy (l) Hill (m) House (n) Jellybeans (o) Leaves (p) Lena (q) Man (r) Monarch (s) Peppers (t) Zelda

Figure 2 . 1 :

 21 Figure 2.1: 20 natural test images.

2 . 1 .

 21 For simplicity, we only consider the luminance channel. Each image was degraded by adding Gaussian noise of zero mean and standard deviation σ = 10 and σ = 30, respectively. The noisy image y is first partitioned into a set of overlapping patches of size 8 × 8 (m = 64) and the denoising was performed on each patch using the 0 sparse model as: αi = arg min

Fig. 2 .

 2 Fig. 2.2 displays these three overcomplete dictionaries of K = 256 atoms and the

Figure 2 . 2 :

 22 Figure 2.2: Different types of overcomplete dictionaries of 256 atoms. (a)-(c) Fixed overcomplete dictionaries constructed from Discrete Cosine Transform (DCT) bases, Haar Wavelet Transform bases, and Daubechies D4 Wavelet Transform (Db4) bases. (d) Data-adaptive dictionary trained from noisy image of Barbara with σ = 30 using K-SVD algorithm [2].

Figure 2 . 3 :

 23 Figure 2.3: Denoising results on image of Barbara at noise level σ = 30 using 4 types of dictionaries of 256 atoms shown in Fig. 2.2.

(a) σ = 10 (b) σ = 30 (

 1030 c) σ = 10 (d) σ = 30

Figure 2 . 4 :

 24 Figure 2.4: Denoising performance on 20 natural test images shown in Fig. 2.1 with 4 different types of dictionaries: DCT, Haar, Db4 and K-SVD. Distribution of differences in term of PSNR between denoising with DCT, Haar, Db4 dictionaries and K-SVD dictionary at two noise levels σ = 10 (a) and σ = 30 (b). Average PSNR results on 20 test images with respect to variation of dictionaries' sizes at σ = 10 (c) and σ = 30 (d).

  2.4(a)-(b) show the distribution of difference (in term of PSNR) between the reconstruction of 3 analytic dictionaries DCT, Haar, Db4 and the result of K-SVD dictionary on 20 images. It can be seen that for complete dictionary (K = 64) and low noise σ = 10, a large portion of images can be better restored by K-SVD dictionary. Furthermore, the K-SVD surpasses the other dictionaries for the case of overcomplete dictionaries (K > 64). In addition, Fig 2.4(c)-(d) show the average PSNR results of 20 images with respect to the number of atoms in the dictionaries which infers

be the first histogram of m bins and h 2 =

 2 [h 2[START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF], . . . , h 2 (j), . . . , h 2 (n)] T ∈ R m to be the second histogram of n bins. Then the EM D distance between two histograms h 1 and h 2 is defined as:

42 )( 2 . 43 )

 42243 Pele et al. have demonstrated that with α > 0.5 the ground distance d ij is a metric, the EM D is a metric distance.

) 52 Chapter 3 .

 3 Patch-based super-resolution for medical images corrupted by Poisson noise

10 ) 1 ( 3 . 11 )

 101311 The problem(3.10) has a close-form solution described in (3.11) Âi = D h i R T RR T + λI -After obtaining the optimal values of basic vectors { αj ∈ Âi }, we can achieve an estimation xi of the latent high-resolution image patch x i from its low-resolution degraded version y i by the linear regression mapping: xi = f SR (y i ) = K j=1 κ(y i , d l j ) αj (3.12) After performing the super-resolution on every low-resolution patches y i , the entire latent high-resolution image x can be found by putting the super-resolved patches xi on their proper locations in the image and averaging the overlapping regions between adjacent patches.

54 Chapter 3 .Figure 3 . 1 :

 331 Figure 3.1: (a)-(e) Standard image and (f)-(j) test images. (a) & (f) synthetic images, (b) &

  Tomography (PET) images as shown in Fig.3.1(f)-(j) with different levels of Poisson noise ξ = 1, 3, 5. We consider them as the high-resolution images and create the corresponding low-resolution degraded images using the model in (3.1), with the 7 × 7 Gaussian kernel blur of standard deviation σ = 1, and the downsampling with magnigication factor s = 2.For each test image, we use corresponding standard image displayed in Fig.4.4(a)-(e), to construct the couple of high-and low-resolution dictionaries {D h , D l } of P = 30000 patches according to section 2.1. The size of high-resolution and low-resolution patches are set to 9 × 9 and 5 × 5 pixels, respectively. The number of atoms in local dictionaries is set to K = 5, and the value of L in the coarse search step is chosen as L = 5 × K.

Fig. 3 .Figure 3 . 2 :Figure 3 . 3 :

 33233 Figure 3.2: Super-resolution results on synthetic image with noise level ξ = 3. (a) Original highresolution image. (b) The low-resolution noisy image (shown with nearest neighbor interpolation. (c) Result of bicubic interpolation (PSNR = 24.47, SSIM = 0.411). (d) Result of NE method (PSNR = 28.89, SSIM = 0.884). (e) Result of ScSR method (PSNR = 31.54, SSIM = 0.891). (f) Result of the proposed method with low-resolution patch size 5 × 5 (PSNR = 30.22, SSIM = 0.859). (g) Result of the proposed method with low-resolution patch size 7 × 7 (PSNR = 32.30, SSIM = 0.937).

Figure 3 . 4 :Figure 3 . 5 :

 3435 Figure 3.4: Super-resolution results on PET image of abdomen with noise level ξ = 3. (a) Original high-resolution image in Fig. 3.1(i). (b) The low-resolution noisy image (shown with nearest neighbor interpolation. (c) Result of bicubic interpolation (PSNR = 25.87, SSIM = 0.718). (d) Result of NE method (PSNR = 26.15, SSIM = 0.863). (e) Result of ScSR method (PSNR = 27.72, SSIM = 0.876). (f) Result of the proposed method (PSNR = 29.05, SSIM = 0.891).

Figure 3 . 6 :

 36 Figure 3.6: Effect of number of atoms K of the local dictionaries with noise levels ξ = 1 (a) and ξ = 5 (b).

Figure 3 . 7 :

 37 Figure 3.7: Effect of the size of low-resolution patch ( √ m × √ m) with noise levels ξ = 1 (a), ξ = 3 (b) and ξ = 5 (c).

Fig. 4 .Figure 4 . 1 :

 441 Fig. 4.1 a simple example of synthetic image patches, where (a) is a reference patch, (b) and (c) are two query patches. The intensity of yellow and green pixels of these patches are 200 and 0, respectively. It is easily to compute the Euclidean distances between these patches as 2 (a, b) = a -b 2 = 282.8 and 2 (a, c) = a -c 2 = 282.8. In addition, with the EMD, we have EM D(a, b) = 200 and EM D(a, c) = 600. Therefore, using the Euclidean assessment, we can not distinguish the two image patches (b) and (c). On the other hand, by adopting the EM D, we can determine that the patch in (b) is closer to the reference patch (a) than the other in (c). In addition, it can be observed that the EM D correctly describes the perceptual similarity of patches in Fig 4.1(a) and Fig 4.1(b), while the Euclidean distance falsely state that patches (b) and (c) are similar.

Figure 4 . 2 :

 42 Figure 4.2: Comparison of the F astEM D and the Euclidean distance ( 2 ) for patch similarity.

  (a) and (e) The zoom-in selected regions with one reference patch and two candidate patches. (b) Reference image patch (blue square in (a)). (c) First candidate patch p 1 (red square in (a)) with 2 = 361.1, EM D = 812. (d) Second candidate p 2 (yellow square in (a)) with 2 = 296.8, EM D = 1027. (f) Reference image patch in (e). (g) First candidate patch in (e) with 2 = 326.8, EM D = 1282. (h) Second candidate in (e) with 2 = 326.6, EM D = 2207.patches. In each row, the left image shows a crop of a region in image of Baboon, with the reference patch is marked by red contour and the two query patches are drawn with blue contours. On the right, we show the zooming of the reference patch and the two query, respectively. On the first row of Fig 4.2, the Euclidean distance selects the patch (d) as a more similar candidate to represent the reference patch (a), while the EM D indicates that the query patch in (b) is more suitable to approximate the reference in (a). The same conclusion can be drawn in the second row. In summary, the EM D metric may helps us to collect a set of patches with higher perceptual similarity for a reference patch than the conventional Euclidean distance.

  ), . . . , y sh i (l), . . . , y sh i (m) T be the subpixel shifted version of y i . Inspired from the definition of EM D [133] presented in (2.43), (2.44) in chapter 2, the EM D between two patches y i and y sh i is formulated as: EM D y i , y sh i

Figure 4 . 3 :

 43 Figure 4.3: Illustration of the transportation of units of masses between two corresponding pixels in image patches.

Algorithm 4 . 1 :1

 41 Image Super-Resolution with Earth Mover's Distance (SREMD) Input : Couple of high-and low-resolution dictionaries {D h ∈ R n×P , D l ∈ R m×P }, low-resolution observation y, size of low-resolution patch √ m × √ m, regularization λ in (4.4), number of shifted pixel p s , thresholded ground distance d max , number of iteration in back-projection step T Partition the low-resolution image into a set of low-resolution patches

7αi 8 end 9

 9 Estimate the high-resolution patch xi = D hi αi and the denoised low-resolution patch ŷi = D l i Aggregate the patches to obtain the initial high-resolution image X 0 and denoised low-resolution image Ŷ.10 Apply iterative back-projection as (4.13) to enhance the reconstruction Output : The estimate of the latent high-resolution image XEnforcing the Reconstruction of entire imageFor achieving the whole high-resolution image, the super-resolved patch xi and the denoised patch ŷi are put on their proper locations in the corresponding grids and averaged in overlapping regions to produce an initial estimation of the high-resolution image X 0 , as well as a denoised low-resolution image Ŷ. The final high-resolution image X is determine by project X 0 onto the solution space of SBX = Ŷ to enforce the constraint of the degradation model. Similar to[START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF], we have:

Figure 4 . 4 :

 44 Figure 4.4: (a)-(e): Standard images used to establish the database. (a) CT image of abdomen, (b) CT image of thorax, (c) and (d) CT images of chest, (e) MRI image of knee. (f)-(j): the corresponding test images.

For 74 Chapter 4 .Figure 4 . 5 :

 74445 Figure 4.5: The super-resolution results of a Region Of Interest (ROI) of the CT image of thorax with magnification s = 2 and noise level σ = 20. (a) LR image (size 270 × 180) created from Fig. 4.4(g) with a ROI (the red rectangle). (b)-(f) The ROI up-scaled by the Bicubic interpolation, the NE method (K = 4), the ScSR method (λ = 0.8), the SRSW method and the proposed SREMD method. (g) The ROI in the original test image (Fig. 4.4(g)).

Figure 4 . 6 :

 46 Figure 4.6: PSNR values of different SR methods on MRI images of brain (1-11), CT images of

Figure 4 . 7 :

 47 Figure 4.7: Effect of the parameter λ on the SR performance of CT image of chest (Fig. 4.4(i)) through PSNR and SSIM values.

5 C h a p t e r

 5 Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior

  for noise removal. The results of our work were published at the 25-th European Signal Processing Conference in 2017 1 . Abstract 1 D. V. Tran, S. Li-Thiao-Té, M. Luong, T. Le-Tien and F. Dibos, "Patch-based image denoising: Probability distribution estimation vs. sparsity prior", 25th European Signal Processing Conference (EUSIPCO), Kos, 2017, pp. 1490-1494.Chapter 5. Patch-based Image Denoising: Probability Distribution Estimation vs.Sparsity Prior

82 Chapter 5 .

 825 Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior independent of observation y i . The two main issues in solving the problem in (5.3) are how to determine a dictionary D ∈ R n×K and what is the prior model of the distribution

2 .

 2 Problem Statement and Motivation 83 the repetition of image contents across multiple images, as illustrated in section 2 of chapter 1, we can expect that the latent image patches {x i } in the unknown image x and the patches in the database (extracted from the standard images {x s }) share the same distribution of representation coefficients. Therefore, instead of adopting an available sparse prior for representing the distribution of patches in the latent image x, we can estimate the distribution p(α i ) from the empirical distribution p(α s k ) of representation coefficients of patches in the database and then use it to reconstruct the noisy image patch y i in (5.3).

Figure 5 . 1 :

 51 Figure 5.1: Some of standard images in Kodak PhotoCD Dataset (http: // r0k. us/ graphics/ kodak/ )

  5.1) which contains diversity of real-life scenes such as human subjects, building, flowers, etc. From 24 standard images of the Kodak dataset, we randomly extract a set of P = 200000 image patches {x s k |x s k ∈ R 64 ; k = 1, . . . , 200000} and calculate their corresponding representation coefficients vectors {α s k } in the vector space Ω α ⊂ R 256 by using the least mean square error Chapter 5. Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior (a) Distribution p(α s k (128)) of representation coefficients on 128-th atom

Figure 5 . 2 :

 52 Figure 5.2: Empirical distribution p(α s k (128)) of representation coefficients of image patches in the database with respect to the 128-th atoms of the dictionary D are plotted (in log domain) in green curve. The fitting sparse models for the real distribution of patches are also sketched, including the 0 -norm in magenta dash curve, Laplacian 1 -norm in red dash curve and hyper-Laplacian p -norm (with p = 2/3) in blue dash curve

. 2 ,

 2 which corresponds to the representation coefficients {α s k[START_REF] Rubinstein | Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation[END_REF]}. Moreover, we also portray the fittings of true distribution (the green curves) with three sparse models, including the 0 -norm (p(α s k ) ∝ exp(-γ||α s k || 0 )) in magenta dash line, Laplacian (p(α s k ) ∝ exp(-γ||α s k || 1 )) in blue dash line, and the hyper-Laplacian (p(α s k ) ∝ exp(-γ||α s k || p ) with p = 2/3) in red dash curve.An important remark can be drawn from Fig.5.2. The sparse models such as Laplacian and hyper-Laplacian are imperfect, but satisfactory in approximately representing the distribution of patches in the vector space Ω α . However, they fail to describe the true distributions of image patches with the fitting errors between the green curves and the simulated lines of sparse models. Therefore, in this chapter, instead of selecting a sparse hypothesis as the prior model of the distribution p(α i ) of latent patches, we propose to estimate it from the real distribution p(α s k ) (e.g. shown in green curve) of representation coefficients of patches in the database. The purpose of our research is to demonstrate that an accurate estimation of the distribution p(α s k ) can be well employed as a prior information of p(α i ) to regularize the image patch denoising, and leads to outperforming reconstructions than an arbitrary choice of a sparse model.

( a )

 a Distribution p(α s k (128)) of representation coefficients on 128-th atom

Figure 5 . 3 :

 53 Figure 5.3: Empirical distribution p(α s k (128)) of representation coefficients of image patches in the database (green curve). The fitting sparse models, including the 0 -norm (in magenta dash curve), Laplacian (in red dash curve) and hyper-Laplacian (in blue dash curve). The black dash line is the histogram estimation of the real distribution of patches.

Chapter 5 .Figure 5 . 4 :

 554 Figure 5.4: Partition of α-space into grid of 4 × 4 × 4 rectangular cells. (a) View in 3D. (b) A x-y slice at position z = 0 with a zoom in of center.

3 .

 3 Denoising by probability distribution estimation 89 all experimental test, we adopt the median-based partition with the number of bin is set to B = 32 for each dimension of the vector space Ω α . (a) Distribution p(α s k (1)) of representation coefficients on 128-th atom

Figure 5 . 5 :

 55 Figure 5.5: The empirical distribution p(α s k (1)) of representation coefficients of image patches in the database corresponding to the first atom in the dictionary (green curve) and the 32-bins estimated histograms according to median-based division (black dash line) and equal-bin division (red dash curve).

90 Chapter 5 .

 905 Patch-based Image Denoising: Probability Distribution Estimation vs.

  10 vertical structure test images made of constant-gray-value stripes placed on a dark background as show in Fig. 5.6(b)-(e), where both the width and values of stripes are chosen randomly. We also need an adaptive standard image with similar vertical structures to exploit the redundancy of informations between images. Therefore, we create an image with stripes of gray levels gradually increasing from 1 to 255 as shown on Fig. 5.6(a) and use it as standard image for all comparing methods.

Figure 5 . 6 :

 56 Figure 5.6: (a) Vertical structure standard images with gradually increasing values of stripes. (b)-(f) Some of vertical structure test images with random width and values of stripes (from test1 to test5). (g)-(i) The patch-form of the three elements (atoms) of the dictionary D

, 5 . 9 , 5 .

 595 Fig. 5.9(c)-(e), Fig. 5.10(c)-(e), the proposed method ProbaEst is more robust against the vertical artifacts in reconstruction of noisy images and generates more preferable outputs.

( a )Figure 5 . 7 :

 a57 Figure 5.7: Results of denoising on image test1 with σ = 10. (a)-(e) are the original image, noisy image, result of OMP, LARS and ProbaEst, respectively, with the zoom-in of region-of-interest (R.O.I).

( a )Figure 5 . 8 : 98 Chapter 5 .Figure 5 . 9 :

 a5898559 Figure 5.8: Results of denoising on image test9 with σ = 20. (a)-(e) are the original image, noisy image, result of OMP, LARS and ProbaEst, respectively, with the zoom-in of region-of-interest (R.O.I).

( a )Figure 5 . 10 :

 a510 Figure 5.10: Results of denoising on image test15 with σ = 30. (a)-(e) are the original image, noisy image, result of OMP, LARS and ProbaEst, respectively, with the zoom-in of region-ofinterest (R.O.I).

4 .

 4 Denoising performance and evaluation 101 Cuts tends to produce an over-expansion foreground image. Visually, proposed method achieves very competitive denoising performance, where its results are more similar compare to the original images.

Figure 5 . 11 : 102 Chapter 5 .

 5111025 Figure 5.11: Results of denoising on binary image of Baby with σ = 30.

Figure 5 . 12 :

 512 Figure 5.12: Results of denoising on binary image of Monarch with σ = 30.

Figure 5 . 13 :

 513 Figure 5.13: Results of denoising on binary image of Peppers with σ = 30.

106 Chapter 5 .

 1065 Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior image restoration algorithm is an arbitrary decision. Second, the selected model may be insufficient to characterize the true distribution of image patches.In this chapter, we investigate the distribution of representation coefficients of image patches in the vector space generated from the atoms in an available dictionary to study a prior model for patch-based image denoising. The principal idea of our research is built upon the interesting property of redundancy of local patterns across images, where local patches tend to repeatedly appear in an underlying image and the standard images.Therefore, the patches in a latent image and in the database are considered to share the same distribution of representation coefficients. Thus we proposed to estimate a prior model from the empirical distribution of representation coefficients of patches in the database and use it to restore a degraded image.

Figure 5 . 14 : 108 Chapter 5 .Figure 5 . 15 :

 5141085515 Figure 5.14: Results of denoising on image of peppers with σ = 30.

Figure 5 . 16 :

 516 Figure 5.16: Results of denoising on image of airplane with σ = 30.

Chapter 6 .

 6 Number of Useful Components in Gaussian Mixture Models for Patch-based Image Denoising Gaussian components required for recovering patches.The remainder of this chapter is organized as follows. Section 2 briefly introduces the patch-based image denoising problem with the GMM prior, as well as our motivation of exploiting GMM to estimate the probability distribution p(α i ) from the empirical distribution of representation coefficients of patches in the database. Section 3 gives a quickly description of the datasets used for evaluation. The details of the Gaussian mixture model as Prior for Image Denoising (GPID) method are presented in section 4.

Fig. 2 .

 2 Fig. 2.1(f) as follows. We added Gaussian noise with σ = 20 to the image, and randomly extracted 100000 patches of size 8 × 8 (n = 64) to train a dictionary of K = 256 atoms (D ∈ R 64×256 ) using the K-SVD algorithm (please refer to section 2.5 of chapter 2 for detailed description). We collected 200 training images from the Berkeley segmentation dataset [6] as standard images {x s }. A database of P = 200000 clean patches {x s k |x s k ∈ R 64 ; k = 1, . . . , 200000} are randomly extracted from standard images and we compute their corresponding representation coefficients vectors α s k using the least mean square error

Figure 6 . 1 :

 61 Figure 6.1: The empirical distribution p(α s k (128)) of representation coefficients of image patches in the database corresponding to the 128-th atom in the dictionary (green curve) and the estimated distribution using GMM with M = 7 components (black dash line).

Fig. 6 .π

 6 [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] shows the empirical distribution p(α s k[START_REF] Rubinstein | Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation[END_REF]) of representation coefficients (in green curve) of 200000 patches in the database. We also plot the estimated distribution of GMM model in black dash line. It can be observed that, although there are some fitting errors, but using a mixture of M = 7 Gaussian components can acceptably characterize the true distribution p(α s k (128)) of representation coefficients of patches in the database, especially for the tails of the distribution. Therefore, the GMM model can be exploited to estimate the distribution of image patches.Our aim is to estimate the parameter {π m , µ m , Σ m |m = 1, . . . , M } from the empirical distribution p(α s k ) of P patches in the database. After that, the obtained GMM is used Chapter 6. Number of Useful Components in Gaussian Mixture Models for Patch-based Image Denoising to model the prior distribution of representation coefficients p(α i ) of latent patches x. p(α i ) = M m=1 π m N (α i |µ m , Σ m ) m N (α i |µ m , Σ m )

[ 155 ]

 155 contains 100 images of urban scenes with high self-similarity and many repeated patterns. We use 25 images for training and 25 images for denoising. Nature We use 200 training images in the Berkeley Segmentation datasets [6] to learn a GMM model and 20 popular natural test images shown in Fig. 2.1 for testing. Brodatz [156] contains 112 grayscale images of natural textures. We select 30 good quality and content-rich images and split each of them into 4 non-overlapping sub-images. 90 sub-images are used for training the GMM and 30 sub-images for denoising validation. Dtd [157] contains textural images in the wild such as band, braid, spiral, grid, etc. We choose 55 images for training and 40 images for denoising. CT of Thorax and CT of Lung We download 7 sequences of CT lung images and 12 sequences of CT thorax images from [7]. 40 thorax images are used for training the GMM and 40 images for testing. The numbers of images for training and testing of CT images of Lung are 40 and 60. MRI Brain We download 16 sequences of MRI brain images from [8]. 80 images are selected from 7 sequences for training and 60 images are chosen in other sequences for denoising.

  DTD (f) CT of Thorax (g) CT of Lung (h) MRI of Brain

Figure 6 . 2 :

 62 Figure 6.2: Some images in 8 datasets.

π

  patches {x s k ∈ R n |k = 1, . . . , P } of size √ n × √ n. After mean subtraction, each patchx s k is encoded in the vector space Ω α , generated by K atoms of the dictionary D, as in (5.4), withα s k = (D T D) -1 D T x s k .The probability distribution of representation coefficients vectors p(α s k ) of P patches in the database can be modeled by a GMM of M components as indicated in (6.1), withp(α s k ) = M m=1 π m N (α s k |µ m , Σ m ). By assuming that each representation coefficients vector α s k is independently sampled from the GMM distribution, the overall likelihood objective function is determined asL(Θ) = m N (α s k |µ m , Σ m ) (6.5)Where Θ = {π m , µ m , Σ m |m = 1 . . . , M } is the set of parameters of the GMM model, which can be found by maximizing the likelihood L (or log-likelihood ln(L) in equiva-

118 Chapter 6 .

 1186 km (α s k -µ m ) T (α s k -µ m ) Number of Useful Components in Gaussian Mixture Models for Patch-based Image Denoising

γ 1 - 1 D T y i + λΣ -1 1 µ 1 ( 6 . 13 )πAlgorithm 6 . 1 : 3 for each patch y i ∈ Y do 4 - 5 - 6 - 1 - 1 Dµ 1 7 - 8 - 9 end 10 Aggregate

 1116136134561178910 im = p(π m , µ m , Σ m |y i ) = π m N (y i |Dµ m , DΣ m D T + σ 2 I) M l=1 π l N (y i |Dµ l , DΣ l D T + σ 2 I)(6.11) where 0 ≤ γ im ≤ 1 and M m=1 γ im = 1. For convenience, we assume that γ i1 ≥ γ i2 ≥ . . . ≥ γ iK . When only the first Gaussian component with maximum membership probability γ i1 is used to represent the probability distribution of p(α i ), the problem (6.4) becomes: αi = arg minα i ||y i -Dα i || 2 2 -λ(α i -µ 1 ) T Σ -1 1 (α i -µ 1 )(6.12)(6.12) is a convex quadratic problem and has a closed form solution as:αi = D T D + λΣ -1Typically, this approach is acceptable when the first component is dominant and the other components do not contribute much to the optimization. In practice, we define the set of dominant patches N 1 = {y i |γ i1 ≥ 0.9} and we call N 2 the set of the remaining patches. N 1 patches are restored via (6.13) whereas the patches in N 2 are restored by considering the first L components of the GMM with largest posteriori probabilities corresponding to {γ i1 , γ i2 , . . . , γ iL }. Consequently, for all N 2 patches in the noisy image, 4. Image denoising with a Gaussian mixture model 119 we consider the simplification of problem 6.4, as described in (6.14), which can be solved using a gradient descent algorithm. The denoising method is presented in Algorithm 6.1.α i = arg min α i ||y i -Dα i || 2 2l N (α i |µ l , Σ l )GMM as Prior for Image Denoising (GPID)1 Initialization: X (0) = Y.2 for t = 1 to T do Subtract its mean value (µ y ):y i = y i -µ y . Calculate γ im (1 ≤ m ≤ M ) via(6.11) and arrange in descending order. If γ i1 ≥ 0.9 then αi = D T D + λΣ -1 T y i + λΣ -1 1 Else select L components with largest value of γ im , then solve for αi in (6.14) using gradient descent. Estimate the latent clean patch: x i = D αi + µ y . the denoised patches x i to recover the entire denoised image X t 11

  13) that needs O(K 2 N 1 ) operations, and N 2 patches (N 2 ≈ 10%N -20%N ) are recovered using gradient descent with the complexity O(LT gd K 3 N 2 ), where T gd is the number of iterations of the gradient descent algorithm. The computation of the membership probabilities requires O(n 3 N M ) operations. The denoising step is repeated T times and therefore totally takes O(n 3 N M T + K 2 N 1 T + LT gd K 3 N 2 T ) complexity.

( c )

 c Percentage of N 1 pixels in images of 8 datasets (d) Average reconstruction error || XL=1 -XL=5 || L 1

Figure 6 . 4 : 124 Chapter 6 .

 641246 Figure 6.4: Denoising performance for the identity dictionary

( a )Figure 6 . 5 :

 a65 Figure 6.5: Denoising performance for the K-SVD dictionary with 256 atoms

Figure 6 . 5 : 126 Chapter 6 .

 651266 Figure 6.5: Denoising performance for the K-SVD dictionary with 256 atoms

Figure 6 . 6 :

 66 Figure 6.6: Effect of model complexity (a) and dictionary choice (b) on the denoising performance for L = 1

  investigate the use of GMM model in solving the optimization problem, due to the lack of justification in the literature. By studying the number of useful components in the GMM for patch-based image denoising on 8 image datasets, we first remark that most of the patches in an input image are well represented by a single prominent component. We have explored denoising with increasing number of components L ∈ {1, 5, 10, 15, 20}, and shown that only modest gains can be obtained in terms of PSNR and 1 reconstruction

  In order to evaluate the quality of the reconstructed image X, and compare the performances of different competition image restoration methods, we need to use some image quality assessment

	(IQA) metrics.
	Several attempts have been made in the literature to develop good metrics to
	measure the image quality for different applications such as image denoising, deblurring,
	super-resolution. Image quality assessment methods can be divided into two categories:
	subjective assessment and objective assessment. The subjective IQA methods are
	accurate in reflecting the human perception of the visual quality of an image because
	they are carried out by human beings. In practice, however, subjective evaluation
	is usually inconvenient due to the requirement of a large number of observers, time-
	consuming and can not be an automated process.

1 ×N 2 from its degraded version Y because of the irreversible degradation process. By adopting an image restoration algorithm, we can obtain the reconstructed image X ∈ R N 1 ×N 2 from the observation Y, which is considered as an estimation and a distorted version Chapter 1. Introduction of the latent image X and is expected as close as possible to X.

  12) Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Ut purus elit, vestibulum ut, placerat ac, adipiscing vitae, felis. Curabitur dictum gravida mauris. Nam arcu libero, nonummy eget, consectetuer id, vulputate a, magna. Donec vehicula augue eu neque. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris ut leo. Cras viverra metus rhoncus sem. Nulla et lectus vestibulum urna fringilla ultrices. Phasellus eu tellus sit amet tortor gravida placerat. Fusce mauris. Vestibulum luctus nibh at lectus. Sed bibendum, nulla a faucibus semper, leo velit ultricies tellus, ac venenatis arcu wisi vel nisl. Vestibulum diam. Aliquam pellentesque, augue quis sagittis posuere, turpis lacus congue quam, in hendrerit risus eros eget felis. Maecenas eget erat in sapien mattis porttitor. Vestibulum porttitor. Nulla facilisi. Sed a turpis eu lacus commodo facilisis. Morbi fringilla, wisi in dignissim interdum, justo lectus sagittis dui, et vehicula libero dui cursus dui. Mauris tempor ligula sed lacus. Duis cursus enim ut augue. Cras ac magna. Cras nulla. Nulla egestas.

	Curabitur a leo. Quisque egestas wisi eget nunc. Nam feugiat lacus vel est. Curabitur
	consectetuer.

Integer sapien est, iaculis in, pretium quis, viverra ac, nunc. Praesent eget sem vel leo ultrices bibendum. Aenean faucibus. Morbi dolor nulla, malesuada eu, pulvinar at, mollis ac, nulla. Curabitur auctor semper nulla. Donec varius orci eget risus. Duis nibh mi, congue eu, accumsan eleifend, sagittis quis, diam. Duis eget orci sit amet orci dignissim rutrum. Nam dui ligula, fringilla a, euismod sodales, sollicitudin vel, wisi. Morbi auctor lorem non justo. Nam lacus libero, pretium at, lobortis vitae, ultricies et, tellus. Donec aliquet, tortor sed accumsan bibendum, erat ligula aliquet magna, vitae ornare odio metus a mi. Morbi ac orci et nisl hendrerit mollis. Suspendisse ut massa. Cras nec ante. Pellentesque a nulla. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Aliquam tincidunt urna. Nulla ullamcorper vestibulum turpis. Pellentesque cursus luctus mauris. Nulla malesuada porttitor diam. Donec felis erat, congue non, volutpat at, tincidunt tristique, libero. Vivamus viverra fermentum felis. Donec nonummy pellentesque ante. Phasellus adipiscing semper elit. Proin fermentum massa ac quam. Sed diam turpis, molestie vitae, placerat a, molestie nec, leo. Maecenas lacinia. Nam ipsum ligula, eleifend at, accumsan nec, suscipit a, ipsum. Morbi blandit ligula feugiat magna. Nunc eleifend consequat lorem. Sed lacinia nulla vitae enim. Pellentesque tincidunt purus vel magna. Integer non enim. Praesent euismod nunc eu purus. Donec bibendum quam in tellus. Nullam cursus pulvinar lectus. Donec et mi. Nam vulputate metus eu enim. Vestibulum pellentesque felis eu massa. Quisque ullamcorper placerat ipsum. Cras nibh. Morbi vel justo vitae lacus tincidunt ultrices. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In hac habitasse platea dictumst. Integer tempus convallis augue. Etiam facilisis. Nunc elementum fermentum wisi. Aenean placerat. Ut imperdiet, enim sed gravida sollicitudin, felis odio 3. Image quality assessment metrics 13 placerat quam, ac pulvinar elit purus eget enim. Nunc vitae tortor. Proin tempus nibh sit amet nisl. Vivamus quis tortor vitae risus porta vehicula. Suspendisse vel felis. Ut lorem lorem, interdum eu, tincidunt sit amet, laoreet vitae, arcu. Aenean faucibus pede eu ante. Praesent enim elit, rutrum at, molestie non, nonummy vel, nisl. Ut lectus eros, malesuada sit amet, fermentum eu, sodales cursus, magna. Donec eu purus. Quisque vehicula, urna sed ultricies auctor, pede lorem egestas dui, et convallis elit erat sed nulla. Donec luctus. Curabitur et nunc. Aliquam dolor odio, commodo pretium, ultricies non, pharetra in, velit. Integer arcu est, nonummy in, fermentum faucibus, egestas vel, odio. Sed commodo posuere pede. Mauris ut est. Ut quis purus. Sed ac odio. Sed vehicula hendrerit sem. Duis non odio. Morbi ut dui. Sed accumsan risus eget odio. In hac habitasse platea dictumst. Pellentesque non elit. Fusce sed justo eu urna porta tincidunt. Mauris felis odio, sollicitudin sed, volutpat a, ornare ac, erat. Morbi quis dolor. Donec pellentesque, erat ac sagittis semper, nunc dui lobortis purus, quis congue purus metus ultricies tellus. Proin et quam. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Praesent sapien turpis, fermentum vel, eleifend faucibus, vehicula eu, lacus. 1. State-of-the-art Image Restoration 15

1.2.2 Wavelet-based image restoration

	A critical landmark in the evolution of image restoration was the development of the
	wavelet theory in the late of 1980s, in which an image is transformed into a wavelet
	domain generated by a set of orthogonal basis functions, which are built from a mother
	wavelet with different dilations and translation. Let W ∈ R N ×N denote the orthogonal
	matrix constituted of the wavelet basis. In the wavelet domain, an image x ∈ R N
	is represented by the wavelet coefficients α w determined as α w = Wx ∈ R N , which
	is commonly divided into a low-frequency sub-band (which corresponds to the low-
	frequency contents of images such as smooth regions, blurred structure) and a set of
	high-frequency sub-bands (that convey the high-frequency information in image like
	edges, texture, and noise, which are the most sensitive to human vision). Typically, the
	image restoration is performed on the high-frequency sub-bands.
	Instead of studying the prior model p(x) in the image domain, many researchers
	investigate in analyzing the distribution of wavelet coefficients and thus introducing
	prior models in the wavelet domain. A considerable success in application of wavelet
	transform is in the image denoising. The noising model can be derived from (1.1), with

  where the 1 part (||α i || 1 ) generates a sparse model, and the 2 term (||α i || 2

  K×P be the matrix of P representation coefficients vectors. Yang combined the two objective functions in (2.25), forcing the high-resolution and low-resolution representations to share the same codes, and obtained:

	X s = [x s 1 , . . . , x s k , . . . , x s P ] ∈ R n×P refer to the database of P high-resolution patches and
	A s = [α s 1 , . . . , α s k , . . . , α s P ] ∈ R

y s k , . . . , y s P ] ∈ R m×P denote the database of P low-resolution patches,

  Chapter 2. State-of-the-art image restoration R n , k = 1, . . . , P }, in which the dictionary can well express the structure appearing in the patches, as well as promote the sparse representation of each image patch x s k . The fundamental principle of the K-SVD method is to construct the dictionary D = [d 1 , . . . , d j , . . . , d K ] ∈ R n×K from the training set X s such that each image patchx s k ∈ X s can be approximately represented by only a few number of atoms in the dictionary. In[2], Aharon et al. study the following objective function:

, Aharon et al. proposed a brilliant method, known as K-SVD, for learning a dictionary D from a training set of P image patches, denoted by X s = {x s k |x s k ∈

  ipsum dolor sit amet, consectetuer adipiscing elit. Ut purus elit, vestibulum ut, placerat ac, adipiscing vitae, felis. Curabitur dictum gravida mauris. Nam arcu libero, nonummy eget, consectetuer id, vulputate a, magna. Donec vehicula augue eu neque. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris ut leo. Cras viverra metus rhoncus sem. Nulla et lectus vestibulum urna fringilla ultrices. Phasellus eu tellus sit amet tortor gravida placerat. Integer sapien est, iaculis in, pretium quis, viverra ac, nunc. Praesent eget sem vel leo ultrices bibendum. Aenean faucibus. Morbi dolor nulla, malesuada eu, pulvinar at, mollis ac, nulla. Curabitur auctor semper nulla. Donec varius orci eget risus. Duis nibh mi, congue eu, accumsan eleifend, sagittis quis, diam. Duis eget orci sit amet orci dignissim rutrum. Morbi blandit ligula feugiat magna. Nunc eleifend consequat lorem. Sed lacinia nulla vitae enim. Pellentesque tincidunt purus vel magna. Integer non enim. Praesent euismod nunc eu purus. Donec bibendum quam in tellus. Nullam cursus pulvinar lectus. Donec et mi. Nam vulputate metus eu enim. Methods related to our work 47 sit amet nisl. Vivamus quis tortor vitae risus porta vehicula. Fusce mauris. Vestibulum luctus nibh at lectus. Sed bibendum, nulla a faucibus semper, leo velit ultricies tellus, ac venenatis arcu wisi vel nisl. Vestibulum diam. Aliquam pellentesque, augue quis sagittis posuere, turpis lacus congue quam, in hendrerit risus eros eget felis. Maecenas eget erat in sapien mattis porttitor. Vestibulum porttitor. Nulla facilisi. Sed a turpis eu lacus commodo facilisis. Morbi fringilla, wisi in dignissim interdum, justo lectus sagittis dui, et vehicula libero dui cursus dui. Mauris tempor ligula sed lacus. Duis cursus enim ut augue. Cras ac magna. Cras nulla. Nulla egestas. Curabitur a leo. Quisque egestas wisi eget nunc. Nam feugiat lacus vel est. Curabitur consectetuer. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Super-resolution . . . . . . . . . . . . . . . . . . . . . . . . . 3 Performance evaluation . . . . . . . . . . . . . . . . . . . . . . . . . Effect of the regularization parameter λ . . . . . . . . . . . .

	2	Super-Resolution using Earth Movers Distance (SREMD) . . . . .
		2.1	Dictionary construction . . . . . . . . . . . . . . . . . . . . .
		2.2
		3.1
	Vestibulum pellentesque felis eu massa.
	Quisque ullamcorper placerat ipsum. Cras nibh. Morbi vel justo vitae lacus tincidunt
	ultrices. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In hac habitasse

Nam dui ligula, fringilla a, euismod sodales, sollicitudin vel, wisi. Morbi auctor lorem non justo. Nam lacus libero, pretium at, lobortis vitae, ultricies et, tellus. Donec aliquet, tortor sed accumsan bibendum, erat ligula aliquet magna, vitae ornare odio metus a mi. Morbi ac orci et nisl hendrerit mollis. Suspendisse ut massa. Cras nec ante. Pellentesque a nulla. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Aliquam tincidunt urna. Nulla ullamcorper vestibulum turpis. Pellentesque cursus luctus mauris. Nulla malesuada porttitor diam. Donec felis erat, congue non, volutpat at, tincidunt tristique, libero. Vivamus viverra fermentum felis. Donec nonummy pellentesque ante. Phasellus adipiscing semper elit. Proin fermentum massa ac quam. Sed diam turpis, molestie vitae, placerat a, molestie nec, leo. Maecenas lacinia. Nam ipsum ligula, eleifend at, accumsan nec, suscipit a, ipsum. platea dictumst. Integer tempus convallis augue. Etiam facilisis. Nunc elementum fermentum wisi. Aenean placerat. Ut imperdiet, enim sed gravida sollicitudin, felis odio placerat quam, ac pulvinar elit purus eget enim. Nunc vitae tortor. Proin tempus nibh 2. Suspendisse vel felis. Ut lorem lorem, interdum eu, tincidunt sit amet, laoreet vitae, arcu. Aenean faucibus pede eu ante. Praesent enim elit, rutrum at, molestie non, nonummy vel, nisl. Ut lectus eros, malesuada sit amet, fermentum eu, sodales cursus, magna. Donec eu purus. Quisque vehicula, urna sed ultricies auctor, pede lorem egestas dui, et convallis elit erat sed nulla. Donec luctus. Curabitur et nunc. Aliquam dolor odio, commodo pretium, ultricies non, pharetra in, velit. Integer arcu est, nonummy in, fermentum faucibus, egestas vel, odio. Sed commodo posuere pede. Mauris ut est. Ut quis purus. Sed ac odio. Sed vehicula hendrerit sem. Duis non odio. Morbi ut dui. Sed accumsan risus eget odio. In hac habitasse platea dictumst. Pellentesque non elit. Fusce sed justo eu urna porta tincidunt. Mauris felis odio, sollicitudin sed, volutpat a, ornare ac, erat. Morbi quis dolor. Donec pellentesque, erat ac sagittis semper, nunc dui lobortis purus, quis congue purus metus ultricies tellus. Proin et quam. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Praesent sapien turpis, fermentum vel, eleifend faucibus, vehicula eu, lacus.

1

  redundancy of local information across different images of the same modality to collect a large couple of high-and low-resolution dictionaries D h ∈ R n×P and D l ∈ R m×P of P image patches from the list of standard images. However,

	2. Proposed SRRH method for image super-resolution		51
	we exploit the recovering an image patch y i with given large dictionaries is a high time-consuming
	process. Therefore, similar to the existing super-resolution methods [1, 74, 145], we
	only select a smaller couple of local dictionaries {D h i ∈ R n×K , D l i ∈ R m×K , K	P }
	that match the image patch y i for reconstruction. In this work, we propose to learn a
	regression function (mapping) between the low-resolution patches and high-resolution
	patches in a reproducing kernel Hilbert space (RKHS) generated from the atoms of
	local dictionaries, and then use it to map the degraded low-resolution patch y i into the
	high-resolution space to estimate the latent high-resolution patch x i . The proposed
	method is preferred to as SRRH which stands for Super-Resolution by Regression
	function in a reproducing kernel Hilbert space.
	Fortunately, in [144], Anscombe demonstrated that we can stabilize the noise variance
	in the Poisson distribution by applying a nonlinear transform, which is then named as
	Anscombe transform, on the signal as:
	T (y i ) = y i +	3 8	(3.3)
	As a result, the Poisson distribution of image patch is converted into the approximate
	additive standard normal and thus we can more easily reduce the noise in image patch
	by using a framework for Gaussian noise removal. Second, as mentioned in chapter 1,

i is the high-resolution version of y i . The fundamental proposals of our work are based on two significant characteristics. First, because Poisson noise in (3.2) is a signal dependent noise model, it is difficult to separate the noise from the true image.

The rest of this chapter is organized as follows. Section 2 presents our proposed method for super-resolution of image corrupted by Poisson noise. In section 3, we conduct some experimental tests to evaluate the performance of the proposed method in comparison with other existing methods. The conclusion and future works are presented in section 5.

Table 3 .

 3

	3. Performance evaluation

1: PSNR and SSIM comparison for SR with magnification factor s = 2

Table 4 .

 4

	3. Performance evaluation

1: Comparison of the SR performance of different methods

  Example-based super-resolution for enhancing spatial resolution of medical images 1 -based threshold to dynamically restrict the number of atoms in local dictionaries. The experimental results proves the effectiveness of our method over some Euclideanbased super-resolution methods in the literature either for noisy or noiseless LR images. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Ut purus elit, vestibulum ut, placerat ac, adipiscing vitae, felis. Curabitur dictum gravida mauris. Nam arcu libero, nonummy eget, consectetuer id, vulputate a, magna. Donec vehicula augue eu neque. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Mauris ut leo. Cras viverra metus rhoncus sem. Nulla et lectus vestibulum urna fringilla ultrices. Phasellus eu tellus sit amet tortor gravida placerat. Integer sapien est, iaculis in, pretium quis, viverra ac, nunc. Praesent eget sem vel leo ultrices bibendum. Aenean faucibus. Morbi dolor nulla, malesuada eu, pulvinar at, mollis ac, nulla. Curabitur auctor semper nulla. Donec varius orci eget risus. Duis nibh mi, congue eu, accumsan eleifend, sagittis quis, diam. Duis eget orci sit amet orci dignissim rutrum. Maecenas lacinia. Nam ipsum ligula, eleifend at, accumsan nec, suscipit a, ipsum. Morbi blandit ligula feugiat magna. Nunc eleifend consequat lorem. Sed lacinia nulla vitae enim. Pellentesque tincidunt purus vel magna. Integer non enim. Praesent euismod nunc eu purus. Donec bibendum quam in tellus. Nullam cursus pulvinar lectus. Donec et mi. Nam vulputate metus eu enim.

	Vestibulum pellentesque felis eu massa.
	Quisque ullamcorper placerat ipsum. Cras nibh. Morbi vel justo vitae lacus tincidunt
	ultrices. Lorem ipsum dolor sit amet, consectetuer adipiscing elit. In hac habitasse

conventional Euclidean distance for patch-selection and apply it to search a couple of local dictionaries for recovering each patch in the degraded image. We proposed a simple Chapter 4. Nam dui ligula, fringilla a, euismod sodales, sollicitudin vel, wisi. Morbi auctor lorem non justo. Nam lacus libero, pretium at, lobortis vitae, ultricies et, tellus. Donec aliquet, tortor sed accumsan bibendum, erat ligula aliquet magna, vitae ornare odio metus a mi. Morbi ac orci et nisl hendrerit mollis. Suspendisse ut massa. Cras nec ante. Pellentesque a nulla. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. Aliquam tincidunt urna. Nulla ullamcorper vestibulum turpis. Pellentesque cursus luctus mauris. Nulla malesuada porttitor diam. Donec felis erat, congue non, volutpat at, tincidunt tristique, libero. Vivamus viverra fermentum felis. Donec nonummy pellentesque ante. Phasellus adipiscing semper elit. Proin fermentum massa ac quam. Sed diam turpis, molestie vitae, placerat a, molestie nec, leo. platea dictumst. Integer tempus convallis augue. Etiam facilisis. Nunc elementum fermentum wisi. Aenean placerat. Ut imperdiet, enim sed gravida sollicitudin, felis odio placerat quam, ac pulvinar elit purus eget enim. Nunc vitae tortor. Proin tempus nibh 4. Conclusion 77 sit amet nisl. Vivamus quis tortor vitae risus porta vehicula.

Table 5 .

 5

	1: The PSNR measure of denoising of vertical stripes images in Fig. 5.6 with different
	methods.

Table 5 . 3 :

 53 Comparison of denoising methods for binary images using Dice ratio.

	5.6(f)-(h) for the ProbaEst, OMP and LARS

Table 5 .

 5

4: Comparison of denoising methods for binary images using DRDM measurement.

Table 5 .

 5

3 and 5.4

present the results of denoising of binary images in terms of Dice Ratio and DRDM measurement, respectively, where the best values are shown in bold red numbers. We can observe that the proposed method achieves better assessment metrics than the other methods, which proves the effectiveness of our method for binary image denoising.

Table 5 . 6

 56

	5. Conclusion

: Denoising performance on 20 natural images in term of SSIM.

Super-Resolution by Sparse Weights (SRSW[START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]). Please refer to section 2.1 of chapter 2 for more details.

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

http://www.jdl.ac.cn/user/hchang/publication.htm

http://www.ifp.illinois.edu/~jyang29/index.html

http://www-l2ti.univ-paris13.fr/~luong/SRSW_IEEE_TIP/SRSW.html

http://insight-journal.org/midas/community/view/21

https://imaging.nci.nih.gov

Dai-Viet Tran, Sébastien Li-Thiao-Té, Marie Luong, Françoise Dibos, "Number of Useful Components in Gaussian Mixture Models for Patch-based Image Denoising", accepted to ICISP,

July 2-4, 2018.

(a) Maximum PSNR improvement for small GMM model with M = 20 (b) Maximum PSNR improvement for large GMM model with M = 200Figure 6.4: Denoising performance for the identity dictionary

Cited on page115.

Acknowledgments

List of Algorithms

C h a p t e r

Example-based super-resolution for enhancing spatial resolution of medical images

This chapter investigates the selection of a set of similar patches (called local dictionary) from a large dictionary, which is identical to the database of patches, to recover a degraded image patch under the Bayesian MAP and sparsity framework.

In the literature, the bin-to-bin metrics, e.g. Euclidean distance, are conventionally used to measure the similarity between two image patches. However, these metrics assume that two image patches are well aligned and thus fail to deal with the distortion between two patches such as the translation. In this work, we proposed to adopt an efficient cross-bin metric named the Earth Mover's Distance (EMD) to evaluate the similarity between two image patches, by considering each patch as a distribution of image intensities. We introduce an [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF] where X t is the estimate of the latent high-resolution image at t-iteration (t ≥ 0), ↑ s is the bicubic upscaling operation with magnification factor s, b is a Gaussian blur kernel of size 5 × 5 and standard deviation 1. We note that in case of noise-free low-resolution image y (degradation model in (4.1) without the presence of noise η), we use the original observation image Y to replace the denoised image Ŷ in the back-projection in (4.12). The summary of our proposed super-resolution method is described in algorithm 4.1

Performance evaluation

In this section, the proposed method (SREMD) is compared with some super-resolution algorithms in the literature, including bicubic interpolation, Neighbor Embedding-based method (NE 2 ) of Chang et al. [START_REF] Chang | Super-resolution through neighbor embedding[END_REF], Sparse coding-based method (ScSR 3 ) of Yang et al. [3] and Super-Resolution by Sparse Weight (SRSW 4 ) of Trinh et al. [START_REF] Trinh | Novel Example-Based Method for Super-Resolution and Denoising of Medical Images[END_REF]. The summaries of these methods can be found in section 2 of chapter 2 of this thesis. To evaluate the objective performance of the super-resolved images, we adopt two image quality assessment metrics called PSNR and SSIM. Please consult the section 3 presented in chapter 1 for further details.

We first carried out the experiments on five images of organs as shown in Fig For the ScSR method, the value of regularization parameter λ is set to 0.8, and the size of dictionary is 1024. The parameters of the SRSW method are set to λ = 0.0001 and γ = 64. Sparsity Prior discretize the distribution p(α s k ) of patches, which will be introduced in more details in the next section. For a quick demonstration, we show in Fig. 5.3 the empirical distribution p(α s k [START_REF] Rubinstein | Double Sparsity: Learning Sparse Dictionaries for Sparse Signal Approximation[END_REF]) of representation coefficients of image patches in the database corresponding to the 128-th atoms in the dictionary, as well as its estimation versions in dashed black lines and the fitting simulated sparse models. We can observe that the estimated histogram can well represent the empirical distribution of representation coefficients of patches with smaller fitting errors comparing to the sparse models.

In the next section, we will present the idea of estimation of probability distribution function p(α i ) of patches in the latent image x from the empirical distribution p(α s k ) of representation coefficients of patches in the database.

Denoising by probability distribution estimation

In this section, we consider the distribution p(α i ) of representation coefficients of latent patches as a histogram estimated from the distribution p(α s k ) of representation coefficients of patches in the database. We will discover the concept of our work on estimation of p(α i ) from the empirical distribution p(α s k ) via constructing a histogram of piecewise constant functions. After that, we propose a framework that exploits the obtained probability functions in solving the denoising optimization problem.

Estimation of probability distribution p(α i ) from the database

Building histograms on one-or two-dimensional data is easy and visualizing. However, for high dimensional data, we are likely to run into the curse of dimensionality, where we have an exponential number of hypercube bins and nearly all of them may probably be empty. As estimation is a difficult problem in high dimensional spaces, we consider the vector space with dimensions up to three (Ω α ⊂ R 3 ). This is equivalent to choosing a dictionary with only three atoms

After that, we randomly extract a set of P image patches {x s k ∈ R n , k = 1, 2, . . . , P } from the standard images x s to create a database of patches. Each image patch x s k is encoded in the vector space Ω α by a corresponding representation coefficient vector α s k , determined by the projection as described in (5.4),

In the scope of this work, rather than arbitrarily selecting an available sparse model to represent the probability distribution p(α i ), we prefer to approximately calculate it from the real distribution p(α s k ) in Ω α of representation coefficients α s k of patches in the database. We introduced a simple yet effective method, by constructing a 4 Partition the noisy image y into overlapping patches y i .

5 for each image patch y i ∈ y do 6

Subtract its mean value:

8

Define a set of anchor points: Output : The denoised image x.

Chapter 5. Patch-based Image Denoising: Probability Distribution Estimation vs. Sparsity Prior the optimal value of J(α i ) in (5.6) will take place either at

. . , (B -1) K + 1} denote the set of anchor points. For a noisy image patch y i , to minimize (5.6), we can easily compute the value of cost function J(α i ) at anchor points and choose the optimized value αi among L α such that J( αi ) is smallest. αi = arg min

The value of p(α a l ) in (5.11) and the production Dα a l at each anchor point α a l ∈ L α can be computed offline to accelerate the speed of the optimization of (5.11). When we obtain the solution of optimal representation coefficients αi , the clean version of the noisy image patch y i can be estimated by the weighted combination of atoms in the dictionary: xi = D αi (5.12)

The denoising process is repeatedly applied to all patches in the noisy image y i to get the estimations of the latent clean patches. After that, we aggregate the overlapping regions between adjacent patches to achieve the final denoised image xi . The summary of the image denoising framework, including the estimation of probability distribution function procedure, is presented in algorithm 5.1.

Complexity analysis

In the stage of estimation of probability distribution function (step 1 to 3 in the algorithm 5.1), we firstly extract P patches {x s k } of size √ n × √ n from the standard images and project them into the vector space Ω α ⊂ R K generated from the dictionary D ∈ R n×K . The complexity of this process is O(P K 3 + P K 2 n). After that, the distribution of P representation coefficients vectors {α s k } is estimated via constructing a histogram in R K space. Each dimension of Ω α space is split into B bins using log(B) times of recursive median finding algorithm. The complexity of searching (B -1) K median values of P points in K-dimension is O(P K (log(B)) K ). The calculation of cube-wise histogram in (5.5) takes O(B K ) operations. Thus the overall complexity of the probability estimation is O(P K (log(B))

In the denoising stage (steps 5 to 11 in the algorithm 5.1), the most expensive computational complexity is to solve the optimization problem in (5.11), which takes O(N (Kn+n 2 )B K ) operations, with N is the number of noisy patches y i in the degraded image y.

Denoising performance and evaluation 93

As we can see, the computational complexity of the denoising algorithm is highly depend on the number of atoms K of the dictionary D and the number of bins B in each coordinate of the vector space Ω α . With a fixed value of B, we have an algorithm with exponential complexity in term of K. If K is set to a large value (high dimensional vector space Ω α ), the complexity will dramatically increase and become a big challenge in implementation of the algorithm. Because of the curse of high dimensionality and expensive computational cost, we choose a small dictionary with K = 3 atoms to demonstrate the idea of using estimated histogram of the distribution of external patches in the denoising of an image patch.

Additionally, with a given dictionary (K is fixed), the complexity is described in polynomial expression of the number of bins B in each coordinate of vector space Ω α .

It can be observed that if we increase the number of bins by 2 times, the denoising stage (especially in step 9) needs 2 K times longer to handle the reconstruction of the same image. As indicated in section 3.1, we only need to divide each dimension of Ω α into B = 32 bins using the median-based splitting approach to get a good estimation of the distribution of external patches. Therefore, in our experiments, the value of B is set to B = 32 to ensure the balance between time complexity and the restoration performance.

Denoising performance and evaluation

In this section, our aim is to demonstrate that for a given dictionary D, using the estimated probability distribution from the patches in the database as an image prior is more efficient than the sparsity models for noise removal in the Bayesian MAP framework (5.3). Comparison with other denoising methods such as nonlocal selfsimilarity [START_REF] Buades | A non-local algorithm for image denoising[END_REF][START_REF] Dabov | Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering[END_REF], deep-learning [START_REF] Xie | Image Denoising and Inpainting with Deep Neural Networks[END_REF] is out of the scope of our work on this chapter.

Given a dictionary, our method differs from the existing sparsity models in the way we determine the probability distribution function p(α i ). While the sparsity methods assume that the probability of representation coefficients p(α i ) in Ω α obeys a specific model such as the Laplacian p(α i ∝ exp(-λ||α i || 1 )), we propose to estimate p(α i ) via constructing a histogram of the empirical distribution of α s k of external patches in Ω α .

In the remaining of this section, we make comparisons on the denoising performance of our proposed method with two famous sparse models of 0 -norm (5. The problem in (5.13) can be efficiently solved by the orthogonal matching pursuit (OMP) algorithm [START_REF] Tropp | Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit[END_REF]. In our implementation, we use the LARS algorithm developed by Zou et al. [START_REF] Zou | Regularization and variable selection via the Elastic Net[END_REF] to find the solution of (5.14). For more convenient, we refer to our proposed method as ProbaEst, and the others in (5.13) and (5.14) as OMP and LARS, respectively. To have a nondiscriminatory assessment of these methods, we adopt the same dictionary for all methods and interpret the results in terms of both quantitative measures and visual quality.

Our proposed estimation method is limited to a dictionary with three atoms (K = 3).

For this reason, the method is a priori best suited for low complexity images. To explore its performance, we have constructed a test benchmark on synthetic images in section 4.2. We also discuss the results obtained on binary and natural images in sections 4.3 and 4.4.

Parameter setting

In all experimental tests, the size of an image patch is set to 3 × 3 and the overlap between two adjacent patches is 2 pixels. In cases of binary and natural images, the Kodak PhotoCD Dataset (shown in Fig. 5.1) is used as standard images. For the stage of estimation of p(α i ) as described in section 3.1, we use P = 500000 patches randomly extracted from standard images to form the database of patches.

In our empirical work, the noisy images are generated from the corresponding noise-free versions by adding Gaussian noise with different levels σ = 10, 20, 30. In regard to binary tests, all images are converted into binary versions using a threshold method. For objective quality assessment of denoising of vertical structure and natural images, we employ two widely used metrics named peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). In case of binary images, the PSNR is not sufficient for subjective assessment, since it is a point-based measurement, and mutual relations between pixels are not taken into account. Instead, we adopt two metrics called the Dice ratio and the distance-reciprocal distortion measure (DRDM [START_REF] Lu | Distance-Reciprocal Distortion Measure for Binary Document Images[END_REF]) for evaluation. The DRDM, which exploits the correlation distance between black and white pixels within image, is a objective distortion measure to judge the similarity between two binary images. As demonstrated in [START_REF] Lu | Distance-Reciprocal Distortion Measure for Binary Document Images[END_REF], the DRDM metric matches well to subjective assessment by human visual perception. An essential notice is that the small value of DRDM indicate that two binary images are close.

Denoising of natural images

In the rest of our work in this chapter, we compare the performance of the proposed method with two sparsity-based algorithms OMP and LARS for denoising on 20 widely used natural images as shown in Fig. 2.1. We use all 24 images in the Kodak dataset (Fig. 5.1) as standard images {x s }. Moreover, we carry out the experiments by adopting the same dictionary for all methods, which is directly trained from each noisy image, using the K-SVD algorithm [2].

Experimental results

To show the effect of the restriction to the dominant component, we examine the performance of the GPID method on N 2 patches with a varying number of components L ∈ {1, 5, 10, 15, 20} in step 7 of the optimization algorithm. We study the differences in peak signal-to-noise ratio (PSNR) and mean gray-level reconstruction error for the 8 datasets presented in section 3, for the identity dictionary D = I and a K-SVD dictionary, and for small (M = 20) and large (M = 200) numbers of Gaussian components.

In all experiments, we degrade the images from the database with white Gaussian noise with standard deviation σ = 30. We train the two GMMs for each dataset on From the examples in Figure 6.3, we notice that N 2 patches can usually be found close to the edges or contours. We also compute the PSNR values obtained for the GPID method as a function of L. On these examples, only modest gains can be obtained by considering several components in the reconstruction. These properties are explored further by computing the distributions of PSNR gains and reconstruction error.

Denoising performance

When using the identity matrix as a dictionary, image patches are denoised without transformation. Note that the GPID method coincides with the method EPLL proposed by Zoran and Weiss in [5] when L = 1. We first observe that most image patches correspond to a single dominant component from For each dataset, we learn an over-complete dictionary D with K = 256 atoms as in [2]. Figure 6.5 shows a similar situation as Figure 6.4. Most patches in the test C h a p t e r

Perspective and future work

In this thesis, we study patch-based methods for image restoration in the Bayesian framework. Constructing the dictionary used to represent image patches and the prior distribution in dictionary space is a challenging problem in this setting. We have proposed several contributions, in the case of exhaustive dictionaries in Chapters 3 and 4, and in the estimation of the patch distribution in Chapters 5 and 6.

In particular, we have shown that careful selection of the local dictionary improves image denoising and super-resolution in Chapters 3 and 4. The main ingredient was to find matches in the patch database by selecting an appropriate patch distance such as the EMD. In Chapters 5 and 6, we improve image denoising by selecting a better patch prior, first with a computationally efficient procedure for low dimension dictionaries and then with a Gaussian Mixture Model for higher complexity models. All of this points towards the need to adapt the elements of a patch-based Bayesian method to the current restoration problem.

The results presented in this thesis raise a few questions that we would like to address in the future.

Dictionary construction. We have used several possible choices for dictionary construction (filtered exhaustive dictionaries, identity matrix, K-SVD), but we did not explore the properties that a good dictionary should have. This raises the question of the representation space of image patches, of its dimension, of its redundancy, etc.

Titre : Approaches bayésiennes par patchs pour l'amélioration de la qualité des images. Abstract : In this thesis, we investigate the patch-based image denoising and super-resolution under the Bayesian Maximum A Posteriori framework, with the help of a set of high quality images which are known as standard images. Our contributions are to address the construction of the dictionary, which is used to represent image patches, and the prior distribution in dictionary space. We have demonstrated that the careful selection of dictionary to represent the local information of image can improve the image reconstruction. By establishing an exhaustive dictionary from the standard images, our main attribute is to locally select a sub-dictionary of matched patches to recover each patch in the degraded image. Beside the conventional Euclidean measure, we propose an effective similarity metric based on the Earth Mover's Distance (EMD) for image patch-selection by considering each patch as a distribution of image intensities.

Our EMD-based super-resolution algorithm has outperformed comparing to some state-of-the-art superresolution methods. To enhance the quality of image denoising, we exploit the distribution of patches in the dictionary space as a an image prior to regularize the optimization problem. We develop a computationally efficient procedure, based on piece-wise constant function estimation, for low dimension dictionaries and then proposed a Gaussian Mixture Model (GMM) for higher complexity dictionary spaces. Finally, we justify the practical number of Gaussian components required for recovering patches. Our researches on multiple datasets with combination of different dictionaries and GMM models have complemented the lack of evidence of using GMM in the literature.