Ias Sri

Keywords: Pyramid

Transform (DWT) . . . . . . . . 2.2.3 Image fusion using Principal Component Analysis (PCA) . . . . . . 2.2.4 Image fusion using combination Laplacian pyramid and PCA (LP-PCA) . . . . . . . . . . .

INTRODUCTION

Now a days numerous criteria that define the characteristics of a high quality image such as sharp, focus, be properly exposed, having correct color balance, and not having too much noise have been proposed and discussed extensively in image processing literature. Image quality has as much to do with user applications and requirements as it does with perceived visual quality in general. There are many techniques to get a good image or a high quality image. One of them is image fusion technique. Image fusion is an important research topic in many related areas such as computer vision, automatic object detection, remote sensing, image processing, robotics, and medical imaging [START_REF] Pure | An Overview of Different Image Fusion Methods for Medical Applications[END_REF]. The need for image fusion in current image processing systems is increasing mainly due to the increased number and variety of image acquisition techniques. Multi-focus image fusion is the process of combining relevant information from several images into one image. The final output image can provide more information than any of the single images. In Section 1.1, we detail the objectif behind this thesis along with its related research objectives. Next, we formulate these objectives into a problem statement and summarize the solution, provided by this thesis, in Section 1.2. Technical contributions for each application in Section 1.3.

1.1/ OBJECTIVES OF THE WORK

Image fusion, an important branch of data fusion, is the process of combining relevant information from two or more images into a single image where the resulting image will be more informative than any of the input images. The result image should be more suitable for visual perception and machine perception or computer processing. The goal of image fusion is to reduce uncertainty and minimize redundancy in the output as well maximize relevant information particular to an application or task [START_REF] Goshtasby | Image fusion: Advance in the state of the art[END_REF]. The key of image fusion lies on choosing one reliable and effective fusion method to determine fusion coefficient [START_REF] Deng | An Image Fusion Algorithm Based on Discrete Wavelet Transform and Canny Operator[END_REF]. Now a days, with rapid advancements in technology, it is now possible to obtain information from multi sources images to produce a high quality information from a set of images. However, due to the limited depth-of-focus of optical lenses in camera devices, it is often not possible to get an image with contains all relevant object 'in focus' so that one scene of image can be taken into set of images with different focus settings of every image. Besides solutions making use of specialized optics [START_REF] Cheng | Extended depth-of-field microscope imaging: Mpp image fusion vs. wavefront coding[END_REF], [START_REF] Martinez-Cuenca | Extended depth-of-field 3-d display and visualization by combination of amplitude-modulated microlenses and deconvolution tools[END_REF] and computational imaging [START_REF] Bando | Near-invariant blur for depth and 2d motion via time-varying light field analysis[END_REF], [START_REF] Cossairt | Diffusion coded photography for extended depth of field[END_REF], the way to solve this problem is multi-focus image fusion. Multi-focus image fusion is a branch of image fusion which integrate the source of multiple images with different focus settings at the same scene into a composite image that contain all object in focus. The objective of multi-focus image fusion is to produce an image that contains all relevant objects in focus by extracting and synthesizing the focused objects of source images. The basic assumption of the multi-focus image fusion is that the focused object is sharper than the unfocused object, and the sharpness is linked to some computed information measures. During the last decade, a number of sharpness measures for multi-focus image fusion have been proposed. The objective of the work is to develop multi-focus image fusion techniques that result high precision fused image which is more suitable for human or machine perception and for further image-processing tasks.

1.2/ PROBLEM STATEMENT

Up to now, many multi-focus images fusion methods have been developed. The simplest fusion method is to take average of the source images pixel by pixel but this method usually leads to undesirable effect such as reduction in the contrast of fused image [START_REF] Li | Multisensor image fusion using the wavelet transform[END_REF]. Generally, the method of multi-focus image fusion can be classified into spatial domain and transform domain [START_REF] Zhang | Multi-focus image fusion algorithm based on compound PCNN in Surfacelet domain[END_REF]. The technique for multi-focus image fusion that results high precision fused image usually are complicated and high consuming in time, which are of vital importance to fusion quality. In this thesis we develop some techniques of multi-focus image fusion that are low cost and not time consuming however it results a high quality fused image.

1.3/ AREA CONTRIBUTION

Motivated towards addressing the needs in the applications mentioned in Section 1.1, our research demonstrates the pixel-level image fusion technology towards a multi-focus imaging system. We list the following contributions:

• Multi-focus image fusion algorithm:

1.4. DOCUMENT ORGANIZATIONS 1. Wavelet decomposition in Laplacian Pyramid for image fusion. Dempster Shafer Theory based on local variability.

Multi-focus image fusion using

3.

Pixel level multi-focus image fusion based on neighbor local variability.

• Implementation and testing the methods:

We implement the multi-focus image fusion algorithms using MATLAB.

1.4/ DOCUMENT ORGANIZATIONS

After Introduction in the current chapter, the rest of the thesis is presented in the following outline.

Chapter 2 gives some introductions to image fusion, in context of our research. Chapter 3 presents our results on Multi-focus image fusion using combination Laplacian pyramid (LP) and the discrete wavelet transform (DWT) image fusion. In this technique, we decompose the source images into different level of Laplacian pyramid and we fuse the images of each level using DWT method. The final fused image is obtained after reconstruction Laplacian pyramid. Chapter 4 presents our results on multi-focus image fusion using Dempster Shafer Theory based on local variability. We perform multi-focus image fusion by exploiting pixels surrounding a pixel to be fused and using Dempster Shafer Theory to decide which pixel that we take as the final fused image. Chapter 5 presents our results on pixel level multi-focus image fusion based on neighbor local variability. This method takes into consideration the information in the surrounding region of pixels. The wide of the neighborhood or the kernel size depends on the quality of the blurring area that is represented by the variance and the kernel size of the blurring filter, assumed that the blurring filter is Gaussian [START_REF] Nayar | Shape from Focus System[END_REF], [START_REF] Petland | A new sense for depth of field[END_REF]. The fusion is done by weighting each pixel by the exponential of the local variability.

STATE OF THE ART 2.1/ IMAGE FUSION

Initially, the main aim of fusion was restricted to human observation and decision making. The first form of fusion is pixel averaging; this method can be mathematically expressed as

f (x, y) = 1 N N i=1 I i (x, y), (2.1)
where f (x, y) is a fused image, I i (x, y) 's are the source images, i = 1, 2, • • • , N. This method is simple, easily to implement, and low cost computationally, by using this method all information content within images are treated the same. However the result of this method is unsatisfactory because this method introduces artifacts, causes pattern cancellation and contrast reduction [START_REF] Omar | Stathaki Image Fusion: An Overview[END_REF]. In mid-eighties, Burt and Adelson introduced a novel method of image fusion based on hierarchical image decomposition, namely Laplacian pyramid [START_REF] Goshtasby | Image fusion: Advance in the state of the art[END_REF]. Image fusion process can be divided into three categories [START_REF] Kaur | Survey on multifocus image fusion techniques[END_REF]:

• Pixel-level: Pixel-level is the lowest level of the image fusion process. It deals directly with the pixels. The advantages of this level are to detect unwanted noise, to provide detail information, less complexity, and ease of implementation. However, these methods do not handle mis-registration and can cause blocking artefact.

7

• Feature-level: In feature level process, features are extracted from input images. Image is segmented in continuous regions and fuse them using fusion rule. Features of images are combined such as size, shape, contrast, pixel intensities, edge and texture.

• Decision-level: Decision level consists of merging information at a higher level of abstraction, combines the results from multiple algorithms to yield a final fused decision.

Input images are processed individually for information extraction. The obtained information is then combined applying decision rules to reinforce common interpretation.

Based on domain, image fusion methods can be categorized into two domains [START_REF] Zhang | Multi-focus image fusion algorithm based on compound PCNN in Surfacelet domain[END_REF]:

• spatial domain Spatial domain deals directly with pixel to integrate relevant information. Some of the spatial domain techniques include Averaging, select maximum/minimum method, Bovey transforms, Intensity hue saturation method (IHS), High pass filtering method (HPF), Principal component analysis method (PCA). Drawbacks of Spatial domain fusion include spatial distortion in new fused image. This spatial distortion problem is solved in frequency domain.

• frequency domain In frequency domain, image is transformed in frequency domain and frequency coefficients are combined to get fused image. Some of the transform domain fusion techniques include discrete wavelet transform, stationary wavelet transform.

Based on the input data and the purpose [START_REF] Kalaivani | Analysis of Image Fusion Techniques based on Quality Assessment Metrics[END_REF], image fusion methods are classified as:

• Multi-view fusion Multi-view fusion combines the images taken by a sensor from different view-points at the same time. Multi-view fusion provides an image with higher resolution and also recovers the 3 -D representation of a scene.

• Multi-temporal fusion Multi-temporal fusion integrates several images taken at various interval time to detect changes among them or to produce accurate images of objects.

• Multi-focus fusion It is impossible for the optical lens to capture all the objects at various focal lengths. Multi-focus image fusion integrates the images of various focal lengths from the imaging equipment into a single image of better quality.

• Multi-modal fusion Multi-modal fusion refers the combination of images from different sensors and is often referred as multi-sensor fusion which is widely used in applications like medical diagnosis, security, surveillance, etc.

2.2/ MULTI-FOCUS IMAGE FUSION

Multi-focus image fusion is a branch of image fusion. Optical lenses suffer from the problem of limited depth of field. So, it is almost impossible to obtain an image which is in focus everywhere. To solve this problem, several pictures of the same scene are taken by a camera with different focal lengths and the focused parts of the images are then fused to form a single image. The fused image becomes the focused image of the scene.

The methods that we use in this work are called multi-focus image fusion. The general principle of these methods is from two images of the same scene with different focus to give an image with in focus everywhere (without blurring). There are numerous methods with different aims. In the following we study a large number of multi-focus methods, particularly the most recent methods and widely used. The Laplacian pyramid was proposed by [4]. Each level of the Laplacian pyramid is recursively constructed from its lower level by applying the following four basic steps: blurring (low-pass filtering), sub-sampling (reduce size), interpolation (expand), and differencing (to subtract two images pixel by pixel). In the LP, the lowest level of the pyramid is constructed from the original image.

2.2.1.1/ GAUSSIAN PYRAMID DECOMPOSITION

We assume that the source image g 0 with size R × C columns is equal to the zero level of the pyramid, which on the bottom of the pyramid, and denoted the l-th level of Gaussian pyramid which can be obtained by the following equation

g l = REDUCE (g l-1 ) (2.2)
which means, for level l ∈ {1, 2, • • • , p} and nodes (x, y) such that 0

< x < C l , 0 < y < R l g l (x, y) = 2 m=-2 2 n=-2 w (m, n)g l-1 (2x + m, 2y + n), (2.3)
where p + 1 refers to the number of levels in the pyramid and C l × R l is the size of the lth level image. w (m, n) is generating kernel which is separable: w (m, n) = w(m)w(n) and satisfies some conditions, explained in Chapter 3.

Through recursive use equation (2.3) we can obtain p + 1 levels of Gaussian pyramid, g 0 , g 1 , • • • , g p . Due to the source image g 0 at the bottom level of Gaussian pyramid, so the total levels of Gaussian pyramid is p + 1, and the top level is g p . A function EXPAND is the reverse of function REDUCE. Its effect is to expand an (M 1 + 1) × (M 2 + 1) array into a (2M 1 + 1) × (2M 2 + 1) array by interpolating new node values between the given values. Thus, expand applied to array g l of the Gaussian pyramid would yield an array ĝl which is the same size as g l-1 .

ĝl = EXPAND(g l ), (2.4)
by expand it means, for level l ∈ {1, 2, • • • , p} and pixel (x, y),

ĝl (x, y) = 4 2 m=-2 2 n=-2 w (m, n)g l x -m 2 , y -n 2 , (2.5)
where

g l x -m 2 , y -n 2 =        g l x-m 2 , y-n 2 , if x-m 2 , y-n 2 integer 0, otherwise. 2.2.1.2/ LAPLACIAN PYRAMID GENERATION The Laplacian pyramid is a sequence of error images L 0 , L 1 , L 2 , • • • , L p .
Each is the difference between two levels of the Gaussian pyramid

L l =        g l -ĝl+1 , if l = 0, 1, 2, • • • , p -1 g p , if l = p.
(2.6)

The reconstruction of image from Laplacian pyramid is a inverse process of decomposition, and in the reverse direction, from the top to the bottom level with the definition as follows. The original image, g 0 , can be obtained by expanding then summing all the levels of LP:

       g p = L p g l = L l + ĝl+1 if l = p -1, p -2, • • • , 0. (2.7)

2.2.1.3/ IMAGE FUSION USING LAPLACIAN PYRAMID

Based on [1], Laplacian pyramid can be used for multi-focus image fusion. It started with two or more images focused on different distances and fuse them in a way that retains the sharp regions of each. Let L A and L B be Laplacian pyramids for the two original images. Thus, in focus image components can be selected pixel-by-pixel in the pyramid.

A pyramid L C is constructed for the composite image:

L C (x, y) =        L A (x, y), if |L A (x, y)| > |L B (x, y)|, L B (x, y), otherwise. (2.8)
The composite image is then obtained simply by expanding and summing the levels of L C 's as in (2.7).

The other way of image fusion using Laplacian pyramid is by taking average for each level of Laplacian pyramid, as follows

L C (x, y) = L A (x, y) + L B (x, y) 2 (2.9)
and again the composite image is obtained by expanding and summing L C 's .

2.2.2/ IMAGE FUSION USING DISCRETE WAVELET TRANSFORM (DWT)

The DWT is obtained by applying a low or high pass filter along the rows of the source image and down sampling, then applying a low or high pass filter along the columns of the intermediate image and down sampling once more. The low frequency subbands corresponds to approximation part, which contains average information of the entire image and is represented as (LL) subband. Whereas, the high frequency subbands are considered as detail parts containing the sharp information of images. The detail parts consist of three high frequency subbands (LH, HL and HH). DWT can be applied in performing image fusion. The coefficients of frequency subbands preserved the image transformation. Such coefficients that come from different images can be combined to get new coefficients where the information in the new or fused coefficients is also preserved. Once the coefficients are merged, the final fused image is obtained by performing the inverse discrete wavelet transform (IDWT).

2.2.3/ IMAGE FUSION USING PRINCIPAL COMPONENT ANALYSIS (PCA)

[50] proposed image fusion using PCA. PCA is a mathematical tool which transforms a number of correlated variables into a reduced number of uncorrelated variables. The fusion using PCA method is achieved by weighted average of images to be fused. The weights for each source image are obtained from the eigen vector corresponding to the largest eigenvalue of the covariance matrices of each source. Let X be a d-dimensional random vector and assume it to have zero empirical mean. Orthonormal projection matrix V would be such that Y = V T X with the following constraints: covariance of Y, cov(Y) is a diagonal and the inverse of V is equal to its transpose

(V -1 = V T ). Using matrix algebra cov(Y) = E(YY T ) = E (V T X)(V T X) T = E (V T X)(X T V) (2.10) = V T E(XX T )V = V T cov(X)V.
Multiplying both sides by V, it is obtained

Vcov(Y) = VV T cov(X)V = cov(X)V, (2.11)
where

V = [V 1 , V 2 , • • • , V d ] and cov(Y) =                        λ 1 0 . . . 0 0 0 λ 2 . . . 0 0 . . . . . . . . . . . . . . . 0 0 . . . λ d-1 0 0 0 . . . 0 λ d                        . (2.12)
by substituting the equation (2.11) into the equation (2.12), we get

[λ 1 V 1 , λ 2 V 2 , • • • , λ d V d ] = [cov(X)V 1 , cov(X)V 2 , • • • , cov(X)V d ] λ i V = cov(X)V i , (2.13)
where i = 1, 2, • • • , d and V i is eigenvector of cov(X).

In the [START_REF] Naidu | Pixel-level image fusion using wavelet and principal component analysis[END_REF], let the source images (I 1 and I 2 images to be fused) be arranged in two-column vectors. The fused image is

I f (x, y) = P 1 I 1 (x, y) + P 2 I 2 (x, y). (2.14)
where P 1 and P 2 are eigenvalues corresponding to the larger eigenvalue of covariance matrix of X.

The disadvantage of this method is that this method is sensitive to the area to be sharpen and produces fusion result that may vary depending on the selected image subset. The PCA produces the fused image with high spatial quality. However, it causes spectral degradation in the fused image.

2.2.4/ IMAGE FUSION USING COMBINATION LAPLACIAN PYRAMID AND PCA (LP-PCA)

[66] uses combination of Laplacian Pyramid and PCA techniques. Different levels of an input image are created using Laplacian Pyramid method, the top level are fused using PCA algorithm given in [START_REF] Naidu | Pixel-level image fusion using wavelet and principal component analysis[END_REF] and for other levels are fused using traditional image fusion method (DWT-based method). Resultant fused image is reconstructed by the pyramid

2.2.5/ MULTI-FOCUS IMAGE FUSION BASED ON VARIANCE CALCULATED IN DIS-CRETE COSINE TRANSFOM (DCT) DOMAIN (DCT+VAR)

[30] has explained that the image fusion is a technique to combine information from multiple images of the same scene in order to deliver only the useful information. The discrete cosine transformation based methods of image fusion are more suitable and time-saving in real time system. In this paper an efficient approach for fusion of multi-focus images based on variance calculated in DCT domain is presented. Two dimensional DCT transform of an N × N block of an image I(x, y) is defined as

d(k, l) = 2α(k)α(l) N N-1 x=0 N-1 y=0 I(x, y) cos (2x + 1)πk 2N cos (2y + 1)πk 2N , (2.15)
where k, l = 0, 1,

• • • , N -1 and α(k) =        1 √ 2 , if k = 0, 1, otherwise.
The inverse DCT(IDCT) is also defined as

I(x, y) = 2α(k)α(l) N N-1 k=0 N-1 l=0 d(k, l) cos (2x + 1)πk 2N cos (2y + 1)πk 2N , (2.16)
where m, n = 0, 1,

• • • , N -1.
The normalized transform coefficient d(k, l)'s are defined as below:

d(k, l) = d(k, l) N .
(2.17)

Mean value, µ, and variance, σ 2 , of N × N block in spatial domain are calculated as:

µ = 1 N 2 N-1 x=0 N-1 y=0 I(x, y) (2.18)
and

σ 2 = 1 N 2 N-1 x=0 N-1 y=0 I 2 (x, y) -µ 2 . (2.19)
It is known that mean value µ = d(0, 0) and for variance

N-1 x=0 N-1 y=0 I 2 (x, y) = N-1 x=0 N-1 y=0 I(x, y)I(x, y) = N-1 x=0 N-1 y=0 I(x, y)         2α(k)α(l) N N-1 k=0 N-1 l=0 d(k, l) cos (2x + 1)πk 2N cos (2y + 1)πk 2N         = N-1 k=0 N-1 l=0 2α(k)α(l) N d(k, l) N-1 x=0 N-1 y=0 I(x, y) cos (2x + 1)πk 2N cos (2y + 1)πk 2N (2.20) = N-1 k=0 N-1 l=0 2α(k)α(l) N d(k, l) Nd(k, l) 2α(k)α(l) = N-1 k=0 N-1 l=0 d 2 (k, l) then σ 2 = N-1 k=0 N-1 l=0 d 2 (k, l) N 2 -d2 (0, 0). (2.21)
In multi-focus images, the focus area contains more information. This information corresponds to the high variance. Suppose there are two source image I A and I B . The steps of the fusion images with this method as follows.

Step 1: Divide the source images into 8 × 8 block partitions and compute the DCT coefficients for each block.

Step 2: Calculate the variances of the corresponding blocks from source images as the activity measures.

Step 3: Determine the block with high activity level as the appropriate one from source image either

I A or I B .
It is observed that fusion performance is not good while using the algorithms with block size less than 8 × 8 and also the block size equivalent to the image size itself. It is also able to generates all in focus image but not generate desirable focused image.

2.2.6/ MULTI-FOCUS IMAGE FUSION USING A BILATERAL GRADIENT-BASED CRI-TERION (BILATERAL GRADIENT)

This method [START_REF] Tian | Multi-focus image fusion using a bilateral gradient-based sharpness criterion[END_REF] assesses the local content (sharp) information of the input image by using a bilateral sharpness criterion that exploits both the strength and the phase coherence that are calculated using gradient information of the images. The statistics of image's gradient is used to propose a new sharpness measurement criterion, that utilize bilateral statistics of image's gradient information. Image structure can be measure effectively by using image gradients. Consider an image I(x, y).

Step1: Calculate the gradient covariance matrix of a region within M × N local window

           w I 2
x (x, y) w I x (x, y)I y (x, y) w I x (x, y)I y (x, y) w I 2 y (x, y)

           , (2.22)
where I x (x, y) and I y (x, y) represents image's gradient at the row and column direction respectively.

Step2: Decompose the gradient covariance matrix as

C = V DV T = v 1 v 2 λ 1 0 0 λ 2 v T 1 v T 2 . (2.23)
where V denotes a 2x2 matrix whose column vectors are eigen vector v 1 and v 2 and D represents a 2x2 diagonal matrix whose diagonal elements are eigenvalues λ 1 and λ 2 (λ 1 > λ 2 ) that correspond to eigenvectors v 1 and v 2 , respectively, and the superscript T represents the transpose. The geometrical structure at pixel in an image can be described by eigenvalues λ 1 and λ 2 the above gradient covariance matrix above.

Step3: Compute the first criterion, the strength of the image's gradient, which is defined as

A(x, y) = λ 1 -λ 2 .
(2.24)

Step4: Calculate the second criterion, the phase coherence of image's gradient, that is

P(x, y) = -cos θ(x, y) -θ(x, y) , (2.25)
where θ(x, y) is the phase information at the position (x, y) determined by eigenvector v 1 associated with the largest eigenvalue λ 1 . θ(x, y) is the average of phases of the neighboring position.

Step5: From the first criterion and the second criterion, develop a bilateral sharpness criterion as

S BS C = A α (x, y)P β (x, y), (2.26)
where α and β are two factors to adjust contributions of two criterions.

The proposed criterion is exploited to develop a weighted aggregation approach to perform image fusion.

2.2.7/ MULTI-FOCUS IMAGE FUSION USING THE ENERGY OF LAPLACIAN (EOL), MAJORITY FILTER AND GUIDED IMAGE FILTER (EOL)

The method is developed by [START_REF] Zhan | A novel explisit multi-focus image fusion method[END_REF]. The energy of Laplacian is evaluated in order to find the most suitable region from multi-focused images. It is then followed by the majority filters that offers a means spreading the focused regions to neighborhood and a guided image filter to overcome blocking artefacts. The EOL focus measure, M p , of source images I A and I B as follows.

M p = (I * L) 2 (2.27)
where * denotes the convolution, I is an input image, L is the Laplacian operator,

L = 4 1 + α                         α 4 1-α 4 α 4 1-α 4 -1 1-α 4 α 4 1-α 4 α 4                         .
where α is a proportion coefficient. The majority filter is utilized in window-based consistency verification and is given by

D b =        1, if D * W l > l 2 2 , 0, otherwose, (2.28)
where D is the input decision map, D b is the filtered decision map, W is a sliding l × l matrix in which all values are set to 1, the term l 2 2 can be obtained by the term 0.5 * W l . Step 1. Normalize the grayscale intensity of I A and I B so that they lie within the range [0, 1].

Step 2. Calculate the EOL focus measure of the input images are computed to obtain M p , respectively.

Step 3. Divide EOL, the matrix M p , into non-overlapping 8 × 8 blocks. The focus measure of each block is the sum of the values in the block

M b (m, n) = M p (x, y), (2.29)
where x and y are pixel indexes, m and n are block indexed, and M b (m, n) is the focus bmeasure for each block.

Step 4. Compare the focus measures in the matrix M b to obtain a binary decision map by

D(m, n) =        1, if M 1 b (m, n) > M 2 b (m, n), 0, otherwise (2.30)
where M 1 b (m, n) is focus measure for image I A and M 2 b (m, n) is focus measure for image I B . The focused image area yield larger EOL metrices.

Step 5. To modify the majority filter, and the decision map D is filtered twice to obtain D b by the modified majority filter. The decision map D b is expanded to D p by

D p = D b ⊗ W 8 (2.31)
where ⊗ denotes the Kronecker product and W 8 is an 8 × 8 matrix in which every value is 1.

Step 6. The fused result under the decision map D p has block effect and lose edge and texture pinformation from input images, the guided filter is applied to optimize the decision map. To set the image I A and decision map D p to guidance image and the fimtering image of the guided filter respectively, and the filtered result is denoted by D 2 . Therefore, the fused image is obtained from original input images using decision map D 2 ,

I f (x, y) = D 2 (x, y)I 1 (x, y) + (1 -D 2 )(x, y)I 2 (x, y), (2.32)
where I f is the fused image.

Even though this image fusion algorithms provide significant results, there is still place for some improvement in spatial and spectral quality.

2.2.8/ MULTI-FOCUS METHOD BASED ON SALIENCY DETECTION AND MULTI-SCALE AVERAGE FILTERING (MSSF)

The method [5] exploits maximum surround saliency detection algorithm for the purpose fusion and designs an optimal weight construction based on visual saliency with simple normalization process, which is capable of identifying focused and defocused regions.

The steps of this method as follow.

Step 1. Decompose the input image into approximation and detail layers by using an average filter.

Consider the input images {I n (x, y)} N n=1 of the same size N 1 × N 2 which are coregistered pixel by pixel. These N-images are decomposed into approximation layers and detail layers as follows:

B k+1 n = B k n * A, (2.33)
where

k = 0, 1, 2, • • • , K, K
D k+1 n = B k+1 n -B k n .
(2.34)

Step 2. Calculate the saliencies of input image at different levels by using maximum surround saliency detection method. The saliency map defined for an image I of width w and height h:

S (u, v) = I µ (u, v) -I f (u, v) (2.35)
where I µ (u, v) is the mean of the sub image centered at pixel (u, v) is denoted as follows

I µ (u, v) = u+u 0 i=u-u 0 v+v 0 j=v-v 0 I(i, j) u 0 = min(u, w -u) v 0 = min(v, h -v) A = (2u 0 + 1)(2v 0 + 1)
I f is the Gaussian blurred version of the image I. This saliency detection process is denoted as follows:

S = MS S S (I).

(2.36)

The process of saliency extraction from the approximation layer B k+1 n at level k is represented as follow

S k+1 n = MS S S (B k n ).
(2.37)

Step 3. Determine weight maps w k+1 i from the extracted saliency maps by normalizing them as

w k+1 i = S k+1 i n i=1 S k+1 i .
(2.38)

Step 4. Multiply detail layers with the weight maps and fuse weighted detail layers to obtain a final detail layer

D D = N K=1 N k=1 w k n D k n .
(2.39)

Step 5. Compute the final approximation layer, B,

B = 1 NK N k=1 N n=1 B k n .
(2.40)

Step 6. Sum final approximation and final detail layers to obtain the fused image

F F = B + D. (2.41)

2.3/ DEMPSTER-SHAFER THEORY

Let Θ represent a finite set of hypotheses for a problem domain, called frame of discernment. Define a function m from 2 Θ to [0, 1] where 2 Θ be the set of all subsets of Θ

2 Θ = {A|A ⊆ Θ} . (2.42)
The function m is called a basic probability assignment whenever

m (∅) = 0 and A⊆Θ m (A) = 1. (2.

43)

m(A) is the measure of the belief that is committed exactly to A. According to [START_REF] Klir | Fuzzy sets, uncertainty and information[END_REF], m(A) is the degree of evidence supporting the claim that a specific element of Θ belongs to the set A, but not to any special subset of A. Each A of Θ such that m(A) > 0 are called the focal element of m. By applying the basic assignment function, several evidential functions can be created. A belief measure is given by the function Bel : 2 Θ → [0, 1]:

Bel(A) = B⊆A m (B) . (2.44)
The plausibility measure Pl : 2 Θ → [0, 1] is defined by [START_REF] Shafer | A mathematical theory of evidence[END_REF] as follows:

Pl(A) = A∩B ∅ m(B) = 1 -Bel(A). (2.45)
where A is complement of A or doubt of A. Doubt of A is represented by 1 -Pl(A). Bel(A) measures the degree of evidence that the element in question belongs to the set A as well as to the various special subsets of A. As stated in [START_REF] Yuan | Low Level Fusion of Imagery Based on Dempster-Shafer Theory[END_REF], the crucial aspect of DST concerns the aggregation of evidence provided by different sources. If two mass function m 1 and m 2 are from distinct items of evidence such that m 1 (B) > 0 and m 2 (C) > 0 for some non disjoint subsets B and C of Θ, then they are combinable by means of Dempster's rule. DST [START_REF] Dempster | Upper and lower probabilities induced by a multivalued mapping[END_REF], [START_REF] Dempster | A generalization of bayesian Inference[END_REF], [START_REF] Shafer | A mathematical theory of evidence[END_REF] suggested a rule of combination that permits that the basic probability assignments are combined. The combination (joint mass) of two sets of masses m 1 and m 2 is defined as follows

m 1 ⊕ m 2 (∅) = 0 (2.46) m 1 ⊕ m 2 (A) = B∩C=A m 1 (B)m 2 (C) 1 -B∩C=∅ m 1 (B)m 2 (C)
.

(2.47)

The numerator represents the accumulated evidence for the sets B and C, which supports the hypothesis A and the denominator sum quantifies the amount of conflict between the two sets. Equation (2.47) can be written as

m 1 ⊕ m 2 (A) = B∩C=A m 1 (B)m 2 (C) B∩C ∅ m 1 (B)m 2 (C) . (2.48)
As stated in [START_REF] Bloch | Information Fusion in Signal and Image Processing[END_REF], having a zero mass on a subset A does not mean that his set is impossible, simply that we are not capable of assigning a level precisely to A, since we could have non-zero masses on subsets of A, which would lead us to Bel(A) 0.

One of application of Dempster-Shafer theory is image fusion. The image fusion using Dempster-Shafer theory is described in chapter 4.

2.4/ CONCLUSION

The conclusion of this part is given as a table where we give advantages and disadvantages of the existing fusion methods mentioned above.

Method

Advantage Disadvantage Laplacian Pyramid image fusion [Adelson and Burt,1984] It well preserves edge information in image.

Pyramid decomposition does not provide the information about sudden intensity changes in the spatial resolution of the input images. Adelson and Burt used maximum selection as a selection rule, it produces a high contrast in the fused image. DWT method [START_REF] Pajares | A wavelet-based image fusion tutorial[END_REF] The DWT fusion method may outperform pixel based approach fusion in terms of minimizing the spectral distortion. It also provide better signal to noise ratio than pixel based approach.

The final fused image have a less spatial resolution. This method is complex in fusion algorithm. Required good fusion technique for better result.

PCA method [START_REF] Naidu | Pixel-level image fusion using wavelet and principal component analysis[END_REF] PCA is a tools which transforms number of correlated variables into number of uncorrelated variables, this property can be used in image fusion.

It is computationally efficient.

It may produce spectral degradation.

LP-PCA [START_REF] Verma | Hybrid image fusion algorithm using Laplacian Pyramid and PCA method[END_REF] Multi level fusion where the image undergoes fusion twice using efficient fusion technique provide improved result It is reduced in contrast DCT+Var [START_REF] Haghighat | Real-time fusion of multifocus images for visual sensor networks[END_REF] It is able to generate all-in focus images.

It are not generating desirable focused image. It is computationally expensive. It is observed that fusion performance is not good while using the algorithms with block size less than 8x8 and also the block size equivalent to the image size itself. Bilateral Gradient [START_REF] Tian | Multi-focus image fusion using a bilateral gradient-based sharpness criterion[END_REF] It measures the local sharpness of image.

There are some erroneous selections of some blocks in the focus region due to noise or undesired effects EOL [START_REF] Zhan | A novel explisit multi-focus image fusion method[END_REF] It well preserves the detail information without distortions.

It suffers from blur artifact.

It is complicated and suffers from being time-consuming as it is based upon the spatial domain. MSSF [START_REF] Bavirisetti | Multi-focus image fusion using multi-scale decomposition and saliency detection[END_REF] It produces fused images with more sharpened regions.

It does not well preserve the edge. It is difficult to determine the level of the multiscale method.

Table 2.1: Comparison of different multi-focus image fusion methods

To overcome the drawbacks from the existing fusion methods mentioned above, we propose several methods. Our methods exploit local variability to evaluate the blur of each source image. The local variability on the blurred area is smaller than the local varibility on the focus area. With this idea, we develop methods to optimize the choice of pixel of the source images. So that we obtain the fused image without blocking artifact and spatial distortion.

WAVELET DECOMPOSITION IN LAPLACIAN PYRAMID FOR IMAGE FUSION

In this chapter we discuss about image fusion using Laplacian pyramid where we propose discrete wavelet transform as a selection rule.

3.1/ INTRODUCTION

Many methods exist to perform image fusion. In this work, we propose a new method for fusing images, where we use combination Laplacian pyramid (LP) and the discrete wavelet transform (DWT). As explained in chapter 2, the LP image fusion integrates multisource information at the basic level and can provide more abundant, accurate and reliable detail information. The quality of fusion images by LP method is depending on the selection rule used. This selection rule allows to fuse the images at each level of the pyramid. Among the selections rules used in the literature we have: The averaging selection, maximum selection [1], saliency and match measure [START_REF] Burt | Enhanced image capture through fusion[END_REF], and combination of averaging and maximum energy selection [START_REF] Wang | A multi-focus image fusion method based on laplacian pyramid[END_REF] . Recently, [START_REF] Zhao | Medical image fusion algorithm on the Laplace-PCA[END_REF] used PCA as selection rule in LP image fusion.

As we know that LP is good in preserving the edge. The LP image fusion with average selection rule often leads to undesirable side effects such as reduced contrast. While the LP with maximum selection rule tends to have the higher contrast and brightness. On the other hand, As explained in the chapter 2, the wavelet fusion transformation (DWT) method allows the image decomposition in different kind of coefficients subbands see [START_REF] Guihong | Medical image fusion by wavelet transform modulus maxima[END_REF], [START_REF] Li | Multifocus image fusion scheme based on feature contrast in the lifting stationary wavelet domain[END_REF], [START_REF] Pajares | A wavelet-based image fusion tutorial[END_REF], [START_REF] Yang | Medical image fusion via an effective wavelet-based approach[END_REF]. [START_REF] Guihong | Medical image fusion by wavelet transform modulus maxima[END_REF] showed that the modulus maximum in DWT fusion gives better preservation of both edge features and component information of the object in new fused image preserving the detail image information . A maximum absolute value rule effectively retains the coefficients of in focus regions within the image.

In this thesis, we proposed a new method fusion using Laplacian Pyramid (LP) where the selection rule is DWT with modulus maximum for high frequency subbands and average for low frequency subband. Thus, DWT is used in each level of LP before undergoing fusion. This proposed method gives improvement significantly in the resulting fused image developed in section 3.4.

26CHAPTER 3. WAVELET DECOMPOSITION IN LAPLACIAN PYRAMID FOR IMAGE FUSION

In the following section we give more details about Laplacian Pyramid and DWT methods.

3.2/ LAPLACIAN PYRAMID AND DISCRETE WAVELET TRANSFORM

3.2.1/ LAPLACIAN PYRAMID

The Laplacian pyramid was first introduced by [4] as a model for binocular fusion in human stereo vision, where the implementation used a Laplacian pyramid and a maximum selection rule at each point of the pyramid transform. Essentially, the procedure involves a set of band-pass copies of an image is referred to as the Laplacian pyramid due to its similarity to a Laplacian operator. Each level of the Laplacian pyramid is recursively constructed from its lower level by applying the following four basic steps: blurring (low-pass filtering), sub-sampling (reduce size), interpolation (expand), and differencing (to subtract two images pixel by pixel). In the LP, the lowest level of the pyramid is constructed from the original image.

3.2.1.1/ GAUSSIAN PYRAMID DECOMPOSITION

In this section we develop the pyramidal construction that we use in the sequel. For that we consider g 0 is the original image with size R × C. This image becomes the bottom or zero level of pyramid. Pyramid level 1 contains image g 1 , which is reduce and low-pass filtered version of g 0 . Pyramid level 2, g 2 , is obtained by applying reduce and low-pass filtered version of g 1 . The level-to-level process is as followed

g l = REDUCE (g l-1 )
which means, for level l ∈ {1, 2, • • • , p} and nodes (x, y) such that 0

< x < C l , 0 < y < R l . g l (x, y) = 2 m=-2 2 n=-2 w (m, n)g l-1 (2x + m, 2y + n) (3.1)
we denote (3.1) as

g l = w * g l-1 (3.2)
N refers to the number of levels in the pyramid and R l × C l is the size of the lth level image (it is known that the size of image in the lth level is half of the size of image in the l-1th level). w (m, n) is generating kernel which is separable:

w (m, n) = w(m)w(n). The one dimensional w(m), length 5, is (i) Normalized: 2 m=-2 w(m) = 1 (ii) Symmetric: w(-i) = w(i) for i = 0, 1, 2 (iii
) Equal contribution: all nodes at a given level l must contribute the same total weight to nodes at the next higher level l + 1.

Let w(0) = a, w(-1) = w(1) = b, and w(-2) = w(2) = c. It is easy to show that the three constraints are satisfied [4] when

w(0) = a, w(-1) = w(1) = 1 4 , w(-2) = w(2) = 1 4 - a 2 .
Hence, we can write that w = 1 4 -a 2 ; 1 4 ; a; 1 4 ; 1 4 -a 2 . Usually the value of a is [0.3, 0.6] as in [4] and [START_REF] Heeger | Pyramid based texture analysis/Syntesis[END_REF]. The sequence images g 0 , g 1 , g 2 , • • • , g p form a pyramid of p + 1 levels where the bottom level is g 0 and the top level is g p . The image at a higher level l is reduced a half both in resolution and size of the image at the predecessor level l -1. Iterative pyramid generation is equivalent to convolving the image g 0 with a set of 'equivalent weighting functions' h l defined as follows:

g l = h l ⊕ g 0 (3.3)
Thus,

g 1 = w * g 0 h 1 ⊕ g 0 g 2 = w * g 1 = w * w * g 0 h 2 ⊕ g 0 g 3 = w * g 2 = w * w * w * g 0 h 3 ⊕ g 0 g 4 = w * g 3 = w * w * w * w * g 0 h 4 ⊕ g 0 . . . g l = w * g l-1 = w * (w * (• • • (w * (w l w 's * g 0 )) • • • )) h l ⊕ g 0
So that, we can write

g l (x, y) = N l m=-N l N l n=-N l h l (m, n)g 0 (x2 l + m, y2 l + n) (3.4)
The value of N l doubles from one level to next level.

To clarify (3.4), we develop h l given in (3.4) from (3.1) only for l = 2.

g 1 (x, y) = m =-2 2 n =-2 w (m , n )g 0 (2x + m , 2y + n ) (3.5) g 2 (x, y) = 2 m =-2 2 n =-2 w (m , n )g 1 (2x + m , 2y + n ) = 2 n =-2 2 n =-2 w (m , n ) 2 m =-2 2 n =-2 w (m , n )g 0 2(2x + m ) + m , 2(2y + n ) + n = 2 n =-2 2 n =-2 w (m , n ) 2 m =-2 2 n =-2 w (m , n )g 0 2 2 x + 2m + m , 2 2 y + 2n + n (3.6) 28CHAPTER 3
g 2 (x, y) = 2 n =-2 2 n =-2 w (m , n ) 2 m =-2 2 n =-2 w (m , n )g 0 2 2 x + 2m + m , 2 2 y + 2n + n = 2 n =-2 2 n =-2 w (m , n ) 2m +2 m=2m -2 2n +2 n=2n -2 w (m -2m , n -2n )g 0 2 2 x + m, 2 2 y + n = 2 n =-2 2 n =-2 w (m , n ) 6 m=-6 6 n=-6 χ [2m -2,2m +2]×[2n -2,2n +2] (m, n) × w (m -2m , n -2n )g 0 2 2 x + m, 2 2 y + n = 6 m=-6 6 n=-6 2 n =-2 2 n =-2 w (m , n )χ [2m -2,2m +2]×[2n -2,2n +2] (m, n) × w (m -2m , n -2n )g 0 2 2 x + m, 2 2 y + n (3.7)
where

χ ( m, n) =        1, if (m, n) ∈ [2m -2, 2m + 2] × [2n -2, 2n + 2] 0, otherwise .
From (3.7), we denote

h 2 (m, n) = 2 m =-2 2 n =-2 w (m , n )χ [2m -2,2m +2]×[2n -2,2n +2] (m, n)w (m -2m , n -2n ) (3.8)
To calculate g 2 (x, y),

g 2 (x, y) = 6 m=-6 6 n=-6 h 2 (m, n)g 0 (x2 2 + m, y2 2 + n)
so we have the formula (3.4), with N 2 = 6.

[4] has shown that the case where a = 0.4, the shape of equivalent functions closely resemble to Gaussian probability density function. So the sequence image

g 0 , g 1 , g 2 , • • • , g N is called Gaussian pyramid.
On the other hand, the function EXPAND is defined as the reverse of function REDUCE.

Its effect is to expand an

(M 1 + 1) × (M 2 + 1) array into a (2M 1 + 1) × (2M 2 + 1)
array by interpolating new node values between the given values. Thus, expand applied to array g l of the Gaussian pyramid would yield an array g l,1 which is the same size as g l-1 .

g l,0 = g l and g l,k = EXPAND(g l , k -1).

By expanding it means, for level l ∈ {1, 2, • • • , p} and 0 ≤ k and nodes (x, y),

0 < x < C l-k , 0 < y < R l-k g l,k (x, y) = 4 2 m=-2 2 n=-2 w (m, n)g l,k-1 x -m 2 , y -n 2 , (3.9)
where

g l,k-1 x -m 2 , y -n 2 =        g l,k-1 x-m 2 , y-n 2 , if x-m 2 , y-n 2 integer, 0, otherwise.
In the following we expand g l for one time (k = 1). We denote

ĝl = g l,1 (x, y) = 4 2 m=-2 2 n=-2 w (m, n)g l x -m 2 , y -n 2 , (3.10)

3.2.1.2/ LAPLACIAN PYRAMID GENERATION

The Laplacian pyramid is a sequence of error images

L 0 , L 1 , L 2 , • • • , L p .
Each is the difference between two levels of the Gaussian pyramid

L l =        g l -ĝl+1 , for l = 0, 1, 2, • • • , p -1, g p . (3.11)
The original image, g 0 , can be obtained by expanding then summing all the levels of LP:

g l = L l + ĝl+1 for l = p -1, p -2, • • • , 0
, and as we know g p = L p .

(3.12)

3.2.2/ LAPLACIAN PYRAMID IMAGE FUSION

Laplacian pyramid (LP) is used as image fusion technique in [1] where the key step is how to choose selection rule (SR) in merging the images at each level of LP. This selection is important in order to obtain the best quality of the fused image. Denote the fused image using LP method with selection rule SR as follows: F = LP(SR),

For example: Let L A and L B be Laplacian pyramids as defined in Chapter 2 for the two original image A and image B respectively. F = LP(Average) means that we use average method as selection rule in merging images at each level of LP as follows:

L C (x, y) = L A (x, y) + L B (x, y) 2 (3.13)
The final composite image, F, is achieved by using the EXPAND function on L C 's as in (3.11).

3.2.3/ DISCRETE WAVELET TRANSFORM

Discrete Wavelet Transform (DWT) that we use in this thesis is based on Haar wavelet transform. The DWT fusion method allows the image decomposition in different kind of coefficient subbands. In decomposition levels, the DWT gives directional information decomposition levels and contains unique information at different resolutions [START_REF] Mallat | Theory for multiresolution signal decomposition: The wavelet representation[END_REF]. The DWT separately filters and downsamples images in the horizontal direction and vertical directions. This produces four coefficient subbands at each scale. As the development of the wavelet theory, DWT has been paid much attention due to its good time-frequency characteristics.

As presented in [START_REF] Pajares | A wavelet-based image fusion tutorial[END_REF], consider an image I(x, y) of size R × C and denote the horizontal frequency first by using 1-D low pass filter

L = 1 √ 2 [1 1] and highpass filter H = 1 √ 2 [1 -1]
produces the coefficient matrices and and then followed the vertical frequency second by using lowpass filter L and highpass filter H to each column in I L (x, y) and I H (x, y), it produces produces four subimages, I LL (x, y), I LH (x, y), I HL (x, y), and I HH (x, y) for one level decomposition. By recursively applying the same scheme to the low-low subband a multiresolution decomposition can be achieved. The algorithm can be expressed as follows: l(i) the analysis lowpass coefficients of a specific wavelet basis, i = 0, 1, 2, • • • N l -1, where N l is the support length of the filter L. h( j) the analysis lowpass coefficients of a specific wavelet basis,

j = 0, 1, 2, • • • N h -1,
where N h is the support length of the filter H. Then,

I L (x, y) = 1 N l N l -1 i=0 l(i).I ((2x + i)mod R, y) , (3.14)
where a = b mod R, the modulo, is an operation to find the remainder a of Euclidian division of b by R.

I H (x, y) = 1 N h N h -1 j=0 h( j).I ((2x + j)mod R, y) (3.15) for x = 0, 1, 2, • • • , R 2 -1 and y = 0, 1, 2, • • • , C -1. I LL (x, y) = 1 N l N l -1 i=0 l(i).I L (x, (2y + i)mod C) (3.16)
I LH (x, y) = 1 N h N h -1 i=0
h( j).I L (x, (2y + j)mod C) (3.17)

I HL (x, y) = 1 N l N l -1 i=0 l(i).I H (x, (2y + i)mod C) (3.18) I HH (x, y) = 1 N h N h -1 i=0 h(i).I H (x, (2y + j)mod C) (3.19) for x = 0, 1, 2, • • • , R 2 -1 and y = 0, 1, 2, • • • , C 2 -1.
The algorithm can iterate on the subimage I LL (x, y) to obtain four coefficient matrices in the next decomposition level and so on. 3.1 shows 2-level discrete wavelet decomposition and fusion image using wavelet transform. In the DWT, only coefficients of the same level and representation can be fused. The fused coefficient can be achieved by various strategies. The process of fused coefficients in this work is described in Section 3.3. After the new fused multiscale coefficients then by using Inverse Discrete Wavelet Transform (IDWT) as described in [START_REF] Pajares | A wavelet-based image fusion tutorial[END_REF], the final fused image is obtained.

3.3/ THE PROPOSED METHOD

The image fusion methods keep progressing to get the better result of fused image. A Laplacian pyramid can be used for compaction of images. To decrease the large number of unwanted information from Gaussian pyramid, it is required to calculate the difference between the adjoining two pictures (images) and receive the bandpass filtered pictures, this set/process is the Laplacian Pyramid (LP) technique. LP takes the advantage of integrating the details at each Laplacian level by retaining large amount of information as well as reducing maximum redundant details from images. On the other hand, The DWT is a mechanism that differentiates data into different frequency components, and then analyzes each component with constancy related to its scale. This hierarchical structure makes very useful because it contains the local information and the global information of images [START_REF] Nunez | Multiresolution-based image fusion with additive wavelet decomposition[END_REF]. The DWT technique gives detail information of source images into one fused image. The main benefit of DWT is that it gives results with low cost. The DWT gives fine resolution in time as well as frequency. The other benefits of DWT are edge detection, energy compaction and multi resolution analysis.

In this work, we fuse images using combination LP and DWT fusion method where we decompose each source image by LP at first and then apply wavelet decomposition as selection rule at each level of LP. Indeed, we fuse LP images in wavelet decomposition by merging the DWT coefficient of every corresponding frequency subband. We used a maximum absolute value of high frequency bands (HH, HL, LH) that effectively retains the coefficients of in focus regions, and we take average of the coefficients bands LL in contrary of the work [START_REF] Guihong | Medical image fusion by wavelet transform modulus maxima[END_REF] where they use maximum absolute for all coefficients. Because wavelet transform theory shows that the high-frequency coefficients of a clear image are much larger than those of a blurred image. But the low-frequency coefficients are roughly equal [START_REF] Chen | Technology for multi-focus image fusion based on wavelet transform[END_REF].

The steps of image fusion in this work as follows. Suppose there are two original source images, image A and image B, with different focus to be fused:

Step 1: To perform Laplacian pyramid decomposition for each source image;

Step 2: To perform discrete wavelet decomposition to every level of Laplacian pyramid for each image in different kinds of coefficient;

Step 3: To merge an appropriate coefficient of the corresponding subband to obtain new coefficients by using maximum absolute for high frequency bands and taking average for low frequency subband. The fused wavelet image is achieved through the inverse discrete wavelete transform (IDWT);

Step 4: The final fused image is obtained by performing LP reconstruction using EXPAND function and summing as in (3.6) on the all level fused wavelet images.

The process of pyramid image fusion can be seen in Fig. 3.2 which the fusion LP for four levels is obtained by applying DWT image fusion. It can be extended for more than two source images.

3.4/ COMPARISON BETWEEN METHODS

In order to compare our proposed method with other methods, we provide the performance evaluation metrics that we will use from this chapter onwards. In this chapter we do not use all mentioned evaluation metrics in the following however we use root mean square in all methods in this thesis. As we know there are two type of the analysis to evaluate the fused image: qualitative (subjective) analysis and quantitative analysis.

Qualitative methods involve visual comparison between a reference image and the fused image. The advantage of this evaluation is easier to interpret. However, it also has some disadvantages such as: it is subjective and depends heavily on the experience of the respective interpreter as well it cannot be represented by mathematical models, and their technique mainly visual.

In this section, we discuss some quantitative analysis that will be used to evaluate the performance of the result fused image. Let F(x, y) be the gray level intensity of pixel (x, y)

of the fused image and I(x, y) be the gray level intensity of pixel (x, y) of the reference image.

3.4.1/ ROOT MEAN SQUARE ERROR (RMSE)

RMSE gives the information how the pixel values of fused image deviate from the reference image. RMSE between the reference image and fused image is computed as:

RMS E = 1 RC R x=1 C y=1 I(x, y) -F(x, y) 2 , (3.20)
where RxC is the size of the reference image and the fused image, (x, y) represents to the pixel locations. A smaller value of RMSE shows good fusion result. If the value of RMSE is 0 then it means the fused image is exactly the same as reference image.

3.4.2/ PEAK SIGNAL TO NOISE RATIO

PSNR is the ratio between the signal (image data) and the noise. In image processing, PSNR is calculated between two images. We find the peak signal to ratio between the fused image F and the reference image I. PSNR is computed as

PS NR = 10 log         255 2 1 RC R x=1 C y=1 I(x, y) -F(x, y) 2         , (3.21)
where RxC is the size of the reference image and the fused image. A higher value of PSNR gives better fusion results and this value shows how alike the fused and reference image are.

3.4.3/ ENTROPY

Image entropy is to evaluate the richness of image information; it represents the property of combination entropy of an image. The entropy on an image is:

H = - L-1 l=0 p(l) log p(l), (3.22)
where L is the number of possible gray levels, p(l) is probability of gray level l.The larger the combination entropy of an image, the richer the information contained in the image.

If the entropy of used image is higher than the reference image or input images then it indicates that the fused image contains more information.

3.4.4/ AVERAGE GRADIENT

Average gradient, G, reflects the contrast between the detail variation of pattern on the image. The larger value of G, the clearer of image. In image fusion, the larger average gradient means a higher spatial resolution

G = 1 (R -1)(C -1) R-1 x=1 C-1 y=1        ∂F(x, y) ∂x 2 + ∂F(x, y) ∂y 2        /2. (3.23)
where RxC is the size of the reference image and the fused image. ∂F(x,y) ∂x and ∂F(x,y) ∂y are one-order differential of pixel (x,y) in x and y direction respectively.

3.4.5/ NORMALIZED CROSS CORRELATION (NCC)

NCC is used as a measure for calculationg the degree of similarity between two images.

NCC = R x=1 C y=1 I(x, y) -Ī(x, y) R x=1 C y=1 F(x, y) -F(x, y) R x=1 C y=1 I(x, y) -Ī(x, y) 2 R x=1 C y=1 F(x, y) -F(x, y) 2 . (3. 24
)
where RxC is the size of the reference image and the fused image. Ī and F are mean of I and F respectively.

3.4.6/ STRUCTURAL SIMILARITY (SSIM)

SSIM that is developed by [START_REF] Wang | Image quality assesment: from error measurement to structural similarity[END_REF] measure the similarity between two images where quality assessment based on the degradation of structural information. The SSIM index can be viewed as a quality measure of one of the images being compared, provided the other image is regarded as of perfect quality.

S S I M = (2µ

I µ F + c 1 ) (σ IF + c 2 )
µ 2 I + µ 2 F + c 1 σ 2 I + σ 2 F + c 2 (3.25)
where µ I and µ F are mean of I and F respectively, σ 2 I and σ 2 F are variance of I and F respectively, σ IF is covariance of I and F, c 1 and c 2 are variables to stabilize the division with weak denominator, see in [START_REF] Wang | Image quality assesment: from error measurement to structural similarity[END_REF]. The SSIM value is a decimal value between -1 and 1, and value 1 is only reachable in the case of two identical sets of data.

3.4.7/ MUTUAL INFORMATION (MI)

MI was introduced as a similarity measure between two images simoultaneously by [START_REF] Maes | Multimodality image registration by maximization of mutual information[END_REF] and [START_REF] Viola | Alignment by maximization of mutual information[END_REF]. MI assumes no prior functional relationship between the images. It also assumes that a statistical relationship that can be captured by analyzing the images joint entropy. Joint entropy H can be calculated as follow:

H(F, I) = - f,i p FR ( f, i) log p FI ( f, i) (3.26)
MI considers both joint entropy and individual entropy H(F ) and H(I),

H(A) = - a p A (a) log p A (a) (3.27)
for A = F or I. And MI is calculated as follow

MI = H(F) + H(I) -H(F, I) (3.28)
MI measures the reduction in uncertainty about the reference image due to the knowledge of the fused image, and so a larger MI is preferred.

3.5/ EXPERIMENTAL RESULT

In the literature, it is known that the blurred area can be obtained by doing low pass filtering on the clear image [START_REF] Gonzales | Digital Image Processing 2nd edition[END_REF]. The low pass filtering on the clear image is convolution between low pass filter g and clear image I.

f (x, y) = s m,n=-s g(m, n)I(x -m, y -n), (3.29)
where s is the size of the window and the filter satisfies: g(-m, -n) = g(m, n) and

s m,n=-s g(m, n) = 1.
In this experiment, to generate sets of multi-focus images we blur clear image or reference image by using low pass filter (Gaussian filter) because it is shown that all blurred area can be simulated by convolution between Gaussian filter and clear image. Gaussian formula:

g(m, n) = 1 2πσ 2 exp - m 2 + n 2 2σ 2 (3.30)
The parameters of Gaussian filter (s and σ) influence the quality of the blurred. The illustration of Gaussian convolution can be seen on Fig. 3.3 below. Blurring filter with size "s = 2" represents a blurring mask with size (2s + 1) × (2s + 1) = 5 × 5.

Figure 3.3: Gaussian convolution process

To perfom our proposed method, for 150 images on dataset [57] we apply low pass filtering using Gaussian filter with variance = 10 and filter size = 5 to get sets of multi-focus images. All images have size 256x256 pixels and are assumed registered. We fuse the images using four different methods: the proposed method, LP(average), LP(maximum) and DWT method. To compare the different methods we use four evaluation metrics: RMSE, PSNR, entropy, and average gradient. The fused images using the proposed method show improvement visually and quantitatively. We conclude that the proposed method is the best among the methods (LP(average), LP(maximum), DWT, and the proposed method), see on appendix A.1. To ilustrate this result, we present only two examples: first, a set multi-focus image consists of two images and the second, the set consists of three images.

3.5.1/ EXAMPLE OF TWO BLURRED IMAGES

The first set consist of two images with different focus. One image has focus on the small clock and the other image has focus on the bigger clock as shown in Fig. 3.4 where Fig. The result of proposed method has obvious advantages in the details of information.

It also gives the better both in visual clarity and quantitative performance evaluation in comparison to other methods. It is clear that the proposed method produce better quality fusion image than the other methods that are performed in this experiment. It can be seen in

3.5.2/ EXAMPLE OF THREE BLURRED IMAGES

In the previous experiment, the proposed method gives the best result among the methods presented. Hence, we will use the proposed method in this experiment for the fusion of three images. Three images in the second dataset show three different object focuses.

The first image focuses on the small bottle, the left back of the image. The focus on gear is the second image. And the third image has focus on the big bottle. These images are shown in the Fig. 3.9, respectively Fig. 3.9(a), Fig. 3.9(b) and Fig. 3.9(c). We found something interesting while fused these images using different steps combination of images. We fuse these images with several combinations: all three images are fused once at the same time and to fuse every two images firstly then the result to be fused with another image. In the fusing of not all image together at the same time, we fuse two image at first using LP based on wavelet, we decompose two source images using Laplacian pyramid, then decompose images at each level by DWT, and to fuse them by Choose-max and mean method of wavelet coefficients, We apply inverse wavelet on the fused coefficients then to reconstruct them by inverse pyramid to get the fused image. We applied again LP based on wavelet on the first fused image with another image to get all three fused image. Four combination rules are used in this fusion:

• The first combination, F 123 We do the laplacian pyramid decomposition for all three images then we fusel images together at the same time. The result fused images is F 123 .

• The second combination,

F (1,2),3
The laplacian pyramid decomposition is applied to all images. We fuse first two images, image (I 1 ) and image (I 2 ), then we reconstruct the fused laplacian pyramid (F 1,2 ). The result fused F 1,2 we fuse with image (I 3 ) to get th fused image F (1,2),3 .

• The third combination, F (2,3),1 F (2,3),1 is obtained by using similar way with F (1,2),3 but the first fusion is image (I 2 ) and image (I 3 ). The result of first fusion (F 2,3 ), image (I 2 ) and image (I 3 ), is fused with image (I 1 ) to get the fused image F (2,3),1 .

• The fourth combination, F (1,3),2 By fusing image (I 1 ) and image (I 3 ) to get the fused image F 1,3 and then to combine F 1,3 and image (I 2 ) to get the fused image F (1,3),2 .

In this experiment, there is evident that the focus area of image has correlation with the step of combination. The focus areas of image (I 1 ), the focus areas of image (I 2 ), and the focus of areas image (I 3 ) are 8077 units, 15639 units, and 38307 units, respectively. From Table 3.2, in comparing F (1,2),3 , F (2,3),1 , and F (1,3),2 , F (1,2),3 has the highest PSNR that F (1,2),3 is the best result followed F (1,3),2 and F (2,3),1 , as we know that PSNR show how alike the result image and the reference image are. The clear of image can be measured by average gradient, the clearer of image the higher the value of average gradient. Again, F (1,2),3 has the highest value of average gradient, the second is F (1,3),2 , and followed by F (2,3),1 . From these performance evaluation values, F (1,2),3 is better than F (1,3),2 and F (2,3),1 .

image Related to the focus areas, from the result, by combining one by one, it is better to combine from the first two smallest focus area, then the result is combined with the third smaller and so on to the bigger. In this experiment, we see that the first two smallest is combination image (I 1 ) and image (I 2 ) first, F 1,2 , then fused with image (I 3 ) that produced F (1,2),3 , and followed with F 1,3 that yields F (1,3),2 , and F 2,3 that resulted F (2,3),1 . It is because when we fuse from the smallest focus area to the bigger focus area, the loss of originality of the focus areas on the big focus image is not as big as others since it is proceed at last time.

F (1,2),3 F (1,3),2 F (2,

3.6/ CONCLUSION

In the this work, we propose image fusion method using combination Laplacian Pyramid and DWT. We decompose source images into different levels of Laplacian Pyramid and fuse them at each level of LP using DWT as selection rule then we reconstruct the result of fused image at each level to get the final fused image. The result of experiment shows that the proposed method gives improved result in both visually and quantitatively in comparison with the other fusion methods: LP(average), LP(maximum), and DWT method. The fusion of more than two images is better done one by one from the smallest focus image to the bigger focus image. It gives better result than other combinations.

MULTI-FOCUS IMAGE FUSION USING DEMPSTER SHAFER THEORY BASED ON LOCAL VARIABILITY 4.1/ INTRODUCTION

The framework of the study of fusion in this chapter is the same as in the previous chapter, from several images representing the same object with different focus. The intended application is always general way of detecting possible "blurred" present in the images with respect to "clear" images.

As stated in [START_REF] Yuan | Low Level Fusion of Imagery Based on Dempster-Shafer Theory[END_REF], from the evidence point of view, fusion degrades the imprecision and uncertainty by making use of redundancy and complementary information of the source image. That means the weakness evidences from inputs are utilized to give the best estimation . Evidence theory was first proposed by Shafer in 1970s, which is based on Dempster research. The advantage of Dempster-Shafer Theory (DST) is that it allows coping with absence of preference, due to limitations of the available information, which results in indeterminacy, as in [START_REF] Denoeux | Reasoning with imprecise belief structures[END_REF] and [START_REF] Walley | Statistical reasoning with imprecise probabilities[END_REF].

In this thesis, we propose multi-focus image fusion using the evidence functions of images that derived from one information: local variability. Local variability as the activity measure can detect the abrupt image intensity such as edge. This method also takes into consideration the information in the surrounding region of pixels and preserves the edge. We develop a decision fusion using Dempster-Shafer Theory (DST) method. Experiments are performed on different kinds of images and results are compared with other methods in Chapter 2. As the first step, we detail the main elements of the evidence theory of Dempster-Shafer.

4.2/ DEMPSTER-SHAFER THEORY

As in [START_REF] Sentz | Combination of Evidence in Dempster-Shafer Theory[END_REF], Dempster-Shafer Theory (DST) is the significant work on this topic is [START_REF] Shafer | A mathematical theory of evidence[END_REF], which is an extension of [START_REF] Dempster | Upper and lower probabilities induced by a multivalued mapping[END_REF]. DST is a generalization of probability theory in a finite discrete space. In DST, the probabilities are assigned to sets in opposition to mutually exclusive singletons. The probability theory deals with evidence which is associated with only one possible event. On the contrary to the probability theory, DST deals with evi-

4.2.1/ BASIC PROBABILITY ASSIGNMENT

As in [START_REF] Yuan | Low Level Fusion of Imagery Based on Dempster-Shafer Theory[END_REF], let Θ represent a finite set of hypotheses for a problem domain, called frame of discernment. Defined as a function m from 2 Θ to [0, 1] where 2 Θ be the set of all subsets of Θ.

2 Θ = {A|A ⊆ Θ} (4.1)
A piece of evidence that influences our belief concerning these hypotheses induces a mass function m, that satisfies:

m (∅) = 0 and A⊆Θ m (A) = 1 (4.2)
The function m is called a basic assignment. m(A) can be interpreted as the belief that one is willing to commit to hypothesis A (and to none of its subsets) given the available evidence. According to [START_REF] Klir | Fuzzy sets, uncertainty and information[END_REF], m(A) is the degree of evidence supporting the claim that a specific element of Θ belongs to the set A, but not to any special subset of A. Each A of Θ such that m(A) > 0 are called the focal element of m. From the basic probability assignment, the upper and lower bounds of an interval can be defined. This interval contains the precise probability of a set of interest (in the classical sense) and is bounded by two nonadditive continuous measures called Belief and Plausibility.

4.2.2/ BELIEF FUNCTION

As in [START_REF] Yager | Classic Works of the Dempster-Shafer Theory of Belief Functions[END_REF], the lower bound Belief for a set A is defined as the sum of all the basic probability assignments of the proper subsets (B) of the set of interest (A) (B ⊆ A). A belief measure is given by the function Bel : 2 Θ → [0, 1]:

Bel(A) = B⊆A m (B) (4.3)
A real function over the subsets Bel : 2 Θ → [0, 1] is called a belief function if and only if it satisfies the following three axioms as defined by [START_REF] Shafer | A mathematical theory of evidence[END_REF]:

1. Bel(∅) = 0 2. Bel(Θ) = 1

For any whole number n and subsets

A 1 , A 2 , • • • , A n ⊆ Θ, Bel        n i=1 A i        ≥ I⊂{1,2,••• ,n} I ∅ (-1) |I|+1 Bel        i∈I A i       
where |I| is the cardinality of I. Bel(A) measures the degree of evidence that the element in question belongs to the set A as well as to the various special subsets of A. It is possible to obtain the basic probability assignment from the Belief measure with the following inverse function, by using Mobius transformation [START_REF] Yager | Classic Works of the Dempster-Shafer Theory of Belief Functions[END_REF]:

m(A) = B⊆A (-1) |A-B| Bel(B) (4.4)
where |A -B| is the cardinality of A ∩ B, and B is the complement of B.

4.2.3/ PLAUSIBILITY FUNCTION

The upper bound, Plausibility, is the sum of all the basic probability assignments of the sets (B) that intersect the set of interest (A) (B ∩ A ∅). The plausibility measure Pl : 2 Θ → [0, 1]:

Pl(A) = A∩B ∅ m(B) = 1 -Bel(A) (4.5)
where A is complement of A or doubt of A. Doubt of A is represented by 1 -Pl(A).

4.2.4/ RULES FOR COMBINATION OF THE EVIDENCES

As stated in [START_REF] Yuan | Low Level Fusion of Imagery Based on Dempster-Shafer Theory[END_REF], the crucial aspect of DST concerns the aggregation of evidence provided by different sources. If two mass function m 1 and m 2 are from distinct items of evidence such that m 1 (B) > 0 and m 2 (C) > 0 for some non disjoint subsets B and C of Θ, then they are combinable by means of Dempster's rule. DST [START_REF] Dempster | Upper and lower probabilities induced by a multivalued mapping[END_REF], [START_REF] Dempster | A generalization of bayesian Inference[END_REF], [START_REF] Shafer | A mathematical theory of evidence[END_REF] suggested a rule of combination that permits that the basic probability assignments are combined. The combination (joint mass) of two sets of masses m 1 and m 2 is defined as follows

m 1 ⊕ m 2 (∅) = 0 (4.6) m 1 ⊕ m 2 (A) = B∩C=A m 1 (B)m 2 (C) 1 -B∩C=∅ m 1 (B)m 2 (C) (4.7)
DST may be successfully used in some situations, such as in cases where all information regarding to the problem is known and when a source provides information concerning only a few of several classes as stated in [START_REF] Bloch | Fusion of Image Information under imprecision and uncertainty: numerical methods[END_REF]. The advantages of DST, as stated in [START_REF] Yuan | Low Level Fusion of Imagery Based on Dempster-Shafer Theory[END_REF], are that it distinguishes between lack of belief and disbelief and allows the probability to be assigned to the union of the propositions in the frame of discernment. Such lack of belief typically arises in image fusion problems where a 'real scene' image is to be estimated from incomplete and unreliable observations. DST has been successful in many applications in image processing including image segmentation [START_REF] Mena | Color Image Segmentation Using The Dempster-Shafer Theory of Evidence for The Fusion of Texture[END_REF], [START_REF] Rombaut | Study of Dempster-Shafer for image segmentation applications[END_REF], pattern classification [START_REF] Kowsalya | A Survey on pattern classification with missing data using Dempster Shafer theory[END_REF], [START_REF] Zhu | Data fusion for pattern classification via the Dempster-Shafer evidence theory[END_REF], object recognition [START_REF] Hassan | Object recognition based on Dempster-Shafer reasoning[END_REF], medical imaging [START_REF] Bloch | Some aspects of Dempster-Shafer evidence theory for classification of multi-modality medical images taking partial volume effect into account[END_REF], sensor fusion [START_REF] Wu | Sensor fusion using Dempster-Shafer theory[END_REF].

4.3/ THE PROPOSED METHOD: DEMPSTER-SHAFER THEORY -LOCAL VARIBILITY (DST-LV)

The crucial problem of image fusion using Dempster-Shafter Theory is to construct the evidence that represents the images. In this thesis, we propose one information as the evidential representation images based on the measure at each pixel that we call local variability.

4.3.1/ SOURCE: LOCAL VARIABILITY

Our method takes into consideration the information in the surrounding region of pixels. Indeed, at each pixel I(x, y), the method exploits the local variability calculated from quadratic difference between the value of pixel I(x, y) and the value of all pixels that belong to its neighborhood. The idea comes from the fact that the variation of the value in blurred region is smaller than the variation of the value in focused region that we proved in Chapter 5. Furthermore, the local variability expresses the detail information of image.

For image with size (RxC), we use in this work the neighbor of a pixel (x, y) with the size "a" defined as follows:

(x + i, y + j) where i = -a, -a + 1, • • • , a -1, a, j = -a, -a + 1, • • • , a -1, a, x + i ∈ {1, 2, • • • , R}, and y + j ∈ {1, 2, • • • , C}
For example the neighbor with the small size ("a" = 1) contains: (x -1, y -1), (x -1, y), (x -1, y + 1), (x, y -1), (x, y + 1), (x + 1, y -1), (x + 1, y), (x + 1, y + 1) as we can see in Fig. where k is the index of k th source image (k = 1, 2, • • • , p), a is the size of the neighborhood.

I k (x + m, y + n) =        I k (x + m, y + n) , if 1 ≤ x + m ≤ R and 1 ≤ y + n ≤ C, I k (x, y), otherwise T = (2a + 1) 2 -card(S ) S = (m, n) ∈ [-a, a] 2 -{(0, 0)} |I k (x + m, y + n) = I k (x, y)
The variability of image expresses the behavior of pixel relative to all pixels belong to its neighborhood. The variability preserves edge feature because it detects the abrupt image intensity.

4.3.2/ THE FUSION OF IMAGES

In this thesis, we consider two classes in the Dempster-Shafer theory of evidence. Either a pixel belongs to blurred part ω or it belong to the focus part ω. There is also uncertainty θ inherent in the theory of evidence. All this constitute the frame of discernment is Θ in our case [START_REF] Mena | Color Image Segmentation Using The Dempster-Shafer Theory of Evidence for The Fusion of Texture[END_REF].

Θ = {ω, ω, θ} (4.9)
For each pixel one value of evidence for information will be obtained, m. The precision of this fusion is depending on the size of the neighborhood, "a". For each image we try with different values of "a" in the set {1, 2, ..., 10} and we get the value of "a" that corresponds to the minimum of root mean square error (RMSE). This operation is repeated for set of multi-focus images in database [57].

Our method DST-LV consists of the following steps Suppose there are p original source images, I 1 , I

d a,k (x, y) = 1 - v a,k (x, y) -min (x ,y ) v a,k (x , y ) max (x ,y ) v a,k (x , y ) -min (x ,y ) v a,k (x , y ) (4.11)
where k is the k th source image, k ∈ {1, 2, • • • , p} and a is size of neighborhood of local variability. We set the standard deviation of d a,k (x, y) = σ a,k (x, y) for (x, y) belongs to ω, we calculate:

m a,k (ω) = 1 -σ a,k (x,
m a,k (θ) = σ a,k (x, y) (4.13)
for (x, y) belongs to ω, we calculate:

m a,k (ω) = 1 -1 -d a,k (x, y) σ a,k (x, y) -σ a,k (x, y) = 1 -d a,k (x, y) 1 -σ a,k (x, y) (4.14)
To give the final result of the multi-focus image is obtained by showing which pixels belong to focus area or which do not, we use concept plausibility. In our case the plausibility of ω is the sum of the masses of the evidence for ω and the uncertainty θ:

Pl a,k (ω) = m a,k (ω) + m a,k (θ)
and for fusion image of the pixel (x, y), due to ω is a set of pixel on blurred area, we take pixel (x, y) from image k 0 that assigned to minimum Pl k (ω

), k = 1, 2, • • • , p.
Step 2: For (x, y), we take F a as fused image with size of neighborhood = a Pl a,k (ω)(x, y) .

Step 3: For the proposed method, we use different values of size of neighborhood, a ∈ {1, 2, • • • , 10} , and choose the value of a that corresponds to the minimum value of RMSE, such that our final fused image

F = F a 0 where a 0 ∈ {1, 2, • • • , 10} and RMS E(F a 0 ) = min a∈{1,2,••• ,10}
(RMS E(F a ))

4.4/ EXPERIMENTAL RESULT

We generated numerous sets multi-focus image from reference image on dataset [57] using Gaussian filter, all blurred images with the various values of variance and size of blurring filter and we performed the DST-LV method image fusion on those sets of multifocus images using Matlab2013a. All images of database given the same conclusion except ten images of 150 images where RMSE of LP(DWT) is smaller than RMSE of the proposed method however the RMSE difference is very small, see appendix A.2. In this thesis, we choose to present only two multi-focus sets in this experimental section, as shown in Fig. ), we can see that the fused images produced by the PCA method and the bilateral gradient method are not so clear. It can be found that the results of the PCA method, and the bilateral gradient method have a poor contrast compared to other methods, while the MSSF method produces a high visual contrast fused image and not good at preserving the edge. However, it is difficult for us to perceive the difference among the results of the DWT method, LP(DWT) method, LP PCA method, DCT+Var method, EOL method and the proposed method according to the subjective evaluation. Therefore, to objectively evaluate these fusion methods, quantitative measures of the fusion results are needed. The results of the quantitative measures are shown in In this thesis we propose pixel level multi focus image fusion based on the neighbor local variability (NLV). This method takes into consideration the information in the surrounding region of pixels. Indeed, at each pixel I(x, y) , the method exploits the local variability calculated from quadratic difference between the value of pixel I(x, y) and the value of all pixels that belong to its neighborhood. It expresses the behavior of pixel relative to all pixels belong to its neighborhood. The variability preserves edge feature because it detects the abrupt image intensity. The fusion of each pixel (x, y) is done by weighting each pixel by the exponential of the local variability. The precision of this fusion depends on the size of the neighborhood. Firstly, we study the optimal size for having the minimum error for that we show that the size of neighborhood depends on the blurring characterized by the variance and its size of blurring filter. We construct a model that give the value of the size of neighborhood from the variance and the size of blurring filter.

We compare our method with other methods exist in the literature as described in Capter 2, we show that our method gives the best result by using Root Mean Square Error (RMSE). In this work, the experimental for fusion image and compare to other methods. This part is organized as follows: Section 2 gives explanation about the idea of NLV and steps of the proposed method fusion process and a model that give the size of neighborhood from parameter of blurring used are described. The experimental results are shown in section 3.

5.2/ NEIGHBOR LOCAL VARIABILITY

5.2.1/ THE IDEA OF THE PROPOSED METHOD

Consider the fusion of two multi-focus images, I 1 and I 2 , that have respectively blurred parts B 1 and B 2 . These images have the same size: R × C. We study the case where B 1 and B 2 are disjoint. The idea of the NLV fusion method that is to sum the pixel values of the two images weighted by the neighbor local variability of each picture. This neighbor local variability at (x, y) is defined in the previous chapter, equation (4.8).

v a,k (x, y) = 1 T a m=-a a n=-a I k (x, y) -I k (x + m, y + n) 2 (5.1)
where k is the index of k th source image (k = 1, 2), a is the size of the neighborhood.

I k (x + m, y + n) =        I k (x + m, y + n) , if 1 ≤ x + m ≤ R and 1 ≤ y + n ≤ C, I k (x, y), otherwise T = (2a + 1) 2 -card(S ) S = (m, n) ∈ [-a, a] 2 -{(0, 0)} such that I k (x + m, y + n) = I k (x, y)
We show in the following that this local variability is small enough where the location is on the blurred area (B 1 or B 2 ). Indeed, we consider, without loss the generality, that we have a focus pixel (x, y) in image I 1 and blurred in image I 2 ((x, y) ∈ B 2 ) The local variability of image I 1 and image I 2 are respectively:

1 T r 1 (x, y) and 1 T r 2 (x, y), where r 1 (x, y) and r 2 (x, y) can be written as follow:

r 1 (x, y) = 2a m=0 2a n=0 |I 1 (x, y) -I 1 (x + (m -a), y + (n -a))| 2 (5.2) r 2 (x, y) = 2a m=0 2a n=0 |I 2 (x, y) -I 2 (x + (m -a), y + (n -a))| 2 (5.3)
Let I R is the reference image of multi-focus images I 1 and I 2 . Moreover, it is shown in [START_REF] Nayar | Shape from Focus System[END_REF] and [START_REF] Petland | A new sense for depth of field[END_REF] that the blurred image can be seen as the product convolution between the reference image and a Gaussian filter. Let w 1 and w 2 are Gaussian filter, thus

I 1 (x, y) =        w 1 * I R (x, y), (x, y) ∈ B 1 I R (x, y), (x, y) B 1 . , I 2 (x, y) =        w 2 * I R (x, y), (x, y) ∈ B 2 I R (x, y), (x, y) B 2 . , (5.4)
The product convolution is defined as

w 1 * I R (x, y) = s 1 k=-s 1 s 1 l=-s 1 w 1 (k, l)I R (x -k, y -l), w 2 * I R (x, y) = s 2 k=-s 2 s 2 l=-s 2 w 2 (k, l)I R (x -k, y -l),
where s 1 and s 2 are the size of Gaussian filter and

w 1 (k, l) = exp -k 2 +l 2 2σ 2 1 s 1 k=-s 1 s 1 l=-s 1 exp -k 2 +l 2 2σ 2 1 , (k, l) ∈ [-s 1 , s 1 ] 2 . w 2 (k, l) = exp -k 2 +l 2 2σ 2 2 s 2 k=-s 2 s 2 l=-s 2 exp -k 2 +l 2 2σ 2 2 , (k, l) ∈ [-s 2 , s 2 ] 2 . Put r 1 (x, y) = 2a m=0 2a n=0 D 1 (m,n) (x, y) 2 and r 2 (x, y) = 2a m=0 2a n=0 D 2 (m,n) ( x, y) 2 (5.5)
where

D 1 (m,n) (x, y) = I 1 (x, y) -I 1 (x + (m -a), y + (n -a)) (5.6)
D 2 (m,n) (x, y) = I 2 (x, y) -I 2 (x + (m -a), y + (n -a)) . (5.7)
We will show that the local variability on blurred part is smaller than the local variability on focused part. So that we suppose without loss the generality that (x, y) ∈ B 2 (the blurred part of I 2 ) and we show that (r 2 (x, y) ≤ r 1 (x, y)). For that, we use the Plancherel theorem:

2a m=0 2a n=0 D 1 (m,n) (x, y) 2 = 1 (2a + 1) 2 2a p=0 2a q=0 D1 (p,q) (x, y) 2 , (5.8)
where D1 (p,q) (x, y) is the Fourier Transform of D 1 (m,n) (x, y).

D1 (p,q) (x, y) = FT [D 1 (m,n) (x, y))] = FT I 1 (x, y) -I 1 (x + (m -a), y + (n -a)) , (5.9)
56CHAPTER 5. PIXEL-LEVEL MULTI FOCUS IMAGE FUSION BASED ON NEIGHBOR LOCAL VARIABILITY As (x, y) ∈ B 2 therefore (x, y) B 1 , from (5.4), equation (5.9) can be written as follows

D1 (p,q) (x, y) = FT I R (x, y) -I R (x + (m -a), y + (n -a)) . (5.10)
and

I 2 (x, y) = s 2 k=-s 2 s 2 l=-s 2 w 2 (k, l)I R (x -k, y -l). (5.11)
By using the definition of convolution, eq (5.11) can be written as:

I 2 (x, y) = ∞ k=-∞ ∞ l=-∞ w 2 (k, l)χ [-s 2 ,s 2 ] 2 I R (x -k, y -l) (5.12)
and

I 2 (x, y) = w 2 χ [-s 2 ,s 2 ] 2 * I R (x, y) (5.13)
where

χ [-s 2 ,s 2 ] 2 (k, l) =        1, if (k, l) ∈ [-s 2 , s 2 ] 2 0, otherwise , The Fourier transform of D 2 (m,n) (x, y) is D2 (p,q) (x, y) = FT [D 2 (m,n) (x, y)] = FT I 2 (x, y) -I 2 (x + (m -a), y + (n -a))
as (x, y) ∈ B 2 we have

D2 (p,q) (x, y = FT w 2 χ [-s 2 ,s 2 ] 2 * I R (x, y) -w 1 χ [-s 2 ,s 2 ] 2 * I R (x + (m -a), y + (n -a)) = FT w 2 χ [-s 2 ,s 2 ] 2 * (I R (x, y) -I R (x + (m -a), y + (n -a))) = FT w 2 χ [-s 2 ,s 2 ] 2 FT I R (x, y) -I R (x + (m -a), y + (n -a)) (5.14)
Substitute (5.10) into (5.14), we get

D2 (p,q) (x, y) = FT w 2 χ [-s 2 ,s 2 ] 2 D1 (p,q) (x, y) =         ∞ k=-∞ ∞ l=-∞ w 2 (k, l)χ [-s 2 ,s 2 ] 2 (k, l)e -i2(kp+lq)         D1 (p,q) (x, y) =                  s 2 k=-s 2 s 2 l=-s 2 e -k 2 +l 2 2σ 2 2 s 2 k =-s 2 s 2 l =-s 2 e -k 2 +l 2 2σ 2 2 e -i2(kp+lq)                  D1 (p,q) (x, y). (5.15)
Hence from eq. ( 5.15), we can obtain

D2 (p,q) (x, y) = s 2 k=-s 2 s 2 l=-s 2 exp -k 2 +l 2 2σ 2 2 s 2 k =-s 2 s 2 l =-s 2 exp -k 2 +l 2 2σ 2 2 exp (-i2(kp + lq)) D1 (p,q) (x, y) = s 2 k=-s 2 s 2 l=-s 2 exp -k 2 +l 2 2σ 2 2 exp (-i2(kp + lq)) N k =-N s 2 l =-s 2 exp -k 2 +l 2 2σ 2 2 D1 (p,q) (x, y) ≤ s 2 k=-s 2 s 2 l=-s 2 exp -k 2 +l 2 2σ 2 2 s 2 k =-s 2 s 2 l =-s 2 exp -k 2 +l 2 2σ 2 2 D1 (p,q) (x, y) ≤ D1 (p,q) (x, y) . (5.16)
On the other hand from (5.5) and Plancherel-Parseval's theorem, we have

r 2 (x, y) = 2a m=0 2a n=0 D 2 (m,n) (x, y) 2 = 1 (2a + 1) 2 2a p=0 2a q=0 D2 (p,q) ( x, y) 2 , From (5.16)
, we get

r 2 (x, y) ≤ 1 (2a + 1) 2 2a p=0 2a q=0 D1 (p,q) (x, y) 2 , ≤ 2a m=0 2a n=0 D 1 m,n (x, y) 2 = r 1 (x, y).
This proves that the local variability on blurred part is smaller than the local variability value on clear area.

5.2.2/ THE FUSION SCHEME OF THE PROPOSED METHOD

The fact that even though there are already many multi-focus image fusion algorithms however many researchers keep developing the multi-focus image fusion field. In this work, we develop a novel fusion method that consists of weighting each pixel of each image by exponential of local variability. This local variability at (x, y) is calculated from the quadratic difference between the value of the pixel (x, y) and the all pixel values of its neighbors as explained in the previous chapter, in equation (4.8). Consider p original source images, I 1 , I 2 , • • • , I p , with different focus to be fused. The images here have the same size R × C and are assumed registered. Then, the steps of image fusion with size of nieghborhood "a" are as follows:

Step 1: For each pixel of each image, we calculate the local variability of every source image, v a,k (x, y) defined in (5.1):

v a,k (x, y) = 1 T a m=-a a n=-a I k (x, y) -I k (x + m, y + n) 2
(5.17) 58CHAPTER

PIXEL-LEVEL MULTI FOCUS IMAGE FUSION BASED ON NEIGHBOR LOCAL VARIABILITY

Step 2: The fusion image proposed, F, is calculated in the following model:

F(x, y) = p k=1 exp v a,k (x, y) I k (x, y) p k=1 exp v a,k (x, y) . (5.18)
Obviously, this method depends on the size "a". First, we tried with a small size (a = 1), we show that the NLV method is better than DWT. To improve this method and to compare with all other methods we optimize the value of "a" for having the minimum Root Mean Square Error (RMSE), where RMSE is defined in Chapter 3. For that, we show that the value of "a" depends on the blurred area.

Indeed, the choice of the size of the neighborhood "a" used in NLV method depends on variance (v) and the size(s) of the blurring filter. Our objective is to have a model that give the value of the "a" according to the "v" and "s". For that we take sample of 1000 images that we blurred using Gaussian filter with different values of v and s

(v = 1, 2, 3, • • • , 35 and s = 1, 2, 3, • • • , 20).
After that for each image I blurred with parameters "v" and "s", we apply our fusion method with different values of "a" ("a = 1, 2, , 17") and determine the value of "a" that gives the minimum RMSE, denoted by a l (v, s) where l is the index of l th image. Then we take the mean of the a l (v, s) for 1000 images, denoted a(v, s) , because the coefficient of variation is smaller than 0.1.

To propose a model, firstly, we have studied the variation of "a" in according to variance "v" for each fixed size of blurring filter "s". We remark that this variation is logarithmic. For example "s = 8" on Fig. 5.2., by using non linear regression we obtain model: 2.1096 ln v + 2.8689. In general, the model is

a(v, s) = c 1 (s) ln v + c 2 (s) (5.19)
where c 1 and c 2 are functions that depend on "s". The graphs that describe c 1 and c 2 , respectively, Fig.

       -0.5 ln (s) -2.655551 1.225175 2        (5.20)
As "a" is integer, we have two choices of a is either floor of a (v, s), denoted by a(v, s) or ceiling of a(v, s) , denoted by a(v, s) where x = min {n ∈ Z|n ≥ x} and x = max {m ∈ Z|m ≤ x}. Since the RMSE values of both "a" are very slightly different, then we can choose any "a" of them. We use "a" that gives the minimum RMSE in the following of this thesis, either a(v, s) or a(v, s) . We validate our model by applying it to 150 images (we generate 150 pairs multi focus images with various values of variance and size of blurring filter) and the result is as good as expected. It shows that our method is better than DST-LV and LP(DWT) methods.

To use this NLV method, we must firstly estimate the variance and the size of blurring filter. There exists some work that give the methods to estimate variance of blurring filter and the blur detection as in [2], [START_REF] Elder | Local Scale Control for Edge Detection and Blur Estimation[END_REF], [START_REF] Kumar | Tchebichef moment based restoration of Gaussian blurred images[END_REF], [START_REF] Liu | Image partial blur detection and classification[END_REF].

We also propose another method where we combine Laplacian pyramid method and NLV method. We use Laplacian pyramid with NLV as a selection rule, denoted by LP(NLV).

5.3/ EXPERIMENTAL RESULTS

The NLV method is performed on a datasets of images [57] using Matlab2013a. We artificially produce a pair of out of-focus images, by blurring the left part of reference image to obtain the image with in focus on the right and then blurring the right part of the reference image to produce the image with in focus on the left. Blurring process is accomplished by using a Gaussian filter with many values of variance and size. We have applied these methods on a database of 150 images. Furthermore, in the appendix table A.2, we present RMSE of all the images used for the comparison of methods.

The proposed method outperforms other methods. In order not to encumber this work we chose to present only three images (image 'bird', image 'bottle', and image 'building'), all images consist of two images with different focus and one reference image as is shown in A clearer comparison can be made visually by examining the differences between the fused images for different methods (Fig. 5.6) and reference images Fig. 5.5 (a). It can be seen that the fused image produced by NLV method is basically a combination of the good-focus parts in the source images. We can see blocking artifact on the result of DWT image fusion, compare with the other methods image fusion. To measure the differences between methods, we use the evaluation metrics (RMSE, NCC, SSIM, Mutual information). We have found that the NLV method performs better compared to other methods, see Table 5.1. For three images presented in this chapter and blurred with variance = 10 and size of blurring filter = 5, the model (5.20) gives the neighbor size "a" = 5 and "a" = 6. Here we use "a" = 6 because it result the smaller RMSE compared to "a" = 5 however the RMSE values of "a" = 5 and "a" = 6 are very slightly different. From the value of RMSE calculated for eleven methods in the Table 5.1., for image 'bird': the smallest is NLV method, the second smallest is DST-LV, the third is LP(NLV) and so on, as we can see on the table 5.1. NLV method is the best method among the methods and LP-NLV is better than LP-DWT. For image 'bottle', Fig. 5.7, subjectively it can be seen that the Bilateral gradient and MSSF methods does not handle the edge well compared the other method. On the other hand, the result image of NLV contains the balanced color and brightness as the original images to be fused. To confirm our result, we calculate the evaluation metrics: RMSE, NCC, SSIM and mutual information, see The NLV method provides better results as compared to other methods in this thesis after comparing the performance evaluation of all methods with each others.The NLV method gives significant improvement over other methods.

5.4/ CONCLUSION

In this work, we proposed image fusion method based on neighbor local variability (NLV). The precision of this fusion depends on the size of the neighborhood of local variability. The size of neighborhood is characterized by the variance and the size of blurring filter, more often that the blurring filter is Gaussian. The fused image is obtained by weighting each pixel by the exponential of the local variability. The result of experiment shows that the NLV method gives significant improvement result in both visually and quantitatively image fusion in comparison with ten other fusion methods. Laplacian pyramid with NLV as a selection rule is also applied, LP(NLV). Based on the experiment result, LP(NLV) is better than LP(DWT) and DWT.

CONCLUSION AND FUTURE WORK

The major concern of image fusion is to combine the relevant information from multiple image of a scene into a single more informative image. Image fusion methods have shown a great progress in recent years. The aim of this thesis is to study a multi-focus image fusion approach based on local variability.

In this framework, we develop three general fusion methods based respectively on the Laplacian pyramid with discrete wavelet transform (LP(DWT)), the theory of evidence of Dempster-Shafer Theory with a particular distance taking into account a local variability (DST-LV) and a model which consists of weighting the image by their local variability (NLV). These methods are applied to a database of images blurred by Gaussian filters.

The first method is to use the Laplacian pyramid as a technique of reduction and expansion then to fuse the images at each level of the Laplacian pyramid by discrete wavelet transform method. It gives a significant improvement of the two separate techniques.

We also introduce a criterion for each pixel of the image that measures the quadratic difference between this pixel and its neighbors. In this work, this quantization is called local variability at each pixel. It obviously depends on the size of the neighborhood used in image.

The second method proposed is to use the Dempster-Shafer theory by taking the local variability as a measure to calculate the mass function. This mass function allows to calculate the plausibility of each pixel that belongs to blurred part.

We show that the local variability of pixels in the blurred area is smaller than in the focus area. As we know that focus part contains more information than blurred part. This information corresponds to high value of local variability. From this result, we propose a method which consists of creating a fusion model by weighting each pixel by its local variability from each image. Then we use exponential function to boost the contribution of a pixel with higher value of local variability.

In order to optimize the results, a model is also proposed to determine the size of the neighborhood of the local variability according to the parameters of the blurring filter. A study based on several metrics of evaluation which compare the proposed methods to the different existing fusion methods shows the effectiveness of the proposed methods based on the local variability approach.

Compared to other methods given in the state of the art, all proposed methods provide a high spatial fused image, without any distortions and blocking artifacts. The experimental results show that the proposed NLV method is superior to other proposed methods in this 69 thesis and it greatly improves spatial resolution. Indeed, the Laplacian pyramid image fusion integrates multi-source information at the basic level and can provide more abundant, accurate and reliable detail information. It is also effective for preserving the edge. Furthermore, the DWT gives better preservation of both edge features and component information of the object in new fused image preserving in this way the detail image information. The DWT also retains the coefficients of in-focus regions within the image. So that our first proposed method, LP(DWT), which is a combination of Laplacian pyramid and DWT fusion give significant improvement.

The proposed methods based on local variability approach perform well because they work directly on each pixel by exploiting local variability that take into account the neighborhood so that it gives more detail information about the pixel.

Our proposed methods can be used in many applications, such as • Drone is a new technology in digital imaging, it has opened up unlimited possibilities for enhancing photography. Drone can capture images on the same scene that zooms in on different objects, and at various altitudes. It will produces several images on the same scene but with different objects in-focus.

• In medical imaging, the DST-LV can be used to detect an abnormal object or cell using local variability where the behavior of each pixel with its neighborhood is given.

• For quality control in of food industry, cameras are used that take pictures. each camera targets one of several objects to detect an anomaly. The objects are on a conveyor belt. To have a photo containing all the objects in-focus, we can use our proposed methods of fusion which gives more details information.

There are several perspectives of this work:

• As many work on image fusion have implemented on grayscale images. In this thesis, all proposed methods are performed on the grayscale image. However, these proposed methods can be extended to color images as color conveys significant information.

• We are also encouraged to fuse more than two images by taking into account the local variability in each image (intra variability) and variability between image (inter variability). This inter variability can detect the 'abnormal pixels' among the images.

• Very often the blurred image can be represented as the convolution between the clear image and Gaussian filter. We will investigate that if it is possible to approach any blurred image by Gaussian filter or more generally by alpha-stable filter.

• We have built a model of the size of the neighborhood for the third proposed method (NLV). The model is a function of the parameters of Gaussian filter. The future work is to estimate the parameters of Gaussian filter from the source blurred image.

• We are motivated to extend the DST-LV method to fuse images with different objects from different sensors (multimodal).

OUR PUBLICATIONS for j=a+1:S(2)-a d1(i,j)=sqrt((sum(sum((image1(i,j)-image1(i-a:i+a,j-a:j+a)).ˆ2)))./ ((2 * a+1).ˆ2-1)); end end for k=1:a for j=a+1:S(2)-a d1(k,j)=sqrt((sum(sum((image1(k,j)-image1(1:a+k,j-a:j+a)).ˆ2)))/((a+k) * (2 * a+1)-1)); d1(S(1)-k+1,j)=sqrt((sum(sum((image1( S(1)-k+1,j)-image1(S(1)-k+1-a:S(1),j-a:j+a)).ˆ2)))/((a+k) * (2 * a+1)-1)) ; end end for k=1:a d1(k,a)=sqrt((sum(sum((image1(k,a)-image1(1:k+a,1:a+a)).ˆ2)))/((k+a) * 2 * a-1)); end for k=1:a d1(a,k)=sqrt((sum(sum((image1(a,k)-image1(1:a+a,1:k+a)).ˆ2)))/((k+a) * 2 * a-1)); end for k=1:a for j=S(2)-a+1:S(2) d1(k,j)=sqrt((sum(sum((image1(k,j)-image1(1:k+a,j-a:S(2))).ˆ2)))/((k+a) * (S(2)-j+a+1)-1)); end end d1(1,1)=sqrt (sum(sum((image1(1,1)-image1(1:a+1,1:a+1)).ˆ2)))/ ( (a+1)ˆ2-1)); d1(S(1),1)=sqrt((sum(sum((image1(S(1),1)-image1(S(1)-a:S(1),1:a+1)).ˆ2)))/ ((a+1)ˆ2-1)); d1(S(1),S(2))=sqrt((sum(sum((image1(S(1),S(2))-image1(S(1)-a:S(1),S(2)-a:S(2))).ˆ2)))/ ( (a+1)ˆ2-1));

Figure 1 . 1 :

 11 Figure 1.1: Image fusion process

Figure 2 . 1 :

 21 Figure 2.1: Image fusion categorization

Figure 2 . 2 :

 22 Figure 2.2: multifocus mage fusion

Figure 2 . 3 :

 23 Figure 2.3: Flowchart of image fusion based using combination Laplacian pyramid and PCA method.

Figure 2 . 4 :

 24 Figure 2.4: Schematic diagram for the multi-focus image fusion using energy of Laplacian and guided filter.

  . WAVELET DECOMPOSITION IN LAPLACIAN PYRAMID FOR IMAGE FUSION Let m = 2m + m and n = 2n + n , we have m = m -2m and n = n -2n , by substituting m and n in (3.6), we get

Figure 3 . 1 :

 31 Figure 3.1: DWT image fusion

Figure 3 . 2 :

 32 Figure 3.2: The proposed method

  3.4 (a) focuses on the smaller clock and Fig. 3.4(b) focuses on the bigger clock.

Figure 3 . 4 :

 34 Figure 3.4: Source images 'clock': (a) image with focus on the small clock, (b) image with focus on the big clock

Figure 3 . 5 :

 35 Figure 3.5: The 'clock' fusion results

Figure 3 . 6 :

 36 Figure 3.6: RMSE of the LP(average), LP(maximum), DWT, and proposed method

Figure 3 . 7 :

 37 Figure 3.7: PSNR of the LP(average), LP(maximum), DWT, and proposed method

Figure 3 . 8 :

 38 Figure 3.8: Average gradient of the LP(average), LP(maximum), DWT, and proposed method

(I 1 ) (I 2 ) (I 3 )Figure 3 . 9 :

 12339 Figure 3.9: Multi-focus images:(I 1 ) in focus on the small bottle, (I 2 ) in focus on the gear and (I 3 ) in focus on the big bottle

2 Figure 3 . 10 :Fig. 3 .

 23103 Figure 3.10: The results of combination fusion

44CHAPTER 4 .

 4 MULTI-FOCUS IMAGE FUSION USING DEMPSTER SHAFER THEORY BASED ON LOCAL dence that can be associated with multiple possible events. There are three important functions in Dempster-Shafer theory: the basic probability assignment function (bpa or m), the Belief function (Bel), and the Plausibility function (Pl).

Figure 4 . 1 : 2 ( 4 . 8 ) 4 . 3 .

 4124843 Figure 4.1: Pixel at (x, y) within its neighborhood, a = 1

  {m(ω), m(ω), m(θ)} (4.10) with the condition m(ω) + m(ω) + m(θ) = 1.

F

  a (x, y) = I k 0 (x, y), where k 0 ∈ {1, 2, • • • , p} and Pl a,k 0 (ω)(x, y) = min k∈{1,2,••• ,p}

  4.2., with size 256x256 (R = C = 256) where the blurred images are obtained by performing low pas filtering using Gaussian filter with variance = 10 and filter size = 5. The first, image 'People' and the second image 'Bottle', all sets of image consist of different focus and one reference image. In this experiment, for the image 'People', we get the value of a = 7 and a = 9 for image 'Bottle'.

  in focus on the left (c) in focus on the right (d) reference image 'Bottle' (e) in focus on the left (f) in focus on the right

Figure 4 . 2 :Figure 4 . 3 :

 4243 Figure 4.2: Reference image and multi-focus images.

4

 4

Figure 4 . 4 :

 44 Figure 4.4: Comparison of visual quality of fused images various methods for image 'Bottle'.

4. 5

 5 / CONCLUSIONMulti-focus image fusion using Dempster-Shafer theory based on local variability (DST-LV) has been proposed in this Chapter. The method calculates the local variability for each pixel of each image and determines the mass function from local variability. The decision of fusion is obtained by pixels that correspond to minimum plausibility. The fused image produced by DST-LV method is basically a combination of the good-focus parts in the source images.The proposed method was compared to PCA,LP-PCA, DWT, LP(DWT), LP-PCA, DCT+Var, Bilateral gradient, MSSF, and EOL. Experimental results show that the proposed method has better performance in terms of both visual quality and objective evaluation.PIXEL-LEVEL MULTI FOCUS IMAGE FUSION BASED ON NEIGHBOR LOCAL VARIABILITY 5.1/ INTRODUCTIONDue to the limited depth-of-focus of optical lenses, it is often difficult to capture an image that contains all relevant objects in focus. Only the objects within the depth-of-field are in focus, while other objects are blurred. Multi-focus image fusion is developed to solve this problem. There are various approaches have been performed in the literatures.

Figure 5 . 1 :

 51 Figure 5.1: Two multi focus images, the yellow part is blurred area. And the white part is clear(focused) area.

Figure 5 . 2 :

 52 Figure 5.2: graph between "a" and variance of blurring filter where "s"=8

  5.3 and Fig. 5.4.

Figure 5 . 3 :Figure 5 . 4 :

 5354 Figure 5.3: graph of c 1 (s)

  Fig 5.5. In this work the multi-focus images presented in Fig 5.5 are obtained with variance = 10 and filter size = 5.

Figure 5 . 5 :Figure 5 . 6 :

 5556 Figure 5.5: The images used in the experiment.

Figure 5 . 7 :

 57 Figure 5.7: Experimental results of multi-focus image fusion image 'bottle'.

  function[gl1]=expand(gl0,w) [C,R]=size(gl0); gl1=zeros(2 * C-1,2 * R-1); %interpolation gl1(2:2:2 * C-2,2:2:2 * R-2)=4 * (gl0(1:C-1,1:R-1) * w(4,4)+gl0(2:C,1:R-1) * w(2,4)+ gl0(1:C-1,2:R) * w(4,2)+gl0(2:C,2:R) * w(2,2)); gl1(3:2:2 * C-3,2:2:2 * R-2)=4 * (gl0(1:C-2,1:R-1) * w(5,4)+gl0(2:C-1,1:R-1) * w(3,4)+ gl0(3:C,1:R-1) * w(1,4)+gl0(1:C-2,2:R) * w(5,2)+gl0(2:C-1,2:R) * w(3,2)+gl0(3:C,2:R) * w(1,2)); gl1(2:2:2 * C-2,3:2:2 * R-3)=4 * (gl0(1:C-1,1:R-2) * w(4,5)+gl0(1:C-1,2:R-1) * w(4,3)+ gl0(1:C-1,3:R) * w(4,1)+gl0(2:C,1:R-2) * w(2,5)+gl0(2:C,2:R-1) * w(2,3)+gl0(2:C,3:R) * w(2,1)); gl1(3:2:2 * C-3,3:2:2 * R-3)=4 * (gl0(1:C-2,1:R-2) * w(5,5)+gl0(2:C-1,1:R-2) * w(3,5)+ gl0(3:C,1:R-2) * w(1,5)+gl0(1:C-2,2:R-1) * w(5,3)+gl0(2:C-1,2:R-1) * w(3,3)+ gl0(3:C,2:R-1) * w(1,3)+gl0(1:C-2,3:R) * w(5,1)+gl0(2:C-1,3:R) * w(3,1)+gl0(3:C,3:R) * w(1,1)); %compute edges temp=w(5,3)+w(5,1)+w(3,3)+w(3,1)+w(1,3)+w(1,1); gl1(3:2:2 * C-3,1)=(gl0(1:C-2,1) * w(5,3)+gl0(1:C-2,2) * w(5,1)+gl0(2:C-1,1) * w(3,3)+ gl0(2:C-1,2) * w(3,1)+gl0(3:C,1) * w(1,3)+gl0(3:C,2) * w(1,1)); temp=w(5,3)+w(5,5)+w(3,3)+w(3,5)+w(1,3)+w(1,5); gl1(3:2:2 * C-3,2 * R-1)=(gl0(1:C-2,R) * w(5,3)+gl0(1:C-2,R-1) * w(5,5)+gl0(2:C-1,R) * w(3,3)+ gl0(2:C-1,R-1) * w(3,5)+gl0(3:C,R) * w(1,3)+gl0(3:C,R-1) * w(1,5)); temp=w(4,3)+w(4,1)+w(2,3)+w(2,1); gl1(2:2:2 * C-2,1)=(gl0(1:C-1,1) * w(4,3)+gl0(1:C-1,2) * w(4,1)+ gl0(2:C,1) * w(2,3)+gl0(2:C,2) * w(2,1)); temp=w(4,3)+w(4,5)+w(2,3)+w(2,5); gl1(2:2:2 * C-2,2 * R-1)=(gl0(1:C-1,R) * w(4,3)+gl0(1:C-1,R-1) * w(4,5)+ gl0(2:C,R) * w(2,3)+gl0(2:C,R-1) * w(2,5)); temp=w(3,5)+w(1,5)+w(3,3)+w(1,3)+w(3,1)+w(1,1); gl1(1,3:2:2 * R-3)=(gl0(1,1:R-2) * w(3,5)+gl0(2,1:R-2) * w(1,5)+gl0(1,2:R-1) * w(3,3)+ gl0(2,2:R-1) * w(1,3)+gl0(1,3:R) * w(3,1)+gl0(2,3:R) * w(1,1)); temp=w(3,5)+w(5,5)+w(3,3)+w(5,3)+w(3,1)+w(5,1); gl1(2 * C-1,3:2:2 * R-3)=(gl0(C,1:R-2) * w(3,5)+gl0(C-1,1:R-2) * w(5,5)+gl0(C,2:R-1) * w(3,3)+ gl0(C-1,2:R-1) * w(5,3)+gl0(C,3:R) * w(3,1)+gl0(C-1,3:R) * w(5,1)); temp=w(3,4)+w(1,4)+w(3,2)+w(1,2); gl1(1,2:2:2 * R-2)=(gl0(1,1:R-1) * w(3,4)+gl0(2,1:R-1) * w(1,4)+gl0(1,2:R) * w(3,2)+gl0(2,2:R) * w(1,2)); temp=w(3,4)+w(5,4)+w(3,2)+w(5,2); gl1(2 * C-1,2:2:2 * R-2)=(gl0(C,1:R-1) * w(3,4)+gl0(C-1,1:R-1) * w(5,4)+gl0(C,2:R) * w(3,2)+gl0(C-1,2:R) * w(5,2)); %compute corners temp=w(3,3)+w(3,1)+w(1,3)+w(1,1); gl1(1,1)=(gl0(1,1) * w(3,3)+gl0(1,2) * w(3,1)+gl0(2,1) * w(1,3)+gl0(2,2) * w(1,1)); temp=w(3,3)+w(5,3)+w(3,1)+w(5,1); gl1(2 * C-1,1)=(gl0(C,1) * w(3,3)+gl0(C-1,1) * w(5,3)+gl0(C,2) * w(3,1)+gl0(C-1,2) * w(5,1)); temp=w(3,3)+w(3,5)+w(1,3)+w(1,5); gl1(1,2 * R-1)=(gl0(1,R) * w(3,3)+gl0(1,R-1) * w(3,5)+gl0(2,R) * w(1,3)+gl0(2,R-1) * w(1,5)); temp=w(3,3)+w(5,3)+w(3,5)+w(5,5); gl1(2 * C-1,2 * R-1)=(gl0(C,R) * w(3,3)+gl0(C-1,R) * w(5,3)+gl0(C,R-1) * w(3,5)+gl0(C-1,R-1) * w(5,5)); function d1=local variability (im1,a) %input: im1 (image), a (size of neighborhood) %output: d1 (local variability) image1=double(im1); S=size(image1); for i=a+1:S(1)-a

  is the number of levels. A is an average filter..

	B k+1 n its previous level approximation layer B k is the approximation layer of n-th source image at level k + 1 which depend on n . B 0 n represents the n-th input image I n . * denote the convolution operation. The detail layer D k+1 n at level k + 1 are obtained
	by subtracting approximation layers B at previous level k from approximation layers
	B k+1 n	at present level k + 1.

Table 3 .

 3 1.

	Evaluation LP(Average)	LP(Maximum) DWT	Proposed
	RMSE	6.3602	6.4383	6.4685	17.0732
	PSNR	32.0615	31.9553	31.9147	23.4845
	Entropy	7.7500	7.7495	7.7486	7.7690
	Average	12.5887	12.5662	12.5565	10.2972
	Gradient

Table 3 .

 3

1: Performance evaluation of the fused image 'bottle'.

Table 3 .

 3

	3),1	F 123

2: Performance evaluation of the fused image 'bottle'.

Table 4

 4

	Image Evaluation PCA	DWT	LP	LP	DCT	Bilateral	EOL	MSSF DST-
					(DWT)	PCA	+var	Gradient	LV
	PeopleRMSE	9.4592 3.9258 2.6621 2.5962 2.0286 12.6216 1.6241 26.7263 0.3819
	a =	Mutual	4.6241 5.1307 5.4237 5.4778 7.3794 6.1377 7.5192 2.4742 7.6603
	7	information
		NCC	0.9930 0.9988 0.9994 0.9995 0.9997 0.9870 0.9998 0.9587 1.0000
		SSIM	0.9467 0.9808 0.9957 0.9959 0.9942 0.9121 0.9986 0.8436 0.9996
	Bottle RMSE	3.6392 12.1574 2.0605 2.0337 1.5442 12.6469 1.5593 25.0387 0.5958
	a =	Mutual	5.5539 4.6841 5.9879 6.0142 7.6391 6.4657 7.5768 2.5036 7.6999
	9	information
		NCC	0.9987 0.9857 0.9996 0.9996 0.9998 0.9837 0.9998 0.9529 1.0000
		SSIM	0.9856 0.9332 0.9945 0.9964 0.9966 0.9225 0.9983 0.8259 0.9990

.1. According to the four evaluation measures, the comparison results are shown in Table

4

.1., the proposed method gives the best result among the other methods.

Table 4 .

 4

1: Performance evaluation measures of fused images.

Table 5 .

 5 1: Performance evaluation image 'bird'

		PCA	DWT	LP	LP-	DCT+	bilateral	EOL	MSSF DST-	NLV	LP
				(DWT)	PCA	var	gradi-		LV		(NLV)
							ent
	RMSE 6.9205 3.5678 1.5190 1.4681 2.6860 8.8378 2.2792 10.4547 0.5766 0.5466 0.8253
	NCC	0.9881 0.9965 0.9994 0.9994 0.9980 0.9784 0.9986 0.9786 0.9999 0.9999 0.9998
	SSIM 0.9508 0.9747 0.9965 0.9966 0.9880 0.9220 0.9957 0.9477 0.9983 0.9981 0.9992
	Mutual	3.3233 3.7903 4.7704 4.8136 5.9425 4.5153 6.3188 3.3133 6.4519 6.1738 6.0114
	information

Table 5 .

 5 2. From the value of RMSE calculated for eleven methods in the Table 5.2., we can classify these methods from smaller values of RMSE that the smallest is NLV method, the second smallest is DST-LV, the third smallest is LP(NLV). RMSE 15.0059 5.3846 2.5282 2.4850 2.6421 20.3804 3.6811 16.9197 0.9761 0.9020 1.5843 NCC 0.9792 0.9971 0.9994 0.9994 0.9993 0.9570 0.9986 0.9774 0.9999 0.9999 0.9997 SSIM 0.8770 0.9625 0.9905 0.9933 0.9878 0.7870 0.9915 0.9231 0.9966 0.9968 0.9978 Mutual information 2.9261 4.1448 5.1300 5.1502 7.3880 4.8356 7.2055 3.0480 7.5430 7.3880 6.9144

	PCA	DWT	LP	LP-	DctVar Bilateral	EOL	MSSF DST-	NLV	LP
			(DWT)	PCA	gradient		LV		(NLV)

Table 5 . 2 :

 52 Performance evaluation image 'bottle' FigureFig. 5.8 shows the fused image results of multi-focus images 'Building', in visual term, we can see that the fused result of PCA method is reduced in contrast and the fused result of Bilateral gradient is not so clear compared to other methods. Moreover, to ensure that our proposed method is superior, four evaluation metrics are calculated: RMSE, NCC, SSIM, and mutual information, see Table5.3. From Table5.3, it shows that that NLV performed well.

	PCA	DWT	LP(DWT)
	LP-PCA	DCT+var	Bilateral Gradient
	EOL	MSSF	DST-LV
	Proposed: NLV		LP(NLV)

Figure 5.8: Experimental results of multi-focus image fusion image 'building'.

Table 5 .

 5 3: Performance evaluation image 'building'

  • I. S. Wahyuni, R. Sabre. Wavelet Decomposition in Laplacian Pyramid for Image Fusion. International Journal of Signal Processing Systems (IJSPS), Vol. 4, No. 1, pp.37-44, 2016. • I. S. Wahyuni, R. Sabre. Multi-focus image fusion using Laplacian Pyramid technique based on Alpha-Stable filter (accepted on Canadian Journal of Basic and Applied Sciences). .3854 0.7102 2.9530 1.7988 4.2671 17.3117 18.2511 2.8636 0.4865 1.6401 0.3117 0.6886 0.3233 1.5088 3.9754 4.8035 1.1074 0.5930 4.4319 0.5468 2.3495 1.7171 3.2360 11.6129 12.4607 1.7920 0.7430 5.2616 0.5460 2.6204 1.4519 3.1350 16.2315 17.8097 1.7217 0.4883 3.7527 0.4079 1.8593 1.3171 2.1649 10.5771 12.6049 2.5390 0.7653 5.0342 0.6916 2.4471 1.7512 3.1950 14.1441 14.8117 2.6779 1.4035 3.8599 0.5783 1.7523 1.2394 2.4210 8.5165 12.1122 1.8232 0.3846 2.1902 0.4146 0.7701 0.3732 1.6981 6.3697 9.2985 0.4679 0.7385 4.2009 0.7531 2.0868 1.3213 2.1937 11.5892 15.1582 2.6999 0.7359 5.9656 0.4109 2.0094 0.5254 6.3150 9.7482 9.8887 3.4813 0.3776 3.6548 0.4416 1.4770 0.5028 3.8559 4.3750 7.1215 3.6482 0.7003 5.7301 0.7028 3.0534 1.8419 4.5155 19.1566 15.7565 2.0156 0.4805 2.8528 0.4596 1.7552 0.6159 1.8010 10.0959 10.8574 3.3373 0.3290 3.0473 0.3405 1.3381 0.9109 1.4802 8.6130 9.6944 1.6873 0.9216 1.9525 0.3233 1.0203 0.9261 1.3925 5.8216 10.1735 1.0934 0.5546 3.5129 0.6223 2.0774 1.1388 2.1469 10.5076 13.0998 2.5408 1.7412 4.7572 0.7178 2.3047 1.3087 3.0818 11.2322 12.0947 2.7173 0.7078 4.6260 0.6018 2.1802 1.4538 2.1152 12.7995 13.6753 3.2565 0.9761 5.3846 0.9121 2.5282 1.5843 2.6421 20.3804 16.9197 3.6811 0.4984 2.9368 0.4674 1.8748 1.0595 1.5780 8.9907 9.7476 2.3928 1.1307 4.0736 0.9023 1.7742 0.9760 3.6102 8.2387 10.8573 1.6476 0.6383 2.7753 0.3655 1.2944 1.1004 1.8445 6.5271 9.2677 1.0160 0.3089 3.9911 0.4582 1.1222 0.4421 3.4288 6.8041 7.8254 0.8455 0.9196 5.5663 0.8779 2.9682 1.4563 5.6468 12.7165 10.7738 2.3468 0.7319 2.0478 0.3319 1.3271 0.7738 1.5615 4.2419 10.4678 1.4617 0.7499 3.0717 0.6477 1.7610 1.2867 1.4838 9.5144 8.4111 2.5640 0.6924 6.2204 0.8359 2.2511 1.0501 5.9803 11.2019 8.5379 2.5079 0.8624 5.2489 0.8048 2.5357 1.2518 3.9238 12.0070 14.9451 2.8721 1.8964 3.4598 0.6421 1.8000 0.8955 2.6851 8.0037 10.7645 3.1116 1.9635 6.1849 1.2541 2.6361 3.6381 3.7408 15.8929 14.2708 1.8469 0.6749 2.7288 0.3900 1.3402 0.5344 1.3481 8.6736 10.1631 2.0565 0.6701 3.6737 0.6083 1.9123 1.3384 2.9634 8.1539 24.4582 3.0072 0.5905 3.1536 0.4703 0.9927 0.6223 2.7126 5.6868 7.6505 0.9642 0.6940 4.3448 0.5395 1.7107 1.0782 3.1981 10.6561 10.9843 2.0839 0.5918 3.6953 0.5125 1.3463 0.8251 2.4619 8.3673 11.2739 1.6386 0.8187 3.8103 0.5921 1.7678 1.3413 2.2278 10.8730 13.2488 2.4653 0.4627 1.6038 0.3502 0.9118 0.4496 0.5165 5.9926 8.2142 1.5129 0.3668 2.9223 0.3592 1.5283 0.6610 1.2788 8.6825 9.7011 2.1519 0.6112 4.6429 0.6197 2.3445 1.0972 2.5301 12.4243 13.1776 4.1080 0.2702 2.6124 0.3229 1.2630 0.8122 1.2452 7.9279 11.0023 1.5993 0.7655 2.9465 0.6142 1.3157 0.8546 2.3129 9.3074 6.0007 1.8149 0.9989 4.4869 0.9430 2.0450 1.5326 3.1209 12.3171 13.8210 2.5687 0.5128 5.8194 0.4967 1.9990 0.3572 5.5664 8.9167 9.5729 2.3895 0.6927 5.9149 0.7156 3.5258 2.5914 2.5687 21.8379 20.4852 5.0583 0.7256 5.1692 0.6795 2.0261 1.0292 4.5819 9.0432 11.2907 3.5670 1.0160 3.3301 0.4858 1.3613 0.6191 2.5371 7.1254 9.6119 1.3916 1.1404 6.9691 0.9788 3.1263 1.6375 5.3573 17.9154 16.6834 2.9859 0.6669 2.7835 0.5695 1.8634 2.6765 0.9890 7.8641 10.0369 1.4241 0.6669 2.7835 0.5695 1.8634 2.6765 0.9890 7.8641 10.0369 1.4241 0.8008 4.2312 0.7292 2.3683 2.0098 2.4649 11.7454 14.4906 2.9060 1.3346 2.0427 0.3110 0.8051 0.3776 1.4920 4.0403 5.4671 1.4263 0.1707 0.7702 0.2384 0.6275 0.5562 0.1570 2.7531 3.8920 0.7925 1.0274 5.1980 0.9574 3.0429 2.0416 2.9524 15.6404 19.3963 3.4626 1.0071 5.2845 0.8388 2.6140 1.9765 3.5942 11.5004 14.2650 3.5576 0.9641 4.4214 0.8484 3.2627 2.4222 1.4919 16.3686 20.7950 3.9966 0.5924 2.6008 0.4948 1.5426 1.2894 1.6394 8.1598 6.0825 3.1537 0.9399 5.0578 0.6946 2.2345 0.7347 3.7339 10.9129 9.5417 2.9610 0.9471 5.7177 0.8197 2.1486 1.1914 4.9548 9.8391 11.7000 4.4961 0.7839 4.1740 0.5002 1.9516 0.8636 3.0860 9.9996 10.1731 2.7429 0.6228 5.9803 0.5674 3.3464 1.5761 4.8900 10.8907 11.8480 1.8417 0.5009 4.6246 0.5002 1.7986 0.7887 3.4432 11.4172 12.3007 2.1931 0.6791 6.1219 0.6127 3.6793 2.7237 2.9354 22.3216 25.7554 4.4897 0.8770 4.3915 0.6106 2.1682 1.3482 3.0537 11.4224 11.8260 3.4229 0.5437 5.4858 0.5651 2.2774 1.0594 5.3505 9.8912 11.5386 2.1005 0.4742 2.8849 0.3390 1.3849 1.2258 2.2396 8.6000 10.4385 1.8888 0.5448 2.4376 0.3362 1.0017 0.8235 2.3441 4.8856 6.1257 0.8113 0.7206 2.9181 0.6099 1.4530 1.0132 1.1908 6.9525 9.7148 1.4669 0.5570 4.5868 0.6450 1.7555 1.2175 3.8647 8.1368 11.6458 1.5967 0.8778 5.0708 0.8778 3.0527 1.7198 3.2624 17.9502 16.3969 4.1595 0.4286 2.9266 0.4399 0.9563 0.4672 2.7686 5.1030 10.7264 1.7450 1.5104 2.8082 0.3739 1.7942 1.4660 1.7104 6.2452 14.2197 1.2148 0.7979 5.8790 0.9343 2.5846 1.6155 4.3638 10.1335 10.3574 2.9049 0.5177 3.3298 0.4843 2.0040 1.5020 1.1596 10.5161 11.9300 2.1102 0.9394 5.1857 0.5691 2.3660 1.2811 3.7367 14.4164 13.6176 2.5827 0.5344 3.2520 0.4354 1.8440 1.1598 1.5823 10.0576 13.1441 1.7679 0.6721 3.7736 0.6237 1.8936 1.4776 2.0583 11.8497 14.3454 2.3371 0.6610 5.3986 0.4329 2.1841 1.1413 3.9493 13.2749 15.4439 2.9458 0.6144 4.7780 0.6031 3.2559 2.5049 1.8674 17.7929 19.8430 4.1279 0.4066 3.6312 0.4280 1.6082 0.9659 2.6047 8.5756 12.3321 2.2493 0.6268 4.0506 0.6101 2.0575 1.5382 2.3871 11.8575 14.4562 3.5936 0.6344 6.1762 0.6022 3.2957 1.7153 4.0142 18.1177 20.7751 4.1848 1.1952 2.2867 0.3247 0.8807 0.6456 1.3009 4.0007 5.4225 1.0025 1.1404 6.9691 0.9788 3.1263 1.6375 5.3573 17.9154 16.6834 2.9859 0.5403 4.1283 0.4432 2.4296 1.6489 2.0153 16.0467 15.1192 2.0914 Table A.2: Table RMSE of 150 images for DST method and NLV method

	A COMPARISON TABLE 5.9803 3.3464 4.6246 1.7986 6.1219 3.6793 4.3915 2.1682 5.4858 2.2774 2.8849 1.3849 1.7057 4.4629 0.5481 1.6287 1.1052 4.2112 6.7201 9.8758 1.7826 126 7.7982 10.5130 127 8.7562 11.6427 128 17.1037 22.4735 129 8.3532 11.5101 130 7.2792 9.9167 131 6.4475 8.4207 0.7576 5.5163 0.5102 2.5516 1.5857 3.1291 15.3525 17.8907 3.4541 0.5317 3.3771 0.4873 2.0591 1.6408 1.8465 12.3752 16.5393 2.7365 0.7449 4.5632 0.6730 2.3655 0.7299 3.4793 11.6880 9.3041 4.2320 0.2371 0.9222 0.2325 0.5219 0.2726 0.6936 1.9063 3.7717 0.7871 0.4867 2.4917 0.3967 1.1739 0.8300 1.7747 6.0726 8.4616 1.6548 1.2282 3.2261 0.3602 1.3899 1.0402 2.5323 5.9887 9.2390 1.2462 1.1671 4.0773 0.9956 2.3910 1.6026 2.2288 11.8693 9.4105 3.0072 0.8716 3.0058 0.3732 1.2506 0.6551 2.2520 7.0086 8.6464 1.7624 1.4664 5.9553 0.8851 3.0731 1.9290 3.0401 16.6178 17.4253 3.2712 0.4155 2.7129 0.3580 1.0907 0.9293 2.1263 5.4103 8.4434 0.8201 SOFTWARE IMPLEMENTATION 0.7388 6B B.2.2/ EXPAND FUNCTION
	132 1.0509 5.9271 0.8521 2.1197 0.9780 5.0875 11.4113 9.5884 2.8874 3.2537 4.6083 2.4376 1.0017
	133 0.9399 5.0578 0.6946 2.2345 0.7347 3.7339 10.9129 9.5417 2.9610 5.1697 6.9625 2.9181 1.4530
	134 0.9380 3.7722 0.4518 1.4146 0.7325 3.0072 8.0140 12.4737 1.5138 7.5748 10.0242 4.5868 1.7555
	135 0.7982 2.0777 0.4213 0.9389 0.5160 1.8682 3.1150 4.6310 1.5949 12.1422 15.7355 5.0708 3.0527
	136 0.6315 6.0786 0.7754 2.2316 1.0477 5.1572 10.8521 11.0927 3.3117 4.1710 5.7269 2.9266 0.9563
	137 0.4570 4.7092 0.4639 2.0839 1.2097 3.7234 12.7628 13.8193 2.1084 6.1089 7.8800 2.8082 1.7942
	A.1/ RMSE OF 150 IMAGES FOR LP(DWT) METHOD 138 8.0650 10.9014 5.8790 2.5846 0.7223 4.9991 0.5954 3.3709 2.5427 1.9039 18.1974 19.9892 3.3580 B.1/ BLURRING IMAGE
	139 1.1951 4.2662 0.5561 2.0033 1.0189 3.0640 10.2646 13.6351 3.3378 8.0450 10.8283 3.3298 2.0040
	Theory based on local variability (submitted on International Journal of Approximate Rea-soning) Image LP(Average) LP(Maximum) DWT 140 9.8665 12.5536 5.1857 2.3660 0.4330 3.5392 0.4321 1.4915 0.7810 2.2477 8.6253 12.9653 1.7949 LP(DWT) 6.9008 8.6008 3.5678 1.5190 10.4622 13.2814 6.7770 141 7.9449 10.9454 3.2520 1.8440 142 8.9358 12.0399 3.7736 2.3955 3.5651 0.7521 1.9504 1.2940 3.2176 5.1710 9.8090 1.4212 function [ im1,im2 ] = blur image( imr,s,v ) 1.8936 0.8139 3.6334 0.6020 2.6702 2.1556 2.1148 9.7981 13.2877 2.2251 %input image: imr (reference image), v (variance of Gaussian filter), 3.3991 7.4398 10.1568 3.6668 143 10.1926 13.1384 5.3986 2.1841 %s(size of Gaussian filter) 1.5639 4.5793 0.5898 2.2625 1.6946 2.6669 13.5146 15.7897 2.7949 1.7530 5.1192 6.7956 2.9586 144 13.9200 18.9018 4.7780 3.2559 %output: 2 blurred images (im1 and im2) 0.8677 4.8290 0.7079 2.0955 1.4841 3.1693 9.6714 14.6599 2.7325 1.2994 8.5383 11.6224 4.1174 145 6.9487 8.9942 3.6312 1.6082 0.9785 4.9964 0.8246 2.6051 1.4933 4.4465 10.2500 12.7287 2.0935 [rows, columns] = size(imr); 2.0467 6.0003 8.1245 3.2838 146 9.6435 13.5412 4.0506 2.0575 0.5519 4.0661 0.5396 2.2392 1.1421 2.4660 10.9568 12.8245 2.5884 midColumn = ceil(columns/2); 1.3721 6.3186 8.5827 3.2818 147 15.1067 20.2606 6.1762 3.2957 leftHalf = imr(:, 1:midColumn); 0.4884 3.3564 0.4629 2.0723 2.2374 1.0260 12.6672 17.0313 2.2655 1.3427 9.4100 12.6952 3.7657 148 3.6152 4.7940 2.2867 0.8807 rightHalf = imr(:, midColumn+1:end); 0.5977 4.1954 0.4652 2.1335 1.4856 1.8999 12.8084 14.7728 1.7152 1.8426 8.5444 11.4765 3.3784 149 13.1864 17.0256 6.9691 3.1263 0.6078 4.7195 0.6484 1.9451 1.0853 3.1984 10.8573 13.3436 2.0207 [x,y]=meshgrid(-s:1:s); 1.6162 6.4576 8.5471 2.6654 150 10.7505 14.0064 4.1283 2.4296 0.6934 6.1495 0.7511 2.9069 1.9587 5.4699 13.6390 16.6838 2.0447 1.4705 6.2681 8.3705 3.1307 1.4953 7.8582 10.4846 4.2101 r=((x).ˆ2+(y).ˆ2).ˆ(0.5); 0.6870 5.1122 0.6214 2.5685 1.4710 3.2146 11.2552 12.8992 2.3869 Table A.1: Table RMSE of 150 images for LP(DWT) method ga=v/2; 1.1781 4.9375 0.7657 1.9237 0.9830 4.5309 9.9621 13.0663 2.3916 2.1434 1.3905 1.8886 0.7199 0.7656 5.4614 0.6977 2.4884 1.7144 3.6305 12.7185 13.1104 4.7045 t=exp(-r.ˆ2/(4 * ga))/(4 * pi * ga); 0.3354 6.8797 9.2365 3.3951 1.4722 5.8954 7.8333 4.4629 1.6287 11.9543 16.3748 5.5163 2.5516 10.1952 12.5527 3.3771 5.1508 3.0536 0.4612 1.7742 2.3312 1.6608 9.1044 10.1143 0.4450 tg=t/sum(sum(t)); A.2/ RMSE OF 150 IMAGES FOR DST AND NLV Image DST-DWT NLV LP LP DCT Bilateral MSSF 0.8349 4.1305 0.6492 2.2643 2.2469 3.3530 10.4302 11.5939 1.6971 blurryLeft = imfilter(leftHalf, tg,'replicate'); 0.5916 4.4485 0.7000 3.2854 2.9196 0.0000 22.4423 23.6231 6.0015 blurryRight = imfilter(rightHalf, tg,'replicate'); EOL 0.5934 2.3804 0.4420 1.0079 0.6665 1.7282 5.4741 7.8809 0.9966 2.0591 9.0674 12.0423 4.5632 LV (DWT) (NLV) +Var Gradient 0.9811 7.6606 1.0529 2.9364 1.4657 5.9611 17.2135 16.2282 3.7555 2.3655 1.5731 2.3402 0.9222 0.5219 4.9178 5.9655 2.4917 1.1739 8.3875 11.6786 5.9271 0.5766 3.5678 0.5466 1.5190 0.8253 2.6860 8.8378 10.4547 2.2792 0.8554 6.7770 0.8422 3.3991 1.6660 5.8342 14.7006 14.1356 3.8413 D1={leftHalf, blurryRight}; 0.6919 5.3114 0.6770 2.9674 1.9514 2.7671 14.8315 13.3398 1.8851 im1=cell2mat(D1); 0.9064 5.7858 0.7982 3.7103 3.3422 2.1991 24.3093 22.9181 5.0594 figure,imshow(im1),title('blurry right') 0.7401 3.6668 0.3834 1.7530 0.8568 2.5415 10.3027 12.8118 1.9301 0.8613 5.9751 0.8304 3.4326 2.2419 2.6743 24.5014 22.1739 5.6461 2.1197 8.9952 12.4162 5.0578 0.2437 2.9586 0.2269 1.2994 0.2576 2.7217 5.2088 5.4510 4.1168 G1={blurryLeft,rightHalf}; 1.5001 7.1275 1.3016 3.5002 2.9765 4.9331 19.6468 18.9522 4.5448 2.2345 6.9439 9.0398 3.7722 1.4146 2.9413 3.8104 2.0777 0.8392 4.1174 0.6223 2.0467 1.0089 3.4253 8.4477 12.5065 3.0234 im2=cell2mat(G1); 0.3972 4.0390 0.2993 2.1493 2.2275 1.6420 19.5601 17.1296 2.8022 figure,imshow(im2),title('blurry left') 0.4965 3.2838 0.4715 1.3721 0.6136 2.8205 6.8490 9.1966 1.8962 0.7739 6.0696 0.8537 3.9630 3.7473 1.4589 26.4794 23.0023 6.2324 0.9389 9.9915 13.6476 6.0786 0.4086 3.2818 0.3592 1.3427 0.8021 1.8261 7.6890 12.2161 0.8328 end 0.8230 6.2952 0.7387 3.2444 2.1007 3.5080 17.7436 17.1508 4.0888 2.2316 9.5174 13.3018 4.7092 1.3426 3.7657 0.5215 1.8426 1.0823 2.2308 13.4102 12.7320 2.2213 0.6913 6.6705 0.6392 2.3117 1.6616 6.2170 18.2523 15.3365 4.3794 2.0839 13.9819 18.2567 4.9991 1.7216 3.3784 0.4607 1.6162 0.8784 2.4775 12.3683 12.1165 1.9937 0.7833 5.5845 0.7632 3.2907 2.7780 3.3811 22.6741 19.5367 6.1956 3.3709 8.5602 10.8736 4.2662 2.1413 2.6654 0.2636 1.4705 0.9869 1.2905 10.5622 11.9749 2.2624 2.9408 4.0191 0.7340 1.2848 0.7641 3.6526 4.4491 9.7584 1.2363 2.0033 7.0418 9.3307 3.5392 1.4915 4.9011 6.7289 3.5651 0.9756 3.1307 0.6026 1.4953 1.1869 2.2186 8.9969 10.4455 1.9645 0.6762 4.2101 0.6184 2.1434 1.3638 3.0857 10.2518 14.5641 1.2481 0.8724 5.1929 0.8934 3.5729 3.0577 3.3331 16.4376 19.4758 5.3093 B.2/ LAPLACIAN PYRAMID IMAGE FUSION 1.0098 2.9089 0.3911 1.4565 1.1549 1.6379 6.5375 9.7325 1.7475 1.9504 7.8913 11.3365 3.6334 0.5303 0.7199 0.1899 0.3354 0.2091 0.6287 1.4871 5.3482 0.4459 1.8802 4.3939 0.6187 2.1251 1.3799 3.1622 8.1497 12.2144 2.1378 2.6702 0.6502 3.3951 0.4370 1.4722 1.2077 2.1175 8.2507 10.9823 1.6850 0.6746 3.0374 0.5039 1.1602 0.7287 2.4244 6.5249 7.5616 1.4657 B.2.1/ REDUCE FUNCTION
	93

• I. S. Wahyuni, R. Sabre. Pixel level multi-focus image fusion based on local variability (submitted on IET Image Processing).

• I. S. Wahyuni, R. Sabre. Multi-focus Image fusion using Dempster Shafer

Mots-cl és : la fusion d'images multi focus, les pyramidales Laplaciennes, la transform ée en ondelettes discr ète, la variabilit é locale, la th éorie de Dempster-Shafer

ACKNOWLEDGEMENTS

First and foremost I would like to express my sincerest gratitude to my supervisor, Dr. Rachid Sabre, who has supported me thoughout my thesis with his patience and knowledge and given me a lot of guidance during my doctoral study and research.

I would like to thank the referees: Professor Jocelyn Chanussot and Professor Christian Germain for their time, interest, and helpful comments. I would also like to thank to all the jury members for their time, insightful questions, review and constructive suggestions on my oral defense.

I gratefully acknowledge the funding sources that made my study possible: the Directorate General of Higher Education (DIKTI), Ministry of Research and Technology Republic of Indonesia and Gunadarma University, Indonesia.

I would like to thank Ecole Doctoral SPIM, the laboratory LE2I, and Agrosup Dijon that have provided the support and equipment I have needed to produce and complete my thesis.

I would also like to thank my parents and family for their support and counsel . I thank my friends for supporting me throughout all my studies. Finally, I would like to thank all people who made this thesis possible.

 ---------------------Reconstruction LP DWT-------------------------------rfw2=fw2+expand1(fw3,t); rfw1=fw1+expand1(rfw2,t); rfw0=fw0+expand1(rfw1,t); f=rfw0; end

B.3/ LOCAL VARIABILITY

for l=1:a for i=a+1:S(1)-a d1(i,S(2)-l+1)=sqrt( (sum(sum((image1(i,S(2)-l+1)-image1(i-a:i+a,S(2)-l+1-a:S(2))).ˆ2)))/ ( (2 * a+1) * (l+a)-1)); d1(i,l)=sqrt ( (sum(sum((image1(i,l)-image1(i-a:i+a,1:l+a)).ˆ2)))/ ((2 * a+1) * (a+l)-1 )); end end for k=1:a d1(k,a)=sqrt((sum(sum((image1(k,a)-image1(1:k+a,1:a+a)).ˆ2)))/((k+a) * 2 * a-1)); end for k=S(1)-a+1:S(1) for j=S(2)-a+1:S(2) d1(k,j)=sqrt((sum(sum((image1(k,j)-image1(k-a:S(1),j-a:S(2))).ˆ2)))/((S(1)-k+a+1) * (S(2)-j+a+1)-1)); end end for k=1:a for i=S(1)-a+1:S(2) d1(i,k)=sqrt((sum(sum((image1(i,k)-image1(i-a:S(1),1:k+a)).ˆ2)))/((S(1)-i+a+1) * (k+a)-1)); if pls1(i,j)<pls2(i,j); f dst(i,j)=image1(i,j); elseif pls1(i,j)>pls2(i,j); f dst(i,j)=image2(i,j); elseif pls1(i,j)==pls2(i,j); f dst(i,j)=(image1(i,j)+image2(i,j))/2; for j=1:S(2) f(i,j)=(exp(d1(i,j)). * im1(i,j)+exp(d2(i,j)). * im2(i,j))./ (exp(d1(i,j))+exp(d2(i,j))); end end end Document generated with L AT E X and: the L AT E X style for PhD Thesis created by S. Galland -http://www.multiagent.fr/ThesisStyle the tex-upmethodology package suite -http://www.arakhne.org/tex-upmethodology/

Abstract:

In this thesis, we are interested in the multi-focus image fusion method. This technique consists of fusing several captured images with different focal lengths of the same scene to obtain an image with better quality than the two source images. We propose an image fusion method based on Laplacian pyramid technique using Discrete Wavelet Transform (DWT) as a selection rule. We then develop two multi-focus image fusion methods based on the local variability of each pixel. It takes into account the information in the surrounding pixel area. The first method is to use local variability as an information in the Dempster-Shafer theory. The second method uses a metric based on local variability. Indeed, the proposed fusion method weighs each pixel by an exponential of its local variability. A comparative study between the proposed methods and the existing methods was carried out. The experimental results show that our proposed methods give better fusions, both in visual perception and in quantitative analysis.