Member, IEEE Rocco Restaino

Member, IEEE Mauro Dalla Mura

Gemine Vivone

Fellow, IEEE Jocelyn Chanussot

Context-adaptive Pansharpening Based on Image Segmentation

Keywords: Context-adaptive Algorithms, Binary Partition Tree, Pansharpening, Image Segmentation, Image Fusion, Remote Sensing

Pansharpened images are widely used synthetic representations of the Earth surface characterized by both a high spatial resolution and a high spectral diversity. They are usually generated by extracting spatial details from a high resolution PANchromatic (PAN) image and by injecting them into a low spatial resolution MultiSpectral (MS) image. The details injection is performed through injection coefficients, whose values can be either uniform for the whole image (global methods) or spatially variant (context-adaptive approaches). In this paper, we propose a context-adaptive approach in which the injection coefficients are estimated over image segments achieved via a binary partition tree segmentation algorithm. The approach is applied to two credited pansharpening algorithms based on the Gram-Schmidt orthogonalization procedure and the generalized Laplacian pyramid technique. The performance assessment is performed using two different datasets acquired by the Quick-Bird and the WorldView-3 satellites. The validation procedure, both at full and at reduced resolution, shows the suitability of the proposed approach, which reaches a good trade-off between accuracy and computational burden.

I. INTRODUCTION

Pansharpening refers to the fusion of a high spatial resolution PANchromatic (PAN) channel with an image characterized by a higher spectral diversity (i.e. MultiSpectral (MS) or HyperSpectral (HS)) [START_REF] Aiazzi | Twentyfive years of pansharpening: A critical review and new developments[END_REF]- [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF]. The goal is to produce a synthetic image featuring both a high spatial and spectral resolution that are unachievable by a single sensor due to physical limitations. Pansharpening finds is widely exploited for enhancing images mapping the Earth surface in software platforms such as Google Earth and Microsoft Bing. However, the benefits of using pansharpened images for scientific applications have been demonstrated in many fields, such as spatial feature extraction [START_REF] Mohammadzadeh | Road extraction based on fuzzy logic and mathematical morphology from pansharpened IKONOS images[END_REF], snow mapping [START_REF] Sirguey | Improving MODIS spatial resolution for snow mapping using wavelet fusion and ARSIS concept[END_REF], and change detection [START_REF] Souza | Mapping forest degradation in the Eastern Amazon from SPOT 4 through spectral mixture models[END_REF], [START_REF] Bovolo | Analysis of the effects of pansharpening in change detection on VHR images[END_REF].

Pansharpening has gathered an increasing interest in the scientific community and in the last two decades a huge amount of methods have been proposed [START_REF] Aiazzi | Twentyfive years of pansharpening: A critical review and new developments[END_REF], [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], [START_REF] Thomas | Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics[END_REF], [START_REF] Amro | A survey of classical methods and new trends in pansharpening of multispectral images[END_REF]. R. Restaino is with the Department of Information Eng., Electrical Eng. and Applied Math. (DIEM), University of Salerno, 84084 Fisciano (SA), Italy; M. Dalla Mura is with GIPSA-Lab, Grenoble Institute of Technology, France; G. Vivone is with the North Atlantic Treaty Organization (NATO) Science and Technology Organization (STO) Centre for Maritime Research and Experimentation, 19126 La Spezia, Italy. J. Chanussot is with GIPSA-Lab, Grenoble Institute of Technology, France and with the Faculty of Electrical and Computer Engineering, University of Iceland. E-mails: restaino@unisa.it; {mauro.dalla-mura, jocelyn.chanussot}@gipsalab.grenoble-inp.fr; gemine.vivone@cmre.nato.int.

The pansharpening techniques found in the literature can be divided in "classical approaches" that are based on a simple model of fusion and that have been extensively studied and benchmarked [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF], and recent methods based on more complex models. These latter techniques can in some cases outperform classical techniques but with a significant increase of the computational burden. Within this family we can find Bayesian methods exploiting Total Variation penalization terms [START_REF] Palsson | A new pansharpening algorithm based on total variation[END_REF], [START_REF] He | A new pansharpening method based on spatial and spectral sparsity priors[END_REF] and the sparse representations of signals [START_REF] Li | A new pan-sharpening method using a compressed sensing technique[END_REF]- [START_REF] Vicinanza | A pansharpening method based on the sparse representation of injected details[END_REF] for regularizing the reconstruction of the ideal high resolution multichannel image from the available data.

A key result of the pansharpening literature states that nearly all the classical approaches can be recast into a unique framework [START_REF] Aiazzi | Twentyfive years of pansharpening: A critical review and new developments[END_REF], [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF]. More specifically, the pansharpening operation can be divided into two sequential phases: i) The extraction of the spatial details from the PAN image; and ii) The injection of the extracted details into the original MS image, upsampled to the PAN size. Thus, pansharpening algorithms differ in the way they extract the spatial details and they inject them into the MS image. According to the approach used to extract the PAN details, classical pansharpening algorithms can be divided into two classes: i) Component Substitution (CS) methods, which exploit the difference between the PAN image and a linear combination of the MS channels to extract details, and ii) MultiResolution Analysis (MRA) techniques, which extract the details from the PAN image and its low resolution version derived from a multiscale decomposition [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF].

The injection step is also a method-dependent operation which depends on the way the injection coefficients are defined. The definition of the injection coefficients is either done globally (i.e. only a single injection coefficient per spectral band is used for the whole image) or in a Context-Adaptive (CA) approach, in which the injection coefficients are spatially variant. Global estimation procedures of the injection coefficients have a low computational burden, whereas contextadaptive techniques can lead to better results since the injection coefficients are inferred locally on the image (via sliding windows), thus reducing the variance of the estimate [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], [START_REF] Aiazzi | A comparison between global and context-adaptive pansharpening of multispectral images[END_REF]. A simple solution to get a trade-off between performance and computational complexity consists in employing a contextadaptive approach based on non overlapping windows (i.e., a partition of the image on a grid) on which the estimation procedure is independently carried out [START_REF] Wang | A robust image fusion method based on local spectral and spatial correlation[END_REF]. A more sophisticated method instead involves a multiscale approach that is able to adapt the choice of a global and a local MRA method for each pixel by estimating the scale of the corresponding object in the considered scene [START_REF] Luo | Pansharpening with a decision fusion based on the local size information[END_REF], [START_REF] Luo | Decision-based fusion for pansharpening of remote sensing images[END_REF]. A further computational efficient method to implement a CA procedure is achievable by dividing the image in segments obtained by clustering the pixels according to a suitable criterion. This approach has been recently used in [START_REF] Garzelli | Pansharpening of multispectral images based on nonlocal parameter optimization[END_REF] for improving the use of the Minimum Mean Square Error (MMSE) rationale for the pansharpening problem. The k-means algorithm has been applied to the PAN image employing the pixels intensity and the local standard deviation as features in the unsupervised classification. Significant quality enhancement has been demonstrated in the paper with respect to the global estimation approach without significantly affecting the computational burden.

In this paper, we propose to employ a semantic partition of the image using a segmentation approach, whose goal is twofold: i) Group pixels with similar spectral characteristics of the details that are supposed to require equal values of the injection coefficients; ii) Improve the pansharpened product by letting the transitions among the zones characterized by different injection coefficients occur in correspondence of the boundaries of the object present in the scene, thus resulting less evident to the visual inspection. We apply this scheme to two different pansharpening algorithms, the Gram Schmidt Adaptive (GSA) [START_REF] Aiazzi | Improving component substitution pansharpening through multivariate regression of MS+Pan data[END_REF] and the Generalized Laplacian Pyramid with Modulation Transfer Function (MTF)-matched filter and regression based injection model [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF] (GLP) [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF], [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF] that represents two very efficient implementations of GS Mode 1 and GS Mode 2 [START_REF] Laben | Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening[END_REF], respectively. In both cases the coefficients are given by the regression of pixel values of the original MS channels and the corresponding pixel values of the low resolution version of the PAN image. However, the two methods are rather different in the practice, since while the GSA approach belongs to the CS class, the GLP algorithm fits in the MRA class, with substantially different features in their final outcomes.

Many segmentation algorithms can be used for this purpose. In this work we employed a Binary Partition Tree (BPT) [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF] that is a hierarchical representation of an image employing a region-based logic. The BPT has been selected for its ability in precisely delimiting homogeneous regions, by accurately preserving the object edges present in the scene. Furthermore it does not require prior information and the application of the pruning algorithm allows to consider regions coming from different levels of the hierarchy, thus permitting a precise representation of small as well as large regions. In greater details, a BPT is constructed through an iterative procedure that performs successive aggregations of pairs of adjacent regions starting from an initial partition of the image until a single region, correspondent to the whole image, is obtained. The sequence of region mergings corresponds to a binary tree structure in which the segments of the initial partition are the leaves and the whole image is the root. The intermediate nodes are the regions obtained during the merging phase that consists in combining the two most similar nodes at each step. Once the tree is built, image partitions can be obtained by pruning the BPT with a criterion that strongly depends on the specific application.

In order to assess the performance of the proposed algorithms, a dataset acquired by the QuickBird satellite and one by the more recent WorldView-3 satellite are considered. This allows us to test the capability of the proposed algorithm to deal with 4-channels and 8-channels MS images. Both the reduced resolution and the full resolution assessment procedures have been used for numerical evaluation of the proposed methods [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF]. In the former procedure, the original MS image plays the role of a reference, whereas the images acting in the fusion are obtained by properly degrading the available MS and PAN images, according to the Wald's protocol [START_REF] Wald | Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images[END_REF]. In particular, the original MS image is low pass filtered with a Gaussian kernel mimicking the MTF of the MS sensor, while the PAN image is processed by an almost ideal low pass filter [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF]. Both are then downsampled by a factor r equal to the ratio of the pixel sizes of the MS and PAN images. This procedure yields a couple of simulated images characterized by the same starting pixel size ratio r and with the PAN image at the same resolution of the reference image. Thanks to the presence of a reference image, it is possible to apply several quality indexes. However, the spatial degradation phase carried out to simulate the two low resolution starting images, introduces a bias for the filters used in the detail extraction step. Furthermore, the hypothesis of the invariance among scales of the fused outcomes is assumed to hold. For these reasons, a complementary procedure is carried out at full resolution to evaluate the performance of the pansharpening algorithms at the nominal resolution. However, in this case the lack of a reference image limits the validation. In both the cases, we demonstrate the suitability of the proposed approach by highlighting the significant advantages entailed by this procedure with respect to the global approach. Furthermore, the comparison to several state-of art techniques shows that the proposed method is able to achieve very high performance.

The work presented in this paper is an extension of previously reported progress on context-adaptive pansharpening using BPT [START_REF] Dalla Mura | Contextadaptive pansharpening based on binary partition tree segmentation[END_REF], [START_REF] Dalla Mura | Global and local Gram-Schmidt methods for hyperspectral pansharpening[END_REF]. It worths to remark that the main novelties of this paper with respect to the conference versions are: i) A more detailed analysis of the literature; ii) A broader experimental analysis at both reduced and full resolution; iii) A novel proposed approach based on Generalized Laplacian Pyramid, regression based injection model, and the BPT segmentation technique; iv) The application of the recent Extrapolated Q2 n [START_REF] Carla | Full-scale assessment of pansharpening through polynomial fitting of multiscale measurements[END_REF] index to optimize the free parameters of context-adaptive algorithms.

The remaining of the paper is organized as follows. In Sect. II, the context-adaptive pansharpening problem is formulated and reviewed. Sect. III is devoted to the review of the binary partition tree image segmentation approach. Its use in two well-established pansharpening methods is also proposed in this section. Experimental results using both the reduced and the full resolution validation procedures are discussed in Sect. IV. Finally, conclusions and future research lines arising from this work are drawn in Sect. V.

II. CONTEXT-ADAPTIVE PANSHARPENING

Pansharpening algorithms aim at producing a synthetic image M with the same spectral diversity of an available MS image M and the same spatial resolution of a companion PAN image P, whose pixel size is r times smaller than the one of M. The coefficient r is referred to as resize factor. Almost all classical approaches can be formulated with the same fusion model [START_REF] Aiazzi | Twentyfive years of pansharpening: A critical review and new developments[END_REF], [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF]. Specifically, by using the subscript k ∈ {1, . . . , N } to index the channels of the multispectral images involved in the process, the k-th band of the fused MS image M can be written as

M k = M k + G k • D k . (1)
In (1) M is the available MS image, upscaled to the PAN size, and • denotes the element by element multiplication of the three-dimensional injection coefficient matrix G and the detail image D. The channels of D are defined as

D k = P k -P LP k , (2)
namely, they are achieved by subtracting from the PAN image P k (in which the subscript k indicates that is eventually equalized with respect to M k ) its low resolution version P LP k . The technique used for constructing P LP k determines the two main categories of classical pansharpening algorithms, since it is obtained as the combination of the bands of M for Component Substitution (CS) approaches and through a multiscale decomposition of P k for MultiResolution Analysis (MRA) methods.

According to (1), given the P LP image, the subsequent step is the computation of the injection coefficient matrix G that represents another feature for distinguishing the different algorithms proposed in the literature.

An ideal formula could be derived if the target reference image, say M, would be available. In fact the definition

G = • M -M D •, ( 3
)
where • • • • denotes the element by element matrix division, easily leads to a fused product equal to the target image:

M k = M k + ( • M k -M k D k • ) • D k = M k . ( 4
)
However, the lack for the reference M precludes the application of (4) in the practice. On other hand, useful indications can be derived by employing this formula at reduced scale [START_REF] Wald | Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images[END_REF]. The application of this procedure to the images reported in Figs. 1(a) and 1(b) is depicted in Figs. 1(c) and 1(d), respectively; more specifically, the distribution of the optimal coefficients is illustrated in false colors whose intensities are proportional to the value of the injection coefficients of the related channels. Although this check is based on the scale invariance hypothesis, it confirms the intuition that the optimal coefficients are actually spatially variant. Thus, global approaches are expected to perform sub-optimally since they apply a single injection coefficient (possibly different for each band) to all pixels. This applies for example to techniques based on the additive injection method (or High Pass Filtering (HPF) scheme) [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], in which the coefficient matrix is composed of ones:

G k = 1. (5)
This class comprises algorithms such as the Intensity-Hue-Saturation (IHS) image fusion [START_REF] Carper | The use of intensity-huesaturation transformations for merging SPOT panchromatic and multispectral image data[END_REF], [START_REF] Chavez | Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic[END_REF], box filter High-Pass Filtering (HPF) [START_REF] Chavez | Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic[END_REF], Indusion [START_REF] Khan | Indusion: Fusion of multispectral and panchromatic images using the induction scaling technique[END_REF], and Generalized Laplacian Pyramid (GLP) with MTF-matched filter with unitary injection model (GLP-HPF) [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF]. Other approaches exploiting a global estimation method are, for example, the Principal Component Analysis (PCA) [START_REF] Chavez | Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic[END_REF], the Additive À Trous Wavelet Transform with unitary injection model (ATWT) [START_REF] Núñez | Multiresolution-based image fusion with additive wavelet decomposition[END_REF], the Additive Wavelet Luminance Proportional (AWLP) [START_REF] Otazu | Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods[END_REF], the Band-Dependent Spatial-Detail method with Global estimation of coefficients (G-BDSD) [START_REF] Garzelli | Optimal MMSE pan sharpening of very high resolution multispectral images[END_REF], and the classical implementations of Gram-Schmidt (GS) approaches [START_REF] Aiazzi | Improving component substitution pansharpening through multivariate regression of MS+Pan data[END_REF], [START_REF] Laben | Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening[END_REF].

On the contrary, algorithms based on the multiplicative injection method (or High Pass Modulation (HPM) scheme) [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF] use a different injection coefficient for each pixel, since in this case the coefficient matrix is defined as

G k = • M k P LP k • . ( 6
)
The Brovey transform (BT) [START_REF] Gillespie | Color enhancement of highly correlated images-II. Channel ratio and "Chromaticity" Transform techniques[END_REF], the Smoothing Filter-based Intensity Modulation (SFIM) [START_REF] Liu | Smoothing filter based intensity modulation: A spectral preserve image fusion technique for improving spatial details[END_REF], [START_REF] Wald | Comment: Liu 'Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details[END_REF], and the GLP with MTF-matched filter and multiplicative injection model (GLP-HPM) [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF], [START_REF] Aiazzi | An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas[END_REF] are successful examples of this class of pansharpening algorithms.

The last option consists in estimating the entries of G locally in the image. Image blocks, either overlapping or not, are often used. For instance, a rectangular sliding window is employed to perform the pixel-wise optimal estimation in the Minimimum Mean Square Error (MMSE) sense [START_REF] Garzelli | Pansharpening of multispectral images based on nonlocal parameter optimization[END_REF], [START_REF] Garzelli | Optimal MMSE pan sharpening of very high resolution multispectral images[END_REF], or to calculate local statistics that represent the heart of the Context-Based Decision (CBD) [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF] and the Enhanced Context-Based (ECB) methods [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF]. Of course, different criteria can be obtained by replacing the spatial distance with a different similarity measure. For example, the intensity and the local standard deviation of the PAN image are used in [START_REF] Garzelli | Pansharpening of multispectral images based on nonlocal parameter optimization[END_REF] for clustering the image pixels before the MMSE estimation of the coefficients. We refer to this algorithm as Context-adaptive Band-Dependent Spatial-Detail C-BDSD.

III. BINARY PARTITION TREE FOR PANSHARPENING

This work is focused on the estimation of the injection coefficients in a context-adaptive manner. Specifically, we rely on a segmentation of the scene for defining L regions in the image domain, on which the injection coefficient will be estimated. As shown in Figs. 1(c)-(d) one can easily note that similar values of the optimal coefficients correspond to pixels related to the same feature of the illuminated scene. We consider a segmentation based on the BPT (Sect. III-A) for partitioning the image which is used to locally estimate the injection coefficients. Thus, in this section, we review first the BPT image segmentation approach. Afterwards, its use with two well-established pansharpening methods belonging to the component substitution and the multi-resolution analysis families is proposed.

A. Binary Partition Tree Image Segmentation

The Binary Partition Tree (BPT) is a tree structure, whose nodes represent image regions obtained by merging segments of an initial fine image partition (i.e., an over-segmentation of the image or even the set of pixels itself) [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF]. The BPT is constructed by progressively merging the two most similar adjacent regions until the whole image is obtained and the merging sequence defines the binary hierarchical structure. The construction phase heavily relies on the way the region similarity is computed. Specifically, this is implemented considering a region model M R , which specifies how a region R is described, and a merging criterion O(M Ri , M Rj ) quantifying the similarity of two regions R i and R j in terms of their models M Ri and M Rj .

The BPT built can start from the finest partition of the image, which is given by the set of connected components (and might correspond to a partition in which each pixel is associated to a region). However, since regions at the pixel level are in many cases not semantically meaningful, a common procedure consists in considering an over-segmentation in which segments correspond to small regions that are spectrally homogeneous (i.e., super-pixels) [START_REF] Valero | Comparison of merging orders and pruning strategies for binary partition tree in hyperspectral data[END_REF], [START_REF]Hyperspectral image representation and processing with binary partition trees[END_REF]. Since the aim is to preserve all the boundaries contained in the image, in this work the initial partition is generated by a watershed algorithm applied to the morphological gradient of the MS image, which is obtained as the maximum of the morphological gradients1 of the single spectral channels.

The subsequent step in BPT construction is constituted by the successive operations of region merging, which iteratively define the partition tree structure. At each step the following three operations are carried out: i) Find the most similar pair of neighboring regions according to O(•, •); ii) Create a new region by merging the two selected regions and insert it as a new node in the tree; iii) Update the values of the similarity between the newly constructed region and its neighboring regions. Several definitions are exploitable as region models and merging criteria [START_REF] Salembier | Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval[END_REF], [START_REF] Valero | Comparison of merging orders and pruning strategies for binary partition tree in hyperspectral data[END_REF], [START_REF]Hyperspectral image representation and processing with binary partition trees[END_REF] and the final choice depends on the desired semantic of the segmentation. In our case, the goal is to partition the scene in spectrally homogeneous regions (corresponding to land cover made up of same materials). Thus, a suitable region model should rely on spectral vectors (i.e., I{p} = [I 1 {p}, I 2 {p}, . . . , I N {p}] containing for each pixel p of image I the values of the spectral bands). Perhaps the simplest option in this case is to consider the first order region model (i.e., the mean vector):

M R = 1 |R| ∑ p∈R I{p}, (7)
in which |R| denotes the number of pixels contained in region R. For computing the similarity between regions, we considered the Spectral Angle Mapper (SAM) as region merging criterion [START_REF] Veganzones | Hyperspectral image segmentation using a new spectral unmixing-based binary partition tree representation[END_REF], which is defined as:

O(M Ri , M Rj ) = SAM [M Ri , M Ri ] = arccos ( ⟨ M Ri , M Rj ⟩ ∥M Ri ∥ M Rj ) , (8)
where ⟨•, •⟩ and ∥ • ∥ denote the scalar product and the norm, respectively. Since the SAM relies on the angles between spectral vectors, it can be a suitable distance for distinguishing different materials regardless their illumination conditions, which is related to their modules [START_REF] Keshava | Distance metrics and band selection in hyperspectral processing with applications to material identification and spectral libraries[END_REF]. The image segmentation is then obtained by pruning the BPT, since each cut in the BPT structure corresponds to a partition of the image domain. In this application, the pruning is obtained by fixing the number L of segments in the segmentation map in order to control the computational complexity of the injection coefficient estimation (which depends on the number of regions in the segmentation map). The segmentation map is thus composed of the regions in the last L -1 merging steps of the BPT construction.

The segmentation algorithm exploited in this work is summarized in Algorithm 1.

B. Local Injection Coefficient Estimation: GSA and GLP using BPT

The focus of this work is the application of the image segmentation to the Gram Schmidt Adaptive (GSA) [START_REF] Aiazzi | Improving component substitution pansharpening through multivariate regression of MS+Pan data[END_REF] and to the Generalized Laplacian Pyramid with MTF-matched filter and regression based injection model [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF] (GLP) [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF], [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF] pansharpening algorithms.

On the one hand they are different approaches, belonging to the CS and the MRA class, respectively. Indeed the low pass version P LP of the PAN image, required for constructing

(R i , R j ) ∈ R (n-1) × R (n-1) (R l , R m ) = arg min Ri,Rj O(M Ri , M Rj ).
-Merge R l and R m in R lm and define

R (n) = R (n-1) \ {R l , R m } ∪ R lm -Compute: -the region model M R lm -for all the neighbors R j of R lm update O(M R lm , M Rj ) -Set R = R (n)
the image of details D = P -P LP , is obtained through the application, for each k = 1, . . . , N of the formulas

P LP k = N ∑ k=1 α k M k , for GSA, (9)
P LP k =P k * h k , for GLP, (10)
where α k are weights estimated through a regression procedure and • * h k is a Linear Time-Invariant (LTI) filter obtained by convolution with a kernel h k whose frequency response amplitude matches the MTF of the sensor acquiring the k-th channel of the MS image.

On the other hand both algorithms are instances of the Gram-Schmidt method, sharing the formula for the evaluation of the injection coefficients. To this aim, the regression between the MS channels and a low resolution version of the PAN image is employed, leading to the following expression

G k = Cov[ M k , P LP k ] Cov[P LP k , P LP k ] , (11)
in which Cov[X, Y] denotes the covariance of X and Y, which is valid for the original implementation employing a global estimation procedure. Eq. ( 11) is easily implemented locally by computing G k on each image region. Formally, the entry G

G l k = Cov[R M l , R P l ] Cov[R P l , R P l ] -Gather {G l k } l=1,...,L in G k -Compute the fused image M k according to M k = M k + G k • D k . pixel p is calculated as G (p) k = Cov[R M p , R P p ] Cov[R P p , R P p ] , ( 12
)
wherein R M p and R P p denote the regions of the M k and P LP k images that include p, respectively. According to this procedure the same G k is applied to all pixels belonging to the same region R. Differently from [START_REF] Aiazzi | A comparison between global and context-adaptive pansharpening of multispectral images[END_REF], in which rectangular (overlapping or not) blocks are considered, in this work the regions R M p and R P p are extracted by image partition based on the BPT algorithm.

The proposed context-adaptive pansharpening algorithm based on segmentation is thus articulated in four steps: i) The segmentation of the image through the application of the BPT method to the image M; ii) The extraction of the detail image D; iii) The evaluation of the injection coefficients matrix G through formula [START_REF] Palsson | A new pansharpening algorithm based on total variation[END_REF]; iv) The fusion of the available M and P according to the expression [START_REF] Aiazzi | Twentyfive years of pansharpening: A critical review and new developments[END_REF]. The pansharpening algorithm based on BPT is summarized in Algorithm 2.

IV. EXPERIMENTAL RESULTS

We considered two datasets for the experimental comparison of the algorithms: the first one has been collected by the Quickbird sensor and is composed by a PAN channel and a 4-bands MS image (Blue, Green, Red and NIR). It will be referred to as the Indianapolis dataset from the city represented in the images. The multispectral bands have a Several algorithms, selected from the CS and the MRA classes, have been employed for comparison in this work. The acronyms BT, AWLP, G-BDSD, and C-BDSD have already been defined in Sect. II. The other compared algorithms are: the MS image interpolation, using a polynomial kernel with 23 coefficients (EXP) [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], the Generalized Intensity-Hue-Saturation (GIHS) [START_REF] Tu | A new look at IHSlike image fusion methods[END_REF], the Partial Replacement Adaptive Component Substitution (PRACS) [START_REF] Choi | A new adaptive component-substitutionbased satellite image fusion by using partial replacement[END_REF], the GLP with additive injection model (GLP-HPF) [START_REF] Aiazzi | Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis[END_REF], [START_REF] Aiazzi | MTFtailored multiscale fusion of high-resolution MS and Pan imagery[END_REF], the GLP with multiplicative injection model (GLP-HPM) [START_REF] Aiazzi | An MTF-based spectral distortion minimizing model for pan-sharpening of very high resolution multispectral images of urban areas[END_REF] and the Sparse Representation based methods proposed in [START_REF] Li | Remote sensing image fusion via sparse representations over learned dictionaries[END_REF] (SR-Li) and in [START_REF] Zhu | A sparse image fusion algorithm with application to pan-sharpening[END_REF] (SR-Zhu). As suggested in the original papers, the SR-Li algorithm was implemented by employing 3 × 3 patches, overlapping each other by 2 pixels, and the SR-Zhu algorithm by employing 7×7 patches overlapping each other by 3 pixels. Only for the reduced resolution assessment, the quality indexes related to the Ground Truth (GT) are reported in the first row of the table and represent the best achievable values.

Finally, the quality indexes corresponding to the selected algorithms are computed; for both GSA and GLP methods, the reported values are obtained by approximating the statistical quantities required by [START_REF] Alparone | Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data fusion contest[END_REF] with sample values calculated in multiple different ways: glob refers to the calculation of a single coefficient for the whole image, which is obtained by using all the pixels; the block method consists in partitioning the images in squared blocks and in computing a single value of the injection coefficient for all the pixels included in each block; ovlp denotes the pixel-wise estimation of the injection coefficients through a sliding window that delimits the pixels included in calculation; the last three methods are based on the application of the segmentation algorithm to partition the pixels before the coefficients estimation. More in details, bpt indicates the method proposed in this paper, in which the BPT algorithm is applied to MS (the original MS image upscaled to the PAN size), kmc-m relies on the segmentation of MS through the k-means clustering (KMC), while kmc-p consists in applying the k-means clustering to the intensity and the local standard deviation of the PAN image, like in [START_REF] Garzelli | Pansharpening of multispectral images based on nonlocal parameter optimization[END_REF].

As in [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], the first two moments of the PAN image have been equalized with respect to the original MS image before the detail extraction phase. Actually, the results of some algorithms are independent of this procedure.

A. Reduced Resolution Validation

The assessment procedure performed at reduced resolution is able to accurately evaluate the algorithms' capability of reproducing the original MS image, which acts as the Ground Truth (GT), starting from a degraded version of the available TABLE I: Quality indexes obtained by the compared algorithms for the two reduced resolution datasets. Values of the Q2 n , ERGAS, and SAM are reported. Val indicates the block dimension for the block method, the window size for the ovlp approach, and the number of segments for the the bpt, kmc-m, kmc-p and the C-BDSD methods. For each dataset, the best results are pointed out in boldface and the second best in italic characters. The best results among the GSA and GLP approaches are underlined. data. Both a quantitative analysis, based on some credited quality indexes, and a visual inspection of the pansharpened products have been carried out.

Val

The main elements for completing the quantitative evaluation are contained in Tab. I that reports the values of three different quality indexes. Among the wide class of measures applicable to this case, we selected the Spectral Angle Mapper (SAM) [START_REF] Yuhas | Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm[END_REF] to quantify the spectral accuracy of the pansharpened product, the Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS) [START_REF] Wald | Data Fusion: Definitions and Architectures -Fusion of images of different spatial resolutions[END_REF], which is primarily related to the radiometric distortion, and the comprehensive vector quality index Q2 n [51], [START_REF] Garzelli | Hypercomplex quality assessment of multi-/hyper-spectral images[END_REF], which quantifies both spatial and spectral quality. Optimal values are 0 for the SAM and the ERGAS, and 1 for the Q2 n . The Q2 n index is also used for the optimization of the parameters characterizing the specific methods, i.e. the size in pixels of the partitioning if a sufficiently large value is chosen. However, the range of suitable values of L has to be estimated for each image, since it depends on the number of objects present in the scene, as well as on the image size, being the output of BPT constituted by connected regions.

The main analysis is devoted to the comparison of the performance achievable by the GSA and GLP pansharpening algorithms. By focusing attention on the two groups of seven methods reported at the bottom of Tab. I, one can easily note that the coefficient estimation technique based on image segmentation turns out to be particularly advantageous, achieving the best performance for both the GSA and GLP approaches. Giving more insight into the comparison of the segmentation methods, the approaches using the spectral features of the images, derived through the application of the BPT and the KMC algorithms to the MS image, achieve superior quality images. Generally, better results are obtained by the BPT with respect to KMC corroborating the choice of the algorithm made in this work.

In the specific case of the GLP methods, the further comparisons to the GLP-HPF and the GLP-HPM method are surely remarkable, since they represent two very efficient implementations of pansharpening algorithms, employing MTFmatched filters for detail extraction [START_REF] Vivone | Contrast and error-based fusion schemes for multispectral image pansharpening[END_REF]. The execution times, calculated on an Intel R ⃝Core TM I7 3.2GHz processor, point out that the ovlp method mainly constitutes a theoretical approach, since the computational burden results unaffordable taking into account the actual size of images used in the practice. On other hand, the execution times of the bpt methods are not significantly greater than those of the competitors. The reported values do not include the time to complete the segmentation step, which is not strictly related to the fusion phase; it is required only when the segmentation of the illuminated scene has never been performed or is outof-date. Fig. 4 justifies the numerical results by showing the comparison of the errors committed in estimating the optimal coefficients given by ( 4). The figure refers to the GLP case and shows the difference between the error related to the proposed bpt approach and three commonly employed procedures, i.e. the HPF scheme, the HPM scheme, and the regressionbased approach applied on the whole image (glob). In all the cases the green zones, highlighting the pixels for which the proposed method entails an advantage, are wider than the complementary red zones. In particular, the figure points out that the proposed scheme achieves a fruitful compromise between the parts of the images that require a point-wise estimation of the injection coefficients, i.e., where the HPM approach is advisable, and more homogeneous zones that benefit from a global estimation procedure, as obtained by the HPF and the glob approach.

Despite the current study is focused on the different ways for estimating the injection coefficients in the GSA and GLP pansharpening schemes, some other high performance methods have been added to link the measured values to an absolute quality scale. Since the original GSA and GLP algorithms are already reference approaches within the CS and the MRA classes [START_REF] Vivone | A critical comparison among pansharpening algorithms[END_REF], respectively, we expected very high performance for the proposed methods. Indeed, the comparison with the other state-of-the-art methods highlights the superior performance obtained, in particular, by the GLP-bpt algorithm. High-quality results are also attained by compressive sensing algorithms that exploit a local injection of coefficients as well, being the image divided in patches during the pansharpening process. However, at the same time, Tab. I evidences that the computational burden of these algorithms remains their main drawback.

The visual analysis, here reported with reference to the Indianapolis dataset, allows for several further considerations. In Fig. 5 we show a portion of the pansharpened images obtained by employing the four versions of the GS-based methods and by some other considered algorithms. The comparison is not simple, but it can be facilitated by also inspecting the weighted injected details, i.e., the difference between the fused image M and the interpolated MS image M, which are reported in Fig. 6. In both the figures, the artifacts involved by the block and, to a minor extent, by the ovlp methods are particularly clear. On the contrary, only with the help of Fig. 6 the spectral improvements achievable by employing the bpt approach instead of the glob estimation approach can be easily remarked. Indeed, the presence of the reference image allows to appreciate the similarity of the detail image colors yielded through the bpt method. The valuable accuracy of the injected details is also confirmed after comparing the corresponding images produced by the other three reported algorithms. In particular, the proposed approaches employing the BPT segmentation are able to obtain a well-balanced reproduction of the missing information, strongly limiting under-and over-injection effects.

The link between the quantitative and the qualitative analyzes can be found by inspecting the distribution of the ERGAS, depicted in the subplots of Fig. 7. Even from this point of view, the homogeneity of the committed error is the most valuable quality of the proposed method; indeed the areas characterized by a large error, which correspond to red pixels, are rather small. For instance, with regard to the reported portion of the image, a remarkable behavior is exhibited by the GLP-bpt method on the areas occupied by the red roofs and in the bottom-right vegetated zone, where the chance of injecting the marked discontinuities present in the PAN image is highly probable.

B. Full Resolution Validation

The full resolution assessment is hard to tackle by a quantitative point of view, but it constitutes a necessary step for evaluating the performance of the pansharpening algorithms at the real scale. A starting point is represented by the consistency property of the Wald's protocol, which requires that the original MS image is obtained again by degrading the pansharpened image [START_REF] Wald | Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images[END_REF]. This procedure has to be implemented by preserving the characteristics of the corresponding sensor, i.e. by employing a degradation filter matched to the MTF of the MS acquisition device. With this caveat, the consistency property has been widely used for the spectral assessment of the final products, by comparing the degraded pansharpened image to the original MS image. Indeed, this procedure constitutes the basis for calculating classical quality indexes, as the SAM, and novel accuracy measures, as the spectral distortion D λ employed in the Khan's protocol [START_REF] Khan | Pansharpening quality assessment using the modulation transfer functions of instruments[END_REF], which coincides with the complement to one of the Q2 n value between the two low frequency images. In this work we report both these values, while we commit the evaluation of the spatial quality of the pansharpened images to the spatial distortion D S of the QNR protocol [START_REF] Alparone | Multispectral and panchromatic data fusion assessment without reference[END_REF]. The suitability of the joint evaluation of such spatial and spectral distortions for assessing the overall quality of the image has been recently stated in [START_REF] Aiazzi | Full scale assessment of pansharpening methods and data products[END_REF], where the comprehensive index obtained by multiplying the quantities 1-D S and 1 -D λ has been named as Hybrid QNR (HQNR).

In the case of tunable algorithms the choice of a suitable index, computable at the original resolution, is required for optimizing the design parameters. As anticipated in the previous section, the Q2 n has been shown to be the most predictable classical index at full resolution [START_REF] Carla | Full-scale assessment of pansharpening through polynomial fitting of multiscale measurements[END_REF]. Accordingly, we maximize the Extrapolated Q2 n (abbreviated as Q2 n -E) to tune the free parameters characterizing the various approaches. For the quantitative evaluation of the compared algorithms, the values of the SAM, D S , D λ , and HQNR are reported in Tab. II, which also contains the maximum value of Q2 n -E achievable with each method and the corresponding value Val of the optimal parameter.

The full scale assessment procedure underlines the need for a local estimation of the coefficients. Indeed, the comparison among the different approaches indicates that the glob method often achieves very low performance, when compared to the other techniques. In particular, only a slight improvement can be observed by employing the block estimation procedure, while the segmentation of the image can appreciably help the pansharpening process by properly partitioning the pixels in homogeneous classes. In that way, the bpt method reaches very high performance for both the datasets. Analogously, the ovlp method represents the strongest competitor, sometimes achieving comparable, or else TABLE II: Quality indexes obtained by the compared algorithms for the two full resolution datasets. Values of the Q2 n , SAM, D S , D λ and HQNR are reported. Val indicates the block dimension for the block method, the window size for the ovlp approach, and the number of segments for the bpt, kmc-m, kmc-p and the C-BDSD methods. For each dataset, the best results are pointed out in boldface and the second best in italic characters. The best results among the GSA and GLP approaches are underlined. better, performance; however the required computational effort surely recommends the bpt technique as the most viable approach to implement a spatially variant version of the injection procedure. Notably, the ranking of the proposed algorithms after the comparison to the other state-of-the-art algorithms is well-preserved at full resolution. As expected, more fluctuations can be observed among the quality indexes, but the best average behavior is surely exhibited by the GLPbpt approach that often achieves the best results. The visual analysis of the pansharpened images allows to draw very similar considerations with respect to the reduced resolution assessment. Even in this case, we focus on the analysis of the injected details, which turn out to be easier to read with respect to the final product. The images related to the Sidney full resolution dataset are reported in Fig. 8. For both the GSA and the GLP techniques, the comparison of the images related to the glob and bpt methods shows a desirable inhomogeneity of the colors obtained with the latter method, whose correctness however cannot be assessed due to the lack of the reference image. For that reason, we report in the third column of Fig. 8 the difference between the SAM maps related to the glob and bpt approaches, highlighting in green the positive values and in red the negative ones. The superior extent of the green zones indicates that the spectral accuracy is surely improved through the bpt approach. We further show, in the last column of Fig. 8, a comparison among the spatial precision of the details of the pansharpened image. To this aim, since the sole full resolution available image is the PAN, we compare it to an equivalent high resolution PAN image obtained as the linear combination of the channels of the pansharpened product. The combination weights are estimated through a MMSE procedure performed at reduced resolution. Fig. 8(d) and Fig. 8(h) report the difference of the local values of the spatial Correlation Coefficient (sCC) [START_REF] Zhou | A wavelet transform method to merge landsat TM and SPOT panchromatic data[END_REF] between the original PAN image and the equivalent PAN image related to the glob and the bpt methods, in the case of GSA and GLP pansharpening methods, respectively. The presence of an almost similar (and small) number of green and red pixels testifies that no significant improvement is obtained in terms of the spatial quality of the image details that is in fact primarily related to the common extraction phase. Namely, as expected, the most significant contribution of the proposed method based on the image segmentation is obtained in terms of an increase of the spectral accuracy.

V. CONCLUSIONS AND FUTURE DEVELOPMENTS

This paper has focused on the estimation of the coefficients used for weighting the spatial details extracted from a PAN image when injected into the MS channels, which is a key step in classical pansharpening approaches. In particular, we have proposed a context-adaptive (i.e., local) estimation of the coefficients based on image segmentation. The injection coefficients are then region-dependent, meaning that all pixels belonging to the same region in the segmentation map will share the same coefficients (which can be with respect to those estimated on other regions). The image defining the set of regions on which the are computed, is obtained a segmentation based the Partition Tree applied to the MS image. The proposed adaptive estimation the injection coefficients has been integrated into two widely used pansharpening schemes. The first one belongs to the component substitution family (i.e. it is based on the Gram-Schmidt orthogonalization procedure), whereas the second one falls into the multi-resolution analysis class based on the generalized Laplacian pyramid decomposition.

The proposed segmentation-based context-adaptive approach has been compared to several other local schemes for the estimation of the injection coefficients considering two real images acquired by the QuickBird and the WorldView-3 satellites. The experimental analysis, conducted both at reduced and at full resolution, demonstrated the significant improvement in performance for both the proposed contextadaptive pansharpening algorithms with respect to their classical global implementation. Furthermore, the comparison to other credited state-of-the-art approaches proved that the proposed coefficient estimation method based on binary partition tree segmentation is able to obtain, on one hand, very high performance, especially when it is used in conjunction with the generalized Laplacian pyramid decomposition scheme, and, on other hand, a reasonable computational burden.

An interesting different testbed that deserves further investigations is represented by hyperspectral images. Another further investigation line goes towards the application of other segmentation techniques in order to yield a better estimate of the injection coefficients.

Fig. 1 :

 1 Fig. 1: Comparison of the datasets used for the reduced resolution assessment and the distribution of the optimal coefficients obtained through (4): Indianapolis (on the left) and Sidney (on the right). (a-b) Original MS image (used as the GT); (c-d) False color representation of the optimal injection coefficients: red, green, and blue colors represent the values related to the red, blue, and green channel coefficients, respectively.

Algorithm 1 :

 1 BPT-Segmentation Algorithm Data: The image I to segment, the number of segments L, the region model M R , the merging criterion O(M Ri , M Rj ). Result: The segmentation R = {R i } i=1,...,L . begin -Initialization: -Generate an over-segmentation of the image composed by N 0 > L regions: R (0) = {R i } i=1,...,N0 -Compute the region models {M Rj } j=1,...,N0 according to (7) -Compute the values of the merging functions O(M Ri , M Rj ) according to (8) for n = 1, . . . , N 0 -L do -Find the two most similar neighboring regions among the set

Algorithm 2 :

 2 Pansharpening Algorithm Based on BPT-Segmentation Data: The available images M and P, the number of segments L, the region model M R , the merging criterion O(M Ri , M Rj ). Result: The pansharpened image M. begin -Upscale image M to the scale P, yielding M -Extract the detail image D k = P k -P LP k , where P LP k is defined according to (9) and (10) for GSA and GLP, respectively -Segment image M in L regions, through Algorithm 1, obtaining the partition: R = {R i } i=1,...,L for k = 1, . . . , N do for l = 1, . . . , L do -Calculate the injection coefficients for each region l as

Fig. 2 :

 2 Fig. 2: Datasets used for the assessment. From the left to the right: Indianapolis reduced resolution dataset and Sidney reduced resolution dataset, Indianapolis full resolution dataset and Sidney full resolution dataset. The first row contains the high resolution PAN image and the second row contains the low resolution MS image upscaled to the PAN size (EXP).

Fig. 4 :

 4 Fig. 4: Reduced resolution Indianapolis dataset: the difference of the errors committed in estimating the optimal injection coefficients; the segmentation-aided (bpt) method is compared to: (a) HPF; (b) HPM; (c) glob. Green values indicate better results for the bpt method, red values indicate better results for the compared method.

Fig. 5 :

 5 Fig.5: Reduced resolution Indianapolis dataset: a small portion of the final products related to some compared algorithms, which are indicated under the corresponding image.

Fig. 6 :Fig. 7 :Fig. 3 :

 673 Fig. 6: Reduced resolution Indianapolis dataset: the details corresponding to the images shown in Fig. 5.

Fig. 8 :

 8 Fig. 8: Sidney dataset: comparison of the details injected through the global (glob) and segmentation-aided (bpt) estimation of the injection coefficients: GSA method (first row) and GLP method (second row). (a) and (e): details injected through the glob approach; (b) and (f): details injected through the bpt approach; (c) and (g): difference of SAM related to the glob and bpt methods; (d) and (h): difference of the equivalent PAN error related to the glob and bpt methods. Green values indicate better results for the bpt method, while red values indicate better results for the glob method.

  Rocco Restaino (M'14) received the Laurea degree in electronic engineering from the University of Naples, Naples, Italy, in 1998, and the Ph.D. degree in information engineering from the University of Salerno, Italy, in 2002. He is currently an Assistant Professor at the University of Salerno. His research interests include probability theory, stochastic geometry and signal processing for remote sensing and networking. He was awarded with the IGARSS 2015 Symposium Best Paper Award. Gemine Vivone received the B.Sc. (summa cum laude), the M.Sc. (summa cum laude), and the Ph.D. (highest rank) degrees in information engineering from the University of Salerno, Salerno, Italy, in 2008, 2011, and 2014, respectively. He is currently a Scientist at the North Atlantic Treaty Organization (NATO) Science & Technology Organization (STO) Centre for Maritime Research and Experimentation (CMRE), La Spezia, Italy. In 2014, he joined the NATO STO CMRE, La Spezia, Italy as a Research Fellow. In 2013, he was as a Visiting Scholar with Grenoble Institute of Technology (INPG), Grenoble, France, conducting his research at the Laboratoire Grenoblois de l'Image, de la Parole, du Signal et de l'Automatique GIPSA-Lab. In 2012, he was a Visiting Researcher with the NATO Undersea Research Centre, La Spezia, Italy. His main research interests focus on statistical signal processing, detection of remotely sensed images, data fusion, and tracking algorithms. Dr. Vivone serves as a Referee for several journals, such as IEEE Transactions on Geoscience and Remote Sensing, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, and IEEE Geoscience and Remote Sensing Letters. Dr. Vivone was the recipient of the Symposium Best Paper Award at the IEEE International Geoscience and Remote Sensing Symposium (IGARSS) 2015.

The morphological gradient is the difference between a dilation and an erosion with a minimal size structuring element[START_REF] Soille | Morphological Image Analysis: Principles and Applications[END_REF].

President of IEEE Geoscience and Remote Sensing French chapter (2007-2010) which received the 2010 IEEE GRS-S Chapter Excellence Award. He was the co-recipient of the NORSIG 2006 Best Student Paper Award, the IEEE GRSS 2011 and 2015 Symposium Best Paper Award, the IEEE GRSS 2012 Transactions Prize Paper Award and the IEEE GRSS 2013 Highest Impact Paper Award. He was a member of the IEEE Geoscience and Remote Sensing Society AdCom (2009-2010), in charge of membership development. He was the General Chair of the first IEEE GRSS Workshop on Hyperspectral Image and Signal Processing, Evolution in Remote sensing (WHISPERS). He was the Chair (2009-2011) and Cochair of the GRS Data Fusion Technical Committee (2005-2008). He was a member of the Machine Learning for Signal Processing Technical Committee of the IEEE Signal Processing Society (2006-2008) and the Program Chair of the IEEE International Workshop on Machine Learning for Signal Processing, (2009). He was an Associate Editor for the IEEE Geoscience and Remote Sensing Letters (2005-2007) and for Pattern Recognition (2006-2008). Since 2007, he is an Associate Editor for the IEEE Transactions on Geoscience and Remote Sensing. He was the Editor-in-Chief of the IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (2011-2015). In 2013, he was a Guest Editor for the Proceedings of the IEEE and in 2014 a Guest Editor for the IEEE Signal Processing Magazine. He is a Fellow of the IEEE and a member of the Institut Universitaire de France (2012-2017).