Jefersson Alex

Philippe-Henri Santos

Sylvie Gosselin

Ricardo Philipp-Foliguet

S Da

Alexandre Xavier Torres

Falcão

Multi-Scale Classification of Remote Sensing Images

Keywords: Multi-Scale Classification, Remote Sensing Image, Boosting, Support Vector Machines, Image Descriptors

A huge effort has been applied in image classification to create high quality thematic maps and to establish precise inventories about land cover use. The peculiarities of Remote Sensing Images (RSIs) combined with the traditional image classification challenges made RSIs classification a hard task. Our aim is to propose a kind of boost-classifier adapted to multi-scale segmentation. We use the paradigm of boosting, whose principle is to combine weak classifiers to build an efficient global one. Each weak classifier is trained for one level of the segmentation and one region descriptor. We have proposed and tested weak classifiers based on linear SVM and region distances provided by descriptors. The experiments were performed on a large image of coffee plantations.

We have shown in this paper that our approach based on boosting can detect the scale and set of features best suited to a particular training set. We have also shown that hierarchical multi-scale analysis is able to reduce training time and to produce a stronger classifier. We compare the proposed methods with a baseline based on SVM with RBF kernel. The results show that the proposed methods outperform the baseline.

I. INTRODUCTION

Since the satellite imagery information became accessible to the civil community in the 1970s, a huge effort has concerned the creation of high quality thematic maps to establish precise inventories about land cover use [START_REF] Wilkinson | Results and implications of a study of fifteen years of satellite image classification experiments[END_REF]. However, the peculiarities of Remote Sensing Images (RSIs) combined with the traditional image classification challenges have turned RSIs classification into a hard task.

The use of RSIs as a source of information in agribusiness is very common. In those applications, it is fundamental to know the ground occupation. However, identification and recognition of crop regions in remote sensing images are not trivial tasks. In Brazil, for example, coffee is one of the most important farming. This work, which is part of a Brazilian project involving a cooperative of coffee producers, aims at finding the coffee plantations in remote sensing images. Classification of RSIs meets some specific issues in agriculture. Concerning the identification of coffee areas, the difficulties come from the fact that coffee usually grows in mountainous regions (as in Brazil). Firstly this infers shadows and distortions in the spectral information, which make difficult the classification and the interpretation of shaded objects in the image because the spectral information is either reduced or totally lost [START_REF] Zhou | Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study[END_REF]. Secondly, the growing of coffee is not a seasonal activity, and, therefore, in the same region, there may be coffee plantations of different ages, which also affects the observed spectral patterns.

Typically, the classification process of RSIs uses supervised learning, which can be divided in the following components: data representation, feature extraction, and training. Data representation indicates the objects for classification (e.g., pixels [START_REF] Showengerdt | Techniques for Image Processing and Classification in Remote Sensing[END_REF], blocks of pixels [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF], regions [START_REF] Benz | Multi-resolution, object-oriented fuzzy analysis of remote sensing data for gis-ready information[END_REF], and hierarchy of regions [START_REF] Chen | Hierarchical object oriented classification using very high resolution imagery and lidar data over urban areas[END_REF]).

Feature extraction provides a mathematical description for each object (for example, spectral characteristics, texture, and shape). Training learns how to separate objects from distinct classes by building a classifier based on machine learning techniques (for instance, support vector machines [START_REF] Tzotsos | Object-based image analysis through nonlinear scale-space filtering[END_REF], genetic programming [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF],

neural networks [START_REF] Ouma | Multiscale remote sensing data segmentation and post-segmentation change detection based on logical modeling: Theoretical exposition and experimental results for forestland cover change analysis[END_REF]). The final quality of the classification depends on the performance of each step. For example, the classification result relies on the accuracy of the employed learning techniques. Regarding the performance of learning algorithms, it is directly dependent on the quality of the extracted image features. Finally, features are extracted according to the model used for data representation.

Regardless of the data representation model adopted in supervised classification of RSIs, both the training input and the result of the classifier can be expressed as sets of pixels. In spite of that, data representation cannot rely only on pixels, because their image characteristics are not usually enough to capture the patterns of the classes (regions of interest). In order to bridge that semantic gap, multi-scale image segmentation can play an important role. As pointed out by Trias-Sanz et al. [START_REF] Trias-Sanz | Using colour, texture, and hierarchial segmentation for high-resolution remote sensing[END_REF], most image segmentation methods use threshold parameters to create a partition of the image. These methods usually create a single-scale representation of the image: small thresholds give segmentation with small regions and many details while large thresholds preserve only the most salient regions. The problem is that various structures can appear at different scales and this segmentation result can be difficult to obtain without prior knowledge about the data or by using only empirical parameters. It is difficult to define the optimal scale segmentation. Some parts of an image may need a fine segmentation, since the plots are small, whereas, in other parts, a coarse segmentation is sufficient. For this reason, the main drawback of classification methods based on regions is that they depend on the segmentation method used. Moreover, it is difficult to define the optimal scale segmentation. Bearing this in mind, many researchers have exploited multiple scales of data [START_REF] Tzotsos | Object-based image analysis through nonlinear scale-space filtering[END_REF][START_REF] Ouma | Multiscale remote sensing data segmentation and post-segmentation change detection based on logical modeling: Theoretical exposition and experimental results for forestland cover change analysis[END_REF][START_REF] Wang | Object-oriented change detection method based on multi-scale and multi-feature fusion[END_REF][START_REF] Kim | Multi-scale geobia with very high spatial resolution digital aerial imagery: scale, texture and image objects[END_REF][START_REF] Tzotsos | Support vector machine classification for object-based image analysis[END_REF][START_REF] Valero | New hyperspectral data representation using binary partition tree[END_REF].

Allied to the problem of finding the best scale of segmentation, there is the problem of selection/combination of extracted features. In addition to this, several studies show that the combination of features can improve classification results [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF][START_REF] Santos | A relevance feedback method based on genetic programming for classification of remote sensing images[END_REF].

Our aim is to propose a kind of boost-classifier adapted to multi-scale segmentation, taking advantage of various region features computed at various levels of segmentation. To building multi-scale classifiers, we propose two approaches for multi-scale analysis of images: the Multi-Scale Classifier (MSC) and the Hierarchical Multi-Scale Classifier (HMSC). The MSC is based on the Adaboost algorithm [START_REF] Schapire | A brief introduction to boosting[END_REF], which builds a strong classifier from a set of weak ones. The HMSC is also based on boosting of weak classifiers, but it relies on a sequential strategy of training, according to the segmentation hierarchy of scales (from coarser to finer). In this work, we proposed two configurations of weak learners: Support Vector Machine (SVM) and Radial Basis Function (RBF). The RBF approach is based on the distances provided by the used descriptors.

We segment the image using several scales of interest regions, from the pixel level to the image level, using Guigues algorithm [START_REF] Guigues | Scale-sets image analysis[END_REF]. Instead of choosing any particular scale, which is usually not enough to represent all regions of interest, we use the segmentation results from several scales and let the choice of the most relevant regions and of the most discriminative features between relevant and non-relevant samples be done by the learning machine. Our method differs from the others in four main aspects. First, it does not rely on any particular scale (empirical parameter) and, thus, it can capture the most relevant regions in different parts and scales of the image and the most discriminative features at various scales, from fine to coarse. Moreover, it exploits the results of auxiliary scales to improve classification, even when there is a single optimum scale. Furthermore, it combines classification results from different scales rather than fusing features. Last, it assigns the same set of classes for all scales, producing a single final result, instead of producing a distinct classification result per scale.

The use of the proposed method depends only on the used descriptors. Thus, the proposed method can be used to classify any image/region, given that the descriptors are suitable for the target image/region.

It is important to clarify that the method will better work in images with a reasonable level of noise and resolution, in which representative features can be extracted from both small and large regions. We validate the proposed approach on a dataset composed by SPOT images obtained from a traditional place of coffee cultivation in Brazil. This paper is outlined as follows. Section II covers related work. Section III presents some concepts related to representation and description necessary to understand our proposed approach. Section IV introduces our method for multi-scale training and classification. Experimental results are presented in Section V. In Section VI, we present our conclusion and discuss future work.

II. RELATED WORK

A study of published works between 1989 and 2003 [START_REF] Wilkinson | Results and implications of a study of fifteen years of satellite image classification experiments[END_REF] examined the results and implications of RSI classification research. According to this study, despite the high number of approaches in that period, there was no significative improvement in terms of classification results. Most of the proposed methods were pixel-based. These methods try to estimate the probability of each pixel to belong to the possible classes by employing statistic measures based only on spectral properties. The Maximum Likelihood Classification (MaxVer) [START_REF] Showengerdt | Techniques for Image Processing and Classification in Remote Sensing[END_REF] remains one of the most popular methods for RSI classification. MaxVer computes the probability of each pixel to belong to each of the defined classes and uses that information to assign the class with the highest probability.

The improvements in sensor technologies have increased the accessibility to high-resolution images.

As a result, new approaches have been developed to make a better use of the available data. This led to researches that take into account the neighborhood of the pixels in the analysis and, thus, texture features in the classification of RSIs. In [START_REF] Lu | A survey of image classification methods and techniques for improving classification performance[END_REF], a general overview of RSI classification until 2005 is presented. That work discusses the challenges and describes all the steps that compose the classification process. Various classification methods are presented and grouped according to its taxonomy.

More recently, a new trend can be observed. Many studies [START_REF] Gigandet | Region-based satellite image classification: method and validation[END_REF][START_REF] Lee | Image classification with a region based approach in high spatial resolution imagery[END_REF][START_REF] Yu | Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery[END_REF][START_REF] Katartzis | A hierarchical markovian model for multiscale region-based classification of vector-valued images[END_REF] consider information encoded in regions (group of pixels) for RSI classification tasks. Gigandet et al. [START_REF] Gigandet | Region-based satellite image classification: method and validation[END_REF] proposed a classification algorithm for high resolution RSIs, combining non-supervised and supervised classification strategies. In this method, regions were classified by using Mahalanobis distance and Support Vector Machines (SVM).

Lee et al. [START_REF] Lee | Image classification with a region based approach in high spatial resolution imagery[END_REF] created a region-based classification method for high resolution images that exploited two approches: MaxVer with region means and MaxVer with Gaussian Probability Density Function.

Both works presented better results than pixel-based classifiers. Yu et al. [START_REF] Yu | Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery[END_REF] also proposed a method to classify RSIs based on regions. The image segmentation and classification were performed by using fractal networks and non-parametric K-Nearest Neighbor (KNN), respectively. Another recent work in this research area was developed by Katartzis et al. [START_REF] Katartzis | A hierarchical markovian model for multiscale region-based classification of vector-valued images[END_REF] who proposed a region-based RSI classification method that uses Hierarchical Markov Models.

The growth of classification approaches based on regions has been analyzed in [START_REF] Blaschke | Object based image analysis for remote sensing[END_REF]. According to Blaschke et al., the goal of Object-Based Image Analysis (OBIA) is to outline objects that are useful in the images, combining at the same time image processing and features of Geographic Information Systems (GIS) with the aim of using spectral and contextual information seamlessly. The article proves that the growth in the number of new approaches published accompanies the increase of the accessibility to high-resolution images and, hence, the development of alternative techniques to classification based on pixels. As pointed out by the authors, the growth in research involving OBIA was motivated in part by the use of commercial software eCognition [START_REF] Benz | Multi-resolution, object-oriented fuzzy analysis of remote sensing data for gis-ready information[END_REF]. This software has allowed research involving classification of regions, enabling the inclusion of data from different scales by using an approach supported on KNN classifier.

These new trends have encouraged research studies which compare techniques based on pixels and/or on regions [START_REF] Zhou | Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study[END_REF][START_REF] Katartzis | A hierarchical markovian model for multiscale region-based classification of vector-valued images[END_REF][START_REF] Castillejo-González | Objectand pixel-based analysis for mapping crops and their agro-environmental associated measures using quickbird imagery[END_REF][START_REF] Myint | Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery[END_REF] and propose new segmentation techniques that support the classification of regions in RSIs [START_REF] Gaetano | Hierarchical texture-based segmentation of multiresolution remote-sensing images[END_REF][START_REF] Wang | An automatic region-based image segmentation algorithm for remote sensing applications[END_REF][START_REF] Li | A novel texture-preceded segmentation algorithm for high-resolution imagery[END_REF][START_REF] Chen | Imageobject detectable in multiscale analysis on highresolution remotely sensed imagery[END_REF].

Likewise, new researches that take advantage of the use of multiple scales of data have been carried out [START_REF] Tzotsos | Object-based image analysis through nonlinear scale-space filtering[END_REF][START_REF] Ouma | Multiscale remote sensing data segmentation and post-segmentation change detection based on logical modeling: Theoretical exposition and experimental results for forestland cover change analysis[END_REF][START_REF] Wang | Object-oriented change detection method based on multi-scale and multi-feature fusion[END_REF][START_REF] Kim | Multi-scale geobia with very high spatial resolution digital aerial imagery: scale, texture and image objects[END_REF][START_REF] Tzotsos | Support vector machine classification for object-based image analysis[END_REF][START_REF] Valero | New hyperspectral data representation using binary partition tree[END_REF]. Both Ouma et al. [START_REF] Ouma | Multiscale remote sensing data segmentation and post-segmentation change detection based on logical modeling: Theoretical exposition and experimental results for forestland cover change analysis[END_REF] and Wang et al. [START_REF] Wang | Object-oriented change detection method based on multi-scale and multi-feature fusion[END_REF] proposed approaches that use multiscale data for land cover change detection. In [START_REF] Ouma | Multiscale remote sensing data segmentation and post-segmentation change detection based on logical modeling: Theoretical exposition and experimental results for forestland cover change analysis[END_REF], Ouma et al. presented a technique for multi-scale segmentation with an unsupervised neural network for vegetation analysis. Wang et al. [START_REF] Wang | Object-oriented change detection method based on multi-scale and multi-feature fusion[END_REF], on the other hand, proposed an approach for change detection in urban areas. The method relies on the fusion of features from different scales based on a combination of means for each pixel in the used scales. The result is a new image which corresponds to the combination of the scales.

Like Wang et al. [START_REF] Wang | Object-oriented change detection method based on multi-scale and multi-feature fusion[END_REF], Kim et al. [START_REF] Kim | Multi-scale geobia with very high spatial resolution digital aerial imagery: scale, texture and image objects[END_REF] used the eCognition software to create the multi-scale segmentation. The objective, however, was to perform multi-class classification. In the segmentation process, the size of the regions is controlled by a scale parameter. For each scale, a different set of classes is defined according to a hierarchy between the classes of each scale. Thus, for each level, a different classification is performed. It includes structural knowledge and high semantic contents. The result of the coarsest scales is used for the classification of the most specific classes, restricting the regions that belong to the same subtree in the hierarchy.

Valero et al. [START_REF] Valero | New hyperspectral data representation using binary partition tree[END_REF] proposed a region-based hierarchical representation for hyperspectral images based on Binary Partition Tree (BPT). They show that the proposed Pruning BPT method can be suitable for classification. Furthermore, they mention that by using different prunings based on the same idea it can be also used for filtering and segmentation purposes. Tzotsos et al. [START_REF] Tzotsos | Object-based image analysis through nonlinear scale-space filtering[END_REF][START_REF] Tzotsos | Support vector machine classification for object-based image analysis[END_REF] used multiple scales for RSIs classification. In [START_REF] Tzotsos | Support vector machine classification for object-based image analysis[END_REF], they proposed a classification based on SVM with Gaussian Kernel that uses multi-scale segmentation. One single segmentation result is used for the extraction of objects by combining segments of various sizes. The size of the selected objects is controlled by a scale parameter as well. In [START_REF] Tzotsos | Object-based image analysis through nonlinear scale-space filtering[END_REF], the authors proposed a method for the fusion of scales by nonlinear scale-space filtering. This technique avoids the use of parameters to control the creation of objects selected for classification.

The method we propose differs from the other studies in several aspects. First of all, if we consider that there is an ideal scale to represent the objects, we consider the cases in which it is not known and, hence, it can not be defined by empirical parameters. Moreover, even if the optimal scale is known, we can not assure that the use of auxiliary scales does not improve the classification accuracy. Another aspect is that our approach does not propose the fusion of features, but the combination of the classification results at different scales. Finally, our proposal uses different scales to classify the image by assigning the same set of classes at all scales, producing a single final result, i.e, a single model for all classification problems.

Our work differs from others that use a set of classes for each scale and consider semantic information to produce a classification result for each scale.

III. IMAGE REPRESENTATION AND DESCRIPTION

In this section, we describe some concepts that are used in the remainder of this work.

A. Hierarchical Segmentation

As mentioned in Section II, new methods of multi-scale segmentation have been recently proposed for remote sensing purposes [START_REF] Gaetano | Hierarchical texture-based segmentation of multiresolution remote-sensing images[END_REF][START_REF] Wang | An automatic region-based image segmentation algorithm for remote sensing applications[END_REF][START_REF] Li | A novel texture-preceded segmentation algorithm for high-resolution imagery[END_REF][START_REF] Chen | Imageobject detectable in multiscale analysis on highresolution remotely sensed imagery[END_REF][START_REF] Tzotsos | A hybrid texture-based and region-based multi-scale image segmentation algorithm[END_REF][START_REF] Li | An efficient multiscale srmmhr (statistical region merging and minimum heterogeneity rule) segmentation method for high-resolution remote sensing imagery[END_REF].

One of the most powerful segmentation method is the scale-set representation introduced by Guigues et al. [START_REF] Guigues | Scale-sets image analysis[END_REF], which builds a hierarchy of regions or a single suite of partitions. As the optimal partitioning of an image depends on the application, this method proposes to keep all partitions obtained at all scales, from the pixel level until the complete image.

Guigues algorithm is, to the best of our knowledge, the segmentation method, which fits at best the regions of the scene, and is especially efficient for aerial and remote sensing images. Among other applications, this method has been successfully used in tasks of multi-scale segmentation of remote sensing images by Trias-Sanz et al. [START_REF] Trias-Sanz | Using colour, texture, and hierarchial segmentation for high-resolution remote sensing[END_REF]. They justify the use of Guigues' algorithm by the fact that it makes both the segmentation criterion and the scale parameter explicit. We concisely introduce the algorithm below.

Let image I be defined over a domain D, a partition P is a division of D into separate regions p i . A partition P 2 is finer than a partition P 1 if each region R of P 2 is included in one and only one region of P 1 . The scale-set representation consists in defining a set of nested partitions P λ of D, indexed by a scale parameter λ, such that if λ 1 ≤ λ 2 then P 2 is finer than P 1 . The transition between P i and P i+1 is obtained by merging some adjacent regions of P i into larger regions by optimizing a criterion. The criterion we use corresponds to Mumford-Shah energy [START_REF] Mumford | Optimal approximations by piecewise smooth functions and associated variational problems[END_REF], which approximates the color image by a piecewise constant model, while minimizing the edge lengths:

E(P ) = R i ∈P E D (R i ) + λE C (R i ) (1)
where E D is the distance with the piecewise constant model and E C is the length of the contour.

The compromise between both constraints is defined by the parameter λ. For small values of λ, the image is over-segmented, the approximation of each region by a constant is perfect, but the total length of all edges is very large. On the contrary, when λ is large, the partition contains few regions (until only one), then the approximation of each region by a constant is poor, but the total length of all edges is very small. The set of partitions has a structure of a hierarchy H of regions: two elements of H that are not disjoint are nested. A partition P λ is composed by the set of regions obtained from a cut in the hierarchy H at scale λ (see Figure 1). Guigues et al. showed that this algorithm can be performed with the worst case complexity in O(N 2 logN ), where N is the size of the initial over-segmentation.

Figure 1 shows the segmentation structure obtained by the Guigues' algorithm. The hierarchy of regions is drawn as a tree and the vertical axis is the a scale axis (in logarithmic representation). A cut in scale λ retrieves a partition P λ .

To automatically select partitions at different scales, Guigues et. al. proposed the use of a dichotomous cutoff-based strategy, which consists of successively splitting the hierarchy of regions in to two. Each ... division is a dichotomous cut and creates a partition at the defined scale.

Let Λ be the maximum scale in hierarchy H, i.e., the one in which the image I is represented by a single region, the cut-scale λ c is defined by λ c = Λ/2 n , where n is the order of each division in the hierarchy. Figure 2 presents some cuts extracted from the hierarchy illustrated in Figure 1.

The highest scale of the hierarchy shown in Figure 1 is Λ = 1.716. Thus, the first cut is defined at the scale λ c = 0.858, the second one at the scale λ c = 0.429, and so on.

B. Image Descriptors

This paper uses the descriptor definition proposed in [START_REF] Torres | Content-Based Image Retrieval: Theory and Applications[END_REF]. According to that work, a descriptor can be characterized by two functions: feature vector extraction and similarity computation. The feature vectors encode image properties, like color, texture, and shape. Therefore, the similarity between two images is computed as a function of their feature vector distance. Note that different types of feature vectors may require different similarity functions. 11 Image Fig. 2. Some cuts of the scale-sets and the original image.

λ c = Λ/2 1 λ c = Λ/2 2 λ c = Λ/2 3 λ c = Λ/2 4 λ c = Λ/2 5 λ c = Λ/2 6 λ c = Λ/2 7 λ c = Λ/2 8 λ c = Λ/2 9 λ c = Λ/2 10 λ c = Λ/2
The multi-scale classification we propose is general and does not depend on the type of features used.

In this article, we use seven descriptors from the literature: three texture descriptors and four descriptors that encode spectral features (color). In our implementation, all these descriptors use the L 1 function to calculate distances between two feature vectors. We briefly present the function extraction of each descriptor below.

1) Global Color Histogram (GCH) [START_REF] Swain | Color indexing[END_REF]: It is one of the most commonly used descriptors, it uses an extraction algorithm which quantizes the color space in a uniform way and it scans the image computing the number of pixels belonging to each bin. The size of the feature vector depends on the quantization used. In the present work, the color space was split into 64 bins, thus, the feature vector has 64 values.

2) Color Coherence Vector (CCV) [START_REF] Pass | Comparing images using color coherence vectors[END_REF]: This descriptor, like GCH, is recurrent in the literature. It uses an extraction algorithm that classifies the image pixels as "coherent" or "incoherent" pixels. This classification takes into consideration whether the pixel belongs or not to a region with similar colors, that is, coherent regions. Two color histograms are computed after quantization: one for coherent pixels and another for incoherent ones. Both histograms are merged to compose the feature vector. In our experiments, the color space was quantized into 64 bins.

3) Color Autocorrelogram (ACC) [START_REF] Huang | Image indexing using color correlograms[END_REF]: The role of this descriptor is to map the spatial information 4) Border/Interior Pixel Classification (BIC) [START_REF] De O. Stehling | A compact and efficient image retrieval approach based on border/interior pixel classification[END_REF]: This descriptor has been successful in many applications. It has presented good results in tasks of image retrieval and classification of RSIs (e.g., [START_REF] Santos | Evaluating the potential of texture and color descriptors for remote sensing image retrieval and classification[END_REF], [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF], and [START_REF] Santos | A relevance feedback method based on genetic programming for classification of remote sensing images[END_REF]). The first step of the feature vector extraction process relies on the classification of image pixels into border or interior ones. When a pixel has the same spectral value in the quantized space as its four neighbors (the ones which are above, below, on the right, and on the left), it is classified as interior.

Otherwise, the pixel is classified as border. Two histograms are computed after the classification: one for the interior pixels and another for the border ones.

IV. MULTI-SCALE TRAINING AND CLASSIFICATION

In the following sections, we describe the basic ideas of our approach, as well as the major processing steps for multi-scale classification. In Section IV-A, we introduce the concepts and the general functioning of the proposed approach. In Section IV-B and IV-C, the two approaches that we propose for training classifiers using multi-scales are presented. Finally, in Section IV-D we describe the weak classifiers used in the proposed method.

A. Classification Principles

The aim of RSI classification is to build a classification function F (p) that returns a classification score (+1 for relevant, and -1 otherwise) for each pixel p of a RSI. Let us note that, even if the classification returns a result at a pixel level, the decision may be based on regions of different scales containing the pixel.

In order to create such classification function F (p), we first extract different features at different scales using multi-scale segmentation. This first step, detailed in the previous section, is fully automatic. After this step, we use boosting to build a linear combination of weak classifiers, each of them related to a specific scale and feature type. The training is performed using a training RSI image I where each pixel is labeled. It starts at pixel level, or better after a watershed process in order to obtain regions more reliable to compute the energy (Equation 1). Figure 3 illustrates the steps of the multi-scale training approach.

The base of hierarchy H is composed of the set of pixels from training image I, and it will be denoted P 0 . We will use several partitions P λ of hierarchy H at various scales λ. At each scale λ, a set of features is computed for each region of P λ . These features can be different according to the level, and, thus, to the size of the regions. For example, a texture feature is not appropriate for too small regions and a histogram (such as color histogram) is less accurate for large regions.

B. Multi-Scale Training

The multi-scale classifier (MSC) aims at assigning a label (+1 for relevant class, and -1 otherwise) the hierarchy. To build multi-scale classifiers, we propose a learning strategy based on boosting of weak learners. This strategy is based on AdaBoost algorithm proposed by Schapire [START_REF] Schapire | A brief introduction to boosting[END_REF], which builds a linear combination M SC(p) of T weak classifiers h t (p):

M SC(p) = sign T t=1 α t h t (p) (2)
The proposed algorithm repeatedly calls weak learners in a series of rounds1 t = 1, . . . T . Each weak learner creates a weak classifier that decreases the expected classification error of the combination. The algorithm then selects the weak classifier that most decreases the error.

The strategy consists in keeping a set of weights over the training set. These weights can be interpreted as a measure of the difficulty level to classify each training sample. At the beginning, all the pixels have the same weight, but in each round, the weights of the misclassified pixels are increased. Thus, in the next rounds the weak learners are forced to focus on harder samples. We will note W t (p) the weight of pixel p in round t, and D t,λ (R) the misclassification rate of region R in round t at scale λ given by the mean of the weights of its pixels:

D t,λ (R) = 1 |R| p∈R W t (p) (3)
Algorithm 1 presents the proposed Multi-Scale Training process. Let Y λ (R), the set of labels of regions R at scale λ, be the input dataset. We divide this set into training (Y t λ (R)) and validation sets (Y v λ (R)). In a serie of rounds t = 1, . . . T , for all scales λ, the weight of each region D t,λ (R) is computed (line 3). This piece of information is used to select the regions to be used for training the weak learners, building a subset of labeled regions Ŷt,λ (line 6). The subset Ŷt,λ is used to train the weak learners with each features F at scale λ (line 9). Each weak learner produces a weak classifier h t,(F ,λ) (line 10). The algorithm then selects the weak classifier h t that most decreases the error Err ht on the validation set Y v λ (line 12). The level of error of h t is used to compute the coefficient α t , which indicates the degree of importance of h t in the final classifier (line 13). The selected weak classifier h t and the coefficient α t are used to update weights W (t+1) (p) which can be used in the next round (line 14).

The classification error of classifier h is:

Err(h, W ) = p|h(p)Y v 0 (p)<0 W (p) (4)
where Y v 0 is the validation set (the label of each pixel in the image). The training is performed on the training set labels Y t λ corresponding to the same scale λ. The weak learners (linear SVM, for example) use the subset Ŷt,λ for training and produce a weak classifier h t,(F ,λ) .

The training/validation set labels Y 0 are the labels of pixels of image I, and training/validation sets labels Y λ with λ > 0 are defined according to the proportions of pixels belonging to one of the two classes (for example, at least 80% of one region).

The idea of building the subset Ŷ is to force the classifiers to train with the most difficult samples. The weak learner should allow the most difficult samples to be differentiated from the other ones according to their weights. Thus, the strategy of creating Ŷ is directly dependent on the configuration of the weak classifier and may contain all regions, since the classifier considers the weights of the samples.

λ i S λ i (p) = sign λ i T t=1 α t,λ i h t,λ i (p) ( 5
)
where T is the number of rounds for each boosting step.

At the end of each stage, we withdraw the easiest samples. Let W i be the weights of the pixels after training with scale λ i , we denote D i (R i+1 ) the weight of the region R i+1 ∈ P λ i+1 , which is given by:

D i (R i+1 ) = 1 |R| p∈R W i (p) (6)
The set of regions Yi+1 to be used in the training stage with scale λ i+1 is composed by the regions

R i+1 ∈ P λ i+1 with mean D i (R i+1 ) > 1 2|Y 0 |
. This means that the regions that ended a training stage with distribution equal to half the initialization value 1 |Y 0 | , are discarded from one stage to another.

D. Weak Classifiers

In this paper, we adopted two configurations of weak learners: Support Vector Machines (SVM) and Radial Basis Function (RBF).

1) SVM-based weak learner:

It is an SVM trainer based on a specific feature type F and a specific scale λ. Given the training subset labels Ŷλ , the strategy is to find the best linear hyperplane of separation between RSI regions according to their classes (coffee and non-coffee regions), trying to maximize the data separation margin. These samples are called support vectors and are found during the training. Once the support vectors and the decision coeffients (α i , i = 1, . . . , N ) are found, the SVM weak classifier can be defined as:

SV M (F ,λ) (R) = sign N i y i α i (f R • f i ) + b ( 7
)
where b is a parameter found during the training. The support vectors are the f i such that α i > 0, y i is the support vector class and f R is the feature vector of the region.

The training subset Ŷt,λ is composed by n labels from Y λ with values of D t,λ (R) larger or equal to

1 |Y 0 |
. This strategy means that only regions considered as the most difficult ones are used for the training.

For the first round of boosting, the regions to compose the subset Ŷ0,λ are randomly selected.

The weakness of the linear SVM classifier is due to our strategy of creating subsets instead of providing all regions of a partition for training. It decreases the power of the produced classifier. Moreover, in our experiments the dimension of the feature space is smaller than the number of samples, which theoretically guarantees the weakness of linear classifiers.

2) RBF-based weak learner:

The RBF approach is based on the distances provided by the used descriptors. It consists of selecting a target region that best separates the other regions between both classes based on a specific image descriptor D and a specific scale λ. The distances are normalized with the sigmoid function.

The RBF-based weak learner tests all training regions (i.e, Ŷλ = Y λ ) as targets in the classification task.

The exception are the regions that have already been used as targets.

Let R t be a target region and y its class (1 for relevant and -1 for irrelevant). The class of region R,

given by the weak classifier (R t , D, λ), is defined by:

RBF (Rt, D,λ) (R) =      y, if d(R t , R) ≤ l -y, otherwise (8)
where d(R t , R) is the distance between the target region R t and region R using descriptor D and l is a threshold value.

V. EXPERIMENTS

In this section, we present the experiments that we performed to validate our method. We have carried out experiments in order to address the following research questions:

• Is the set of used descriptors effective for object-based RSI classification task (Section V-B1)?

• Is the multi-scale classification results effective in RSI tasks (Section V-B2)?

• Are the proposed weak learners effective in the RSI classification problem (Section V-B3)?

• Can the hierarchical strategy for multi-scale improve the classification results (Section V-B4)?

• Are the proposed methods effective in the RSI classification problem when compared to a baseline (Section V-B5)?

In Section V-A, we describe the basic configuration of our experiments. In Section V-B we present the experimental results.

A. Setup 1) Dataset: The dataset used in our experiments is a composition of scenes taken by the SPOT sensor in 2005 over Monte Santo de Minas county, in the State of Minas Gerais, Brazil. This area is a traditional place of coffee cultivation, characterized by its mountainous terrain. In addition to common issues in the area of pattern recognition in remote sensing images, these factors add further problems that must be taken into account. In mountainous areas, the spectral patterns tend to be affected by the topographical differences and interference generated by the shadows. This dataset provides an ideal environment for multi-scale analysis, since the variations in topography require the cultivation of coffee in different crop sizes. Another problem is that coffee is not an annual crop. This means that, in the same area, there may be plantations of different ages, as illustrated in Figure 5. In terms of classification, we have several completely different patterns representing the same class while some of these patterns are much closer to other classes. We have used a complete mapping of the coffee areas in the dataset for training and assessing the quality of experimental results. The identification of coffee crops was done manually in the whole county by agricultural researchers. They used the original image as reference and visited the place to compose the final result.

The dimensions of the image used are 3000 × 3000 pixels with spatial resolution equals to 2.5 meters.

To facilitate the experimental protocol, we divided the dataset into a grid of 3 × 3, generating 9 subimages with dimensions equal to 1000 × 1000 pixels. In the experiments, we used 9 different sets of 1 million pixels each, to be used for training and testing. The results of the experiments described in the following sections are obtained from all combinations of the 9 subimages used (3 for testing, 3 for validation and, 3 for testing).

2) Feature Extraction: Unlike conventional images, RSI bands do not usually correspond exactly to the human visible spectrum. To apply conventional image descriptors (which generally use three color channels), it is necessary to select the most informative bands. Therefore, we used only the bands corresponding to "red", "infrared", and "green" that are fundamental to describe vegetation targets. These spectral bands are the basis for most of the vegetation indexes [START_REF] Lillesand | Remote Sensing and Image Interpretation[END_REF].

We extracted seven different features from the band composition IR-NIR-R (342) by using 4 color descriptors (ACC [START_REF] Huang | Image indexing using color correlograms[END_REF], BIC [START_REF] De O. Stehling | A compact and efficient image retrieval approach based on border/interior pixel classification[END_REF], CCV [START_REF] Pass | Comparing images using color coherence vectors[END_REF], and GCH [START_REF] Swain | Color indexing[END_REF]) and 3 texture descriptors (Unser [START_REF] Unser | Sum and difference histograms for texture classification[END_REF], SID [START_REF] Zegarra | Rotation-invariant and scale-invariant steerable pyramid decomposition for texture image retrieval[END_REF],

and QCCH [START_REF] Huang | An orientation independent texture descriptor for image retrieval[END_REF]). These descriptors were presented in Section III-B.

We considered five different scales to extract features from λ 1 (the finest one) to λ 5 (the coarsest one).

We selected the scales according to the principle of dichotomic cuts (see Section III-A). Figure 6 illustrates the multi-scale segmentation for one of the subimages. At λ 5 scale, subimages contain between 200 and 400 regions while, at scale λ 1 , they contain between 9, 000 and 12, 000 regions.

λ 0 (original RSI) λ 1 λ 2 λ 3 λ 4 λ 5
Fig. 6. One of the tested subimages and the results of segmentation in each of the selected scales.

3) Assessment of results:

To analyze the results, we computed the overall accuracy and Kappa index for the classified images. In our experiments, the overall accuracy is defined as the sum of true positive and true negative samples divided by the total number of samples. Kappa is an effective index to compare classified images, commonly used in RSI classification [START_REF] Congalton | Assessing the Accuracy of Remotely Sensed Data: Principles and Practices[END_REF]. Experiments in different areas show that Kappa could have various interpretations and these guidelines could be different depending on the application. However, Landis and Koch [START_REF] Landis | The measurement of observer agreement for categorical data[END_REF] characterize Kappa values over 0.80 as "almost perfect agreement", 0.60 to 0.79 as "substantial agreement", 0.40 to 0.59 as "moderate agreement", and below 0.40 as "poor agreement".

Negative Kappa means that there is no agreement between classified data and verification data.

B. Results

1) Descriptors Comparison:

The result of classification is directly related to the quality of the features extracted from the image. In this sense, the objective of this experiment is to compare descriptors in region-based classification tasks. To do so, we used MSC approach with linear Support-Vector Machines in an intermediate scale of segmentation (λ 2 ). Table I presents the overall accuracy and Kappa results for each descriptor. BIC obtained the best results among all the descriptors. BIC takes into account the spatial distribution of colors, which in a way encodes both color and texture. QCCH achieved a small highlight among the texture ones. The results present a small difference between GCH and CCV. In fact, we note that their classification results are correlated.

The great difference between the color and texture descriptors classification rates was expected. This fact is consistent with those results obtained in [START_REF] Santos | Evaluating the potential of texture and color descriptors for remote sensing image retrieval and classification[END_REF] and [START_REF] Trias-Sanz | Using colour, texture, and hierarchial segmentation for high-resolution remote sensing[END_REF]. Anyway, we believe that the combination of texture and color descriptors can improve the results.

2) Multi-Scale × Individual Scale: In this section, we compare the classification results obtained by using individual scales against the combination of scales by using the MSC approach presented in Section IV-B with 10 rounds. In this experiments, we used all descriptors referenced in Section V-A2. According to the results, one can observe that the combination of scales ( 5 i=1 λ i ) is slightly better than the best individual scale (λ 4 ). We can conclude that the proposed method MSC not only found the best scale but also could improve the result by adding other less significant scales.

Concerning time, it takes the combination longer to train when compared to scale λ 2 , but not longer than scale λ 1 alone. The same effect can be observed for the classification time.

3) Weak Classifiers Comparison (Linear SVM × RBF): In this section we compare the weak learners presented in Section IV-D. We performed experiments with 10 rounds for SVM-based and 50 rounds for RBF-based weak learners. This is the amount of rounds which normally stabilizes the results using each of the weak learners. In other words, after 10 rounds for SVM and 50 rounds for RBF, the selected weak learner typically gets very small weights and does not interfere in the final classification. Table IV presents the classification results. Although producing almost the same accuracy rates, the main difference in this examples is that HMSC produces less false positives than MSC (HMSC produces also more false negatives). We assume that the HMSC is more efficient to recognize coffee crops.

We observed that most of the classification errors are related to confusion caused by recently planted coffee crops. These regions usually appear in light blue in the composition of colors displayed (see Figure 7). 5) Comparison to the baseline: Although very used in image classification [START_REF] Mountrakis | Support vector machines in remote sensing: A review[END_REF], SVMs are so far less used in remote sensing community than other classifiers (e.g., decision trees and variants of neural networks). However, in recent years there has been a significant increase in SVM works achieving very good results in remote sensing problems. Tzotsos et al. [START_REF] Tzotsos | Support vector machine classification for object-based image analysis[END_REF] have proposed and evaluated SVMs for objectoriented classification. They proposed an approach that uses SVMs with a Gaussian kernel to classify the regions obtained by a multi-scale segmentation process. This approach outperforms the results with the eCognition software [START_REF] Benz | Multi-resolution, object-oriented fuzzy analysis of remote sensing data for gis-ready information[END_REF]. Therefore, we used SVM with Gaussian kernel applied to an intermediate As it can be noticed, both MSC and HMSC overcome the results of the baseline. This shows that the combination of descriptors and scales using the strategies proposed in this work can be powerful tools for classification of remote sensing images.

VI. CONCLUSIONS

We can draw the following conclusions from our studies: region classification is a good alternative to pixel classification; the descriptors computed on regions are more reliable than those computed on pixels or on regular blocks; the most relevant regions in different parts and scales of the image and features at various scales can be learned during the design of the classification machine.

The proposed approaches for multiscale image analysis are the Multi-Scale Classifier (MSC) and the Hierarchical Multi-Scale Classifier (HMSC). The MSC is a boosting-based classifier that builds a strong classifier from a set of weak ones. The HMSC is also based on boosting of weak classifiers, but it adopts a sequential strategy of training, according to the hierarchy of scales (from the coarsest to the finest). In this work, we adopted two configurations of weak learners: SVM and RBF. The SVM approach is based on the SVM classifier with linear kernel. The other one is based on the distances provided by Radial Basis Function. The experimental results shows that the BIC descriptor is presently the most powerful descriptor to detect regions of coffee. The MSC results shows that the combination of scales increase the power of the final classifier.

Our perspectives are firstly to build a very fast classifier by using a cascade of classifiers [START_REF] Viola | Rapid object detection using a boosted cascade of simple features[END_REF] and secondly to improve the learning set through user interaction.

Fig. 1 .

 1 Fig. 1. A scale-sets image representation. Horizontal axis: the regions. Vertical axis: the scales (logarithmic representation).

  of colors by pixel correlations at different distances. It computes the probability of finding in the image two pixels with color C at distance d from each other. For each distance d, probability m are computed, where m represents the number of colors in the quantized space. The implemented version quantized the color space into 64 bins and considered 4 distance values (1, 3, 5, and 7).

  Both histograms are merged to compose the feature vector. The implemented version quantized the color space in to 64 bins. Although the original version uses dlog function distance, the used version was adapted to use L1 distance.5) Invariant Steerable Pyramid Decomposition (SID)[START_REF] Zegarra | Rotation-invariant and scale-invariant steerable pyramid decomposition for texture image retrieval[END_REF]: In this descriptor, a set of filters sensitive to different scales and orientations processes the image. The image is first decomposed into two sub-bands using a high-pass and low-pass filter. After that, the low-pass sub-band is decomposed recursively into K sub-bands by band-pass filters and into one sub-band by a low-pass filter. Various directional information about each scale is captured by each recursive step. The mean and standard deviation of each sub-band are used as feature values. To obtain the invariance to scale and orientation, circular shifts in the feature vector are applied. The implemented version uses 2 scales and 4 orientations, which gives a feature vector with 16 values.6) Unser[START_REF] Unser | Sum and difference histograms for texture classification[END_REF]: This descriptor is based on co-occurrence matrices, still one of the most widely used descriptors to encode texture in remote sensing applications. Its extraction algorithm computes a histogram of sums H sum and a histogram of differences H dif . The histogram of sums is incremented considering the sum, while the histogram of differences is incremented by taking into account the difference between the values of two neighbor pixels. As well as gray level co-occurrence matrices, measures such as energy, contrast, and entropy can be extracted from the histograms. In our experiments, 256 gray levels and 4 angles were used (0 • , 45 • , 90 • , and 135 • ). The feature vector is composed of 32 values and eight different measures were extracted from histograms. 7) Quantized Compound Change Histogram (QCCH)[START_REF] Huang | An orientation independent texture descriptor for image retrieval[END_REF]: It uses the relation between pixels and their neighbors to encode texture information. This descriptor generates a representation invariant to rotation and translation. Its extraction algorithm scans the image with a square window. For each position in the image, the average gray value of the window is computed. Four variation rates are then computed by taking into consideration the average gray values in four directions: horizontal, vertical, diagonal, and anti-diagonal directions. The average of these four variations is calculated for each window position, they are grouped into 40 bins and a histogram of these values is computed.

Fig. 3 .

 3 Fig. 3. Steps of the multi-scale training approach. At the begining, several partitions P λ of hierarchy H at various scales λ are selected. Then, at each scale λ, a set of features is computed for each region R ∈ P λ . Finally, a classifier F (p) is built by using the Multi-Scale Training (Section IV-B) or the Hierarchical Multi-Scale Training (Section IV-C).

Fig. 4 .

 4 Fig. 4. The Hierarchical Multi-Scale training strategy.

Fig. 5 .

 5 Fig.5. Example of coffee and non-coffee samples in the used RSI. Note the difference among the samples of coffee and their similarities with non-coffee samples[START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF].

Figure 7 (

 7 Figure 7 (a) shows a subimage used in these experiments and Figure 7 (b) illustrates the same image with coffee crops, which are the regions of interest in focus. Figures 8 (a) and (b) illustrate an example of results obtained with both methods HMSC and MSC.

Fig. 7 .Fig. 8 .

 78 Fig. 7. The image used for classification in Figure 8 (a) and the same image with coffee crops highlighted (b).

  Select weak classifier h t , the one with minimum error Err * = argmin h t,(F ,λ) Err(h t,(F ,λ) , W t,λ ) In this case, both the selection of scales and features, and the weights of each weak classifier are obtained by a strategy based on AdaBoost. Although this approach provides the selection of the most appropriate scales to the training set, it does not ensure the representation of all scales in the final result. In addition, the cost of training with each scale is proportional to the number of regions it contains. However, the coarse scales are not always selected, which means that training time can be reduced if we avoid this analysis. At the end of each stage, only the most difficult samples are selected, limiting the training set used in the next stage. In each stage, the process is similar to the one described in Algorithm 1. However, the weak learners are trained with only the features related to the current scale. For each scale, the weak learner produces a set H λ of weak classifiers.

		Algorithm 1 Multi-Scale Training
		Given:
		Y λ (R) = labels of regions R at scale λ
		(Y λ = Y t λ ∪ Y t λ , where Y t λ is the training set and Y v λ is the validation set)
		Initialize: For all pixels p, W 1 (p) ← 1 |Y 0 |	, where |Y 0 | is the number of pixels in the image level
			T
	1 For t ← 1 to T do 2 For all scales λ do 3 Training t=1 t t h (p) For all R ∈ P λ do 4 Compute D t,λ (R) 5 End for 6 Build Ŷt,λ ⊂ Y t k Adaboost Image hard samples λ (a training subset based on D t,λ (R)) t=1 T t t h (p) Adaboost hard samples ... 7 End for easy samples k-1 F 1 F n ... F 1 F n ...	T t=1 Adaboost h (p) t t 2 F 1 ... F n	hard samples	T t=1 Adaboost t t h (p) 1 F 1 F n ...
		(Equation 4)		easy samples		easy samples
	11 12	End for	scale k	scale k-1	...	scale 2		scale 1
	13	Compute α t ← 1 2 ln 1+rt 1-rt

8

For each pair feature/scale (F, λ) do 9 Train weak learners using features (F, λ) and training set Ŷt,λ . 10 Evaluate resulting classifier h t,(F ,λ) on the validation set Y v λ : compute Err(h t,(F ,λ) , W t,λ ))

with r t ← p cY 0 (p)h t (p) 14 Update W t+1 (p) ← W t (p) exp (-α t Y 0 (p)h t (p)) p W t (p) exp (-α t Y 0 (p)h t (p))

15 End for Output: Multi-Scale Classifier M SC(p) (Equation

2

)

C. Hierarchical Training

The Multi-Scale Training presented in Section IV-B creates a classifier based on linear combination of weak classifiers.

In order to overcome these problems, we propose a hierarchical multi-scale classification scheme. The proposed strategy is presented in Figure 4. It consists of individually selecting the weak classifiers for each scale, starting from the coarsest one to the finer. Thereby, each scale provides a different stage of training.

TABLE I CLASSIFICATION

 I RESULTS FOR THE USED DESCRIPTORS AT λ2 SCALE.

		Descriptor Overall Acc. (%)	Kappa (κ)
		ACC	78.60 ±1.88	0.7238 ±0.029
	Color	BIC	79.92 ±2.04	0.7447 ±0.033
		CCV	77.38 ±2.72	0.7011 ±0.046
		GCH	77.64 ±2.71	0.7056 ±0.045
		QCCH	69.94 ±4.21	0.5503 ±0.086
	Texture	U N SER	68.72 ±3.67	0.5255 ±0.078
		SID	68.63 ±3.76	0.5215 ±0.078

Table

  II presents the classification results.Table III presents the time spent for training and testing. The experiments were carried out on a 2.40GHz Quad Core Xeon with 32 GB RAM.

TABLE II CLASSIFICATION

 II RESULTS USING INDIVIDUAL SCALES AND THE COMBINATION.

	Scale Overall Acc. (%)	Kappa (κ)
	λ 1	79.07 ±1.60	0.7298 ±0.028
	λ 2	79.90 ±2.04	0.7441 ±0.033
	λ 3	80.43 ±2.11	0.7519 ±0.033
	λ 4	81.04 ±1.70	0.7625 ±0.026
	λ 5	80.31 ±1.23	0.7494 ±0.020
	5 i=1 λ i	82.28 ±1.60	0.7800 ±0.025

TABLE III TIME

 III SPENT ON CLASSIFICATION USING INDIVIDUAL SCALES AND THE COMBINATION.

	Scale Training Time (s) Classification Time (s)
	λ 1	44454.54	103.98
	λ 2	9163.32	36.99
	λ 3	1272.69	14.59
	λ 4	349.27	8.56
	λ 5	84.85	6.25
	5 i=1 λ i	24939.34	38.52

  Table V presents training/classification times.

TABLE IV CLASSIFICATION

 IV RESULTS COMPARING THE MSC APPROACH USING RBF AND SVM-BASED WEAK LEARNERS.

	Weak Learners Overall Acc. (%)	Kappa (κ)
	RBF	77.78 ±3.68	0.6957 ±0.082
	Linear SV M	82.28 ±1.60	0.7800 ±0.025

TABLE V TIME

 V SPENT ON CLASSIFICATION USING THE MSC APPROACH WITH RBF AND SVM-BASED WEAK LEARNERS. to point out that, in these experiments, the distances between regions using the descriptors are computed during the classification stage. If distances are previosly computed, RBF-based weak learners are an alternative since they can be easily implemented.4) Hierarchical MS-Classification:In this section we present the results of the proposed Hierarchical Multi-Scale Classification approach. Table VI presents the overall accuracy and Kappa index for HMSC and MSC approach. Time is presented in TableVII. We used 10 rounds for MSC and 50 rounds for HMSC (10 rounds for each scale). To maintain the detection time of the classifier HMSC equivalent to the MSC, the weak learners with very low weights are excluded from the final classifier: the threshold on the weights is 0.01. This reduces the final classifier to a combination between 10 and 15 weak learners.

	Weak Learners Training Time (s) Classification Time (s)
	RBF	31030.987	327.01
	Linear SV M	24939.34	38.52

We can observe that MSC with SVM-based weak learners produces better results than with RBF-based.

Moreover, the RBF-based weak learner spends more in both training and testing stages. However, it is necessary

TABLE VI CLASSIFICATION

 VI RESULTS COMPARING THE HMSC AGAINST MSC.

	Method Overall Acc. (%)	Kappa (κ)
	HM SC	82.69 ±1.68	0.7875 ±0.024
	M SC	82.28 ±1.60	0.7800 ±0.025

TABLE VII TIME

 VII SPENT ON CLASSIFICATION FOR MSC AND HMSC. Both methods produced similar values of accuracy. The most important point concerns the training time. As the hierarchical approach does not use all regions of all scales, training time is considerably reduced because the training focuses only on the most difficult regions.

	Method Training Time (s) Classification Time (s)
	HM SC	13637.62	39.06
	M SC	24939.34	38.52

TABLE VIII ACCURACY

 VIII ANALYSIS OF CLASSIFICATION FOR THE EXAMPLE PRESENTED IN FIGURE 8 (TP = TRUE POSITIVE, TN = TRUE NEGATIVE, FP = FALSE POSITIVE, FN = FALSE NEGATIVE). ,493 64,228 70,901 HM SC 167,293 705,196 29,525 97,986 segmentation scale obtained by Guigues method as baseline with BIC descriptor. As the baseline was not designed to use the validation set, we performed these experiments with two settings: 3 subimagens for training and 3 for classification; 6 subimagens for training and 3 for classification.Table IX displays the results.

	Method	TP	TN	FP	FN
	M SC	194,378 670

TABLE IX CLASSIFICATION

 IX RESULTS COMPARING THE MSC, HMSC AND THE BASELINES. SV M + Gaussian Kernel (3,3) IS THE BASELINE TRAINED WITH 3 SUBIMAGES. SV M + Gaussian Kernel (6,3) IS THE SAME BASELINE TRAINED WITH 6 SUBIMAGES.

	Method	Overall Acc. (%)	Kappa (κ)
	SV M + Gaussian Kernel (3,3)	77.47 ±2.64	0.7054 ±0.044
	SV M + Gaussian Kernel (6,3)	80.09 ±1.58	0.7478 ±0.025
	M SC (linear SV M learner)	82.28 ±1.60	0.7800 ±0.025
	HM SC (linear SV M learner)	82.69 ±1.68	0.7875 ±0.024

J. A. dos Santos, R. da S. Torres and A. X. Falcão are with the Institute of Computing, University of Campinas, Brazil email: jsantos@ic.unicamp.br J. A. dos Santos, P. Gosselin and S.Philipp-Foliguet are with ETIS, CNRS, ENSEA, University of Cergy-Pontoise, France

Despite the term "iterations" be more common, we use the term "rounds" that is typically applied to refer to the main loop present in boosting-based methods.

ACKNOWLEDGMENTS

The authors are grateful to CAPES, FAPESP, and CNPq for the financial support. We are grateful to Jean Pierre Cocquerez for the support concerning the segmentation tool. We also thank Rubens Lamparelli due to the support related to agricultural aspects and the remote sensing dataset.


J A Dos Santos

F A Faria

R Da

S Torres

A Rocha

P-H Gosselin

S Philipp-Foliguet

A Falcão

Descriptor Correlation Analysis for Remote Sensing Image Multi-Scale Classification

This paper addresses the problem of remote sensing image multi-scale classification by: (i) showing that using multiple scales does improve classification results, but not all scales have the same importance; (ii) showing that image descriptors do not offer the same contribution at all scales, as commonly thought, and some of them are very correlated; (iii) introducing a simple approach to automatically select segmentation scales, descriptors, and classifiers based on correlation and accuracy analysis.

Introduction

Region-based solutions for remote sensing image (RSI) classification normally rely on the use of segmentation techniques and robust pattern recognition methods [START_REF] Blaschke | Object based image analysis for remote sensing[END_REF]. Naturally, both segmentation and machine learning have unsolved problems on their own and often we need to cope with some of them when dealing with a specific scenario. For instance, in RSI, some researchers have addressed these problems in two ways: by exploiting regions with multiple sizes to improve segmentation [START_REF] Tzotsos | Objectbased image analysis through nonlinear scale-space filtering[END_REF], or by combining different features and classifiers [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF][START_REF] Santos | A relevance feedback method based on genetic programming for classification of remote sensing images[END_REF] to improve pattern recognition. In this sense, we proposed the Multi-Scale Classification (MSC) approach to combine different features at multiple segmentation scales [START_REF] Santos | Multiscale classification of remote sensing images[END_REF]. The strategy is based on the boosting paradigm, whose principle is to combine weak classifiers to build an effective and efficient global one. MSC exploits a hierarchy of regions obtained by using the Guigues algorithm [START_REF] Guigues | Scale-sets image analysis[END_REF].

In reviewing RSI works that deal with combination of segmentation scales, features and classifiers, many questions arise related to how these combinations are done: (1) the ideal combination; [START_REF] Brown | Good" and "Bad" Diversity in Majority Vote Ensembles[END_REF] an intelligent way to combine and save time; [START_REF] Santos | A genetic programming approach for coffee crop recognition[END_REF] the best number of segmentation scales; and (4) the best number of descriptors. These are only some examples of questions and several others are possible. To the best of our acknowledge, there is no formal analysis concerning these issues and normally what we see is a compilation of several adhoc methods. In [START_REF] Santos | Multiscale classification of remote sensing images[END_REF], we show that the combination of features at different scales improves the classification results, but these results still lack more explanation about how to select the best scales and descriptors.

In this context, the objective of this paper is to address such questions. First, we show that although using multiple scales does improve the results, not all scales have the same importance. Fusion of classifiers is also important, but not all combinations yield good results. Finally, in face of the observations made, we present a simple approach to automatically select segmentation scales, descriptors, and classifiers based on the correlation and the accuracy analysis, and all this need no a priori knowledge, no learning, everything is performed on line, during the system use.

Experimental Setup

To support our study, we have carried out experiments with the following components: RSI Data: we used a high resolution SPOT scene (3, 000 × 3, 000 pixels) which corresponds to the Monte Santo de Minas county, in the State of Minas Gerais, Brazil. It is a traditional place of coffee cultivation. The distortions in the relief and specific aspects of coffee crops increase the challenge of classifying this dataset. To evaluate the accuracy, we use a ground truth that indicates all coffee crops in the image. We divided the image into a 3 × 3 grid, generating nine 1, 000 × 1, 000 subimages. We carried out experiments with ten different combinations of the nine subimages used (three for training, three for validation, and three for testing). More details about the dataset can be found in [START_REF] Santos | Multiscale classification of remote sensing images[END_REF]. Segmentation Scales: we consider five different scales to extract features: λ 1 , . . . , λ 5 . We define the scales according to the principle of dichotomic cuts proposed in [START_REF] Guigues | Scale-sets image analysis[END_REF]. The higher the index, the coarser is the scale. The scale λ 0 is the pixel level.

Features: we extracted different features from the band composition IR-NIR-R (342) by using four color and three texture descriptors. The color descriptors are: Global Color Histogram (GCH), Color Coherence Vector (CCV), Color Autocorrelogram (ACC), and Border/Interior Pixel Classification (BIC). The texture descriptors are: Invariant Steerable Pyramid Decomposition (SID), Unser, and Quantized Compound Change Histogram (QCCH). These descriptors were selected based on previous results as reported in [START_REF] Santos | Evaluating the potential of texture and color descriptors for remote sensing image retrieval and classification[END_REF]. Classifiers: we use support vector machines (SVMs) with no kernels for each descriptor at scale λ i . In the experiments with the MSC, we used "weakened" SVMs as weak learners. More details about the implementation of SVMs as weak learners can be found in [START_REF] Santos | Multiscale classification of remote sensing images[END_REF]. Effectiveness Measure: We analyze the results by computing the overall accuracy, kappa index, and tau index for the classified images [START_REF] Ma | Tau coefficients for accuracy assessment of classification of remote sensing data[END_REF].

Results and Discussion

This section presents the correlation analysis and the proposed approach for selecting classifiers.

Correlation Analysis

The first study is concerned with the analysis of the accuracy of classifiers at different segmentation scales. The second study is the correlation analysis of each pair of classifiers. In our experiments, a classifier is defined for a descriptor and a segmentation scale. We use Cor [START_REF] Kuncheva | Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy[END_REF] to assess the correlation of two classifiers c i and c j :

COR(c i , c j ) = ad -bc (a + b)(c + d)(a + c)(b + d) ( 1
)
where a is the percentage of images that both classifiers c i and c j classified correctly in the training set, b and c are the percentage of images that c j hit and c i missed and vice versa, and d is the percentage of images that both classifiers missed.

Classifier Accuracy for Different Segmentation Scales

Figure 1 shows the overall accuracy, the kappa index, and the tau index for each SVM classifier implemented using each descriptor/scale. We observe a large difference between the accuracy results (Figure 1(a)) of the classifier implemented with color and texture descriptors for almost all scales. Among the color descriptor accuracies, we have no significant difference, although BIC presents the higher values at all scales. Among the texture ones, they present almost the same accuracies at all scales except for QCCH that presents its best results at the coarser scales.

Regarding the tau indexes (Figure 1(b)), which is more discriminative than overall accuracy, we observe that BIC achieves the best results for all scales. GCH also yields the best result at the coarser scale λ 5 . Among the texture descriptors, all of them are almost random at the finest scales (λ 1 and λ 2 ). QCCH presents the best results at the intermediate scale λ 3 . The texture descriptors present their best results at the coarsest scales λ 4 and λ 5 . At the coarsest scales, QCCH and Unser present better results than SID. The main conclusion of this experiment is that color descriptors are very important at all scales while texture features can contribute only at the coarsest ones.

Classifier Correlation for Different Segmentation Scales

In this section, we analyze the correlation of each pair of classifiers at the segmentation scales.

Figure 2 shows the correlation scores considering the different descriptors and scales. We have observed that the correlation among the descriptors presents minor differences depending on the training set. We report in this paper the most common patterns observed in the experiments. Note that the correlation among the finer scales is large (scales λ 1 and λ 2 ), while the correlation among the coarser scales (λ 4 and λ 5 ) is small. As expected, the overall correlation between scales with regions of different sizes is low. This suggests that the use of different scales improves the classification of RSI according to what have been reported in the literature.

Region A is related to the low correlation among QCCH-based classifiers and classifiers created using other descriptors. Region B refers to the low correlation of ACC-based classifiers with other ones. That suggests that ACC-based classifiers are good candidates to be combined. Region C refers to the high correlation observed among the classifiers created with texture descriptors, mainly when finer scales (small regions) are considered. Finally, the region labeled with D refers to the high correlation score observed for CCV and GCH descriptors. Classifiers based on those descriptors are not good candidates to be combined.

Figure 3 presents the correlation coefficient (see Equation 1) of each pair of descriptors at the segmentation scales λ 1 , . . . , λ 5 . Note that the smaller the segmentation scale, the higher the correlation between the Finally, with this experiment we can conclude that combining descriptors improves the classification results, but some descriptors contribute more than others and that depends on the scale. Furthermore, we assume that low correlated classifiers are good candidates to be combined as proposed in [START_REF] Brown | Good" and "Bad" Diversity in Majority Vote Ensembles[END_REF].

Selection of Descriptors

As we observed that not all scales and descriptors have the same contributions and that some classifiers might be very correlated to others, we need to devise a method to select the most promising combination pair (descriptor, classifier).

The simplest idea is to select the most accurate classifiers/descriptors for combination. However, by using only the overall accuracy as the majority of works in the literature, we can have a wrong notion about the results, mainly in binary classification problems. Therefore, we design a simple strategy using two other variables to select classifiers. The first is the tau index, which can be interpreted as a measure of difference to the classification randomly obtained. We used tau because it is more discriminative than the overall accuracy. The other one is the correlation between pairs of classifiers. Correlation gives a notion of diversity that can be used to select classifiers specialized in different kinds of features or subclasses and captures the ones more appropriate to be combined.

Consider a plane where the x axis and y axis represent the tau index and the correlation of a pair of classifiers, respectively. Let C be the set of pairs of classifiers in a given scale. The position P (ci,cj ) of a pair of classifiers c i ∈ C and c j ∈ C on this plane is defined by the ordered pair P (ci,cj ) = (Cor(c i , c j ),

τc i +τc j 2
), where Cor(c i , c j ) is the correlation of classifiers c i and c j , given by Equation 1, and τ ci and τ cj are the classification effectiveness measured using the tau index for classifiers c i and c j , respectively. Both the correlation and the tau index are computed on the validation set. Figure 4 shows the distribution of pairs of classifiers considering the λ 5 scale for one of the validation sets. Similar distributions are computed for all scales. An ideal pair of classifiers should have low correlation and high tau index. Let P be the position of the ideal pair of classifiers. In our approach, P = (1.0, 0.0). The set R of selected pairs of classifiers for a given scale is defined by the K-nearest neighbours of P:

K -N N (P) = {R ⊆ C, |R| = K (2) ∧∀x ∈ R, y ∈ C -R : ρ(P, x) ρ(P, y)}
where ρ is the distance between two points. In our case, we use the Euclidean distance. We use this strategy with K = 1 to select the nearest pair of classifiers to the ideal position for each scale. Since we consider five scales, 10 classifiers are selected for combination.

We perform experiments using the MSC approach to assess the effectiveness of our selection strategy. However, any other method could be used without loss of generalization. The objective is to show that the effectiveness of MSC is the same, when it uses the small set of relevant classifiers selected by our approach. The MSC, which is based on boosting of weak classifiers, defines a weight for each selected classifier along T rounds. The "strong" final classifier is a linear combination of these weak classifiers [START_REF] Santos | Multiscale classification of remote sensing images[END_REF].

Table 1 presents the average overall accuracy (O. A.), Kappa, and Tau measures of the MSC considering the 10 selected classifiers by our strategy (M SC 10 ) and using all available classifiers (M SC 35 , five classifiers per scale). One can see that the accuracies are almost the same. The time spent for training MSC, however, are very different. M SC 10 takes around 9h, while M SC 35 takes 16h. Table 2 shows the weight computed by M SC for weak classifiers selected across the training rounds, considering all classifiers and those 10 found by our selection strategy. As it can be observed, the set of weak classifiers and their weights are almost the same for both configurations.

Conclusions

Recent works in the literature have showed that combining different scales can improve RSI classification results. In this paper, we carried out experiments that confirm this, but we showed that not all scales contribute in the same way. Coarser scales offer great power of description while the finer ones can improve the classification by detailing the segmentation. Another branch of studies showed that the use of different descriptors is important. However, the descriptors do not contribute equally at all scales. Finally, in face of the observations we presented a simple approach to choose suitable descriptors/scales for RSI classification by using the correlation measure and the tau index, achieving promising results while spending about half of the time of a fusion approach without a selection policy. Future work includes the analysis of the correlation per accuracy plane with more powerful techniques (e.g., clustering algorithms).

Figure 1 .

 1 Figure 1. Classifier effectiveness for each descriptor at various segmentation scales: Overall accuracy (on the left), and Tau index (on the right).

Figure 2 .

 2 Figure 2. Complete correlation coefficients for each descriptors at the segmentation scales λ 1 , . . . , λ 5 . descriptors. The finer scales are composed by more homogeneous and smaller regions. In such scenario, global descriptors as those used in our experiments have less visual patterns to encode. This may be one of the reasons why region-based methods have presented better results than traditional pixel-based classification in the literature when high-resolution RSIs are considered. In face of the results above, most promising combination would involve the classifiers implemented with color descriptors, at all scales. Some examples are ACC and BIC at λ 4 , and BIC and GCH at λ 5 . With regard to texture descriptors, one should consider only the created classifiers considering scales with large regions.

Figure 3 .

 3 Figure 3. Correlation of pairs of classifiers for different segmentation scales.Finally, with this experiment we can conclude that combining descriptors improves the classification results, but some descriptors contribute more than others and that depends on the scale. Furthermore, we assume that low correlated classifiers are good candidates to be combined as proposed in[START_REF] Brown | Good" and "Bad" Diversity in Majority Vote Ensembles[END_REF].

Figure 4 .

 4 Figure 4. Distribution of pairs of classifiers considering the λ 5 scale for one of the validation sets.

Table 1 . Classification results using 10 and 35 classifiers.

 1

	Comb.	O.A. (%)	Kappa (κ)	Tau (τ )
	approach
	M SC 10	82.01 ± 1.11 0.5475 ± 0.02 0.6203 ± 0.02
	M SC 35	82.28 ± 0.99 0.5587 ± 0.02 0.6321 ± 0.01

Table 2 . Weak classifiers chosen by the MSC for each round t considering 10 au- tomatically selected classifiers and all 35 classifiers.

 2

		M SC 10		M SC 35
		Classifier Weight Classifier Weight
	0	BIC,λ 3	0.73	BIC,λ 3	0.73
	1	BIC,λ 5	0.21	BIC,λ 5	0.21
	2	Unser,λ 4	0.10	Unser,λ 4	0.10
	3	Unser,λ 5	0.02	GCH,λ 4	0.10
	4	BIC,λ 5	0.16	BIC,λ 5	0.16
	5	ACC,λ 2	0.25	GCH,λ 5	0.18
	6	Unser,λ 5	0.08	ACC,λ 3	0.20
	7	ACC,λ 1	0.07	CCV,λ 2	0.15
	8	BIC,λ 1	0.21	ACC,λ 5	0.14
	9	BIC,λ 5	0.12	GCH,λ 5	0.08

Acknowledgements

We thank FAPESP (2008/58528-2 and 2010/14910-0), CAPES, and CNPq for the financial support, and Cooxupé for the dataset.