knowledge and very valuable advice throughout my PhD study. His tolerance, patient guidance, continuous encouragement, enthusiastic engagement and his never ending stream of ideas have benefited me a lot.

A big thanks to the reviewers, Prof. Yide WANG and Prof. Ahmed BOURI-DANE, for accepting to be the reviewers, Prof. Eric MOREAU and Dr. Xuefeng LIU to be the examiners of my thesis. In addition, I would like to express my deepest appreciation to Prof. Yide WANG for helping to apply for the project and Dr.

Xuefeng LIU for her continued help during my master's and Ph.D., which has a profound impact on my life. I also would like to thank my super lovely and kind friends that I have met in Marseille. Thank them for accompanying me over the past three years, so I didn't feel lonely and helpless. I am honored to know them during my PhD study, especially Chunyang Li, Jiaqi Yang, Fangfang Yang, Na Shao and Jingke Hou.

I would also like to thank the members of the GSM (Groupe Signaux Multidimensionnels) team of Institut Fresnel. Although a short time spent with them, I am still very happy to know them and cherish every opportunity to meet. Special thanks to Xiaoxi Pan for patiently answering the various questions I encountered, Julien Wojak for his technical support, and Caroline Fossati for her kind help in the office.

Special thanks to the China Scholarship Council and École Centrale Marseille, who offered me the opportunity to study and live in France.

Finally, I would like to thank my grandparents, my parents, my brother, and all my family and friends, especially Bin Liu, for their love, support and encouragement all the time. i

Résumé étendu

Une image hyperspectrale provient de l'observation d'une même scène dans de nombreuses longueurs d'onde esapcées de quelques nanomètres et contigües.

L'intérêt est d'obtenir un spectre que l'on peut assimiler comme continu. En imagerie hyperspectrale on considère que chaque matériau, présent dans la scène, reflète des ondes électromagnétiques de manière spécifique. Ainsi, Le spectre de chaque pixel diffère suivant le ou les matériaux présents et on peut par exemple discriminer deux objets de même couleur composés de matériaux différents. La représentation d'une image hyperspectrale sous forme de tenseur d'ordre trois ou d'un tableau tridimensionnel est la plus utilisée, où les dimensions impliquées sont de même nature : deux dimensions spatiales et une dimension spectrale, comme le montre la figure 1. Ceci nous permet de prendre en compte simultanément l'information spatiale et l'information spectrale.

En raison des caractéristiques des HSI, l'imagerie hyperspectrale est largement utilisée dans de nombreux domaines [1,2], tels que l'exploration minérale [3], l'agriculture [4,5], l'environnement [START_REF] Vaglio Laurin | Biodiversity mapping in a tropical west african forest with airborne hyperspectral data[END_REF]. Dans ces applications on s'intèresse plus particulièrement à la classification et la détection de cibles dans les images hyperspectrales.

Figure 1: Schéma de principe de l'imagerie hyperspectrale.

Cependant, comme les HSI contiennent généralement des centaines de bandes spectrales, la charge de calculs augmente rapidement et les techniques conventionnelles ne sont plus efficaces pour le traitement de données de grande dimension, principalement en raison de la malédiction de la dimensionnalité [START_REF] Hughes | On the mean accuracy of statistical pattern recognizers[END_REF]. La réduction de dimension spectrale ou l'extraction de caractéristiques pertinentes de l'image hyperspectrale sont des moyens efficaces de résoudre le problème de la malédiction de la dimensionnalité. L'extraction des caractéristiques est l'une des étapes les plus importantes pour la classification et la détection. Les données hyperspectrales sont des données typiques non linéaires, qui contiennent de nombreuses informations spectrales et spatiales. Les méthodes traditionnelles d'extraction de caractéristiques sont basées sur des transformations linéaires, telles que l'analyse factorielle (FA), l'analyse en composantes principales (PCA), qui sont limitées pour l'extraction de caractéristiques non linéaires et des caractéristiques à un niveau profond [START_REF] Morgan | Principal component analysis and exploratory factor analysis[END_REF].

Au cours de ces dernières années, considéré comme une branche importante de l'apprentissage automatique [START_REF] Wang | Unified video annotation via multigraph learning[END_REF][START_REF] Yang | Multiview canonical correlation analysis networks for remote sensing image recognition[END_REF][START_REF] Wang | Active learning in multimedia annotation and retrieval: A survey[END_REF][START_REF] Hu | 3d-gabor inspired multiview active learning for spectral-spatial hyperspectral image classification[END_REF], l'apprentissage en profondeur a attiré une large attention en raison de ses bonnes performances pour l'analyse de données et l'extraction de caractéristiques [START_REF] Lee | Fast computation of the compressive hyperspectral imaging by using alternating least squares methods[END_REF][START_REF] Wang | Hyperspectral image compression based on lapped transform and tucker decomposition[END_REF]. En extrayant les caractéristiques des données d'entrée du bas vers le haut du réseau, les modèles d'apprentissage en profondeur peuvent former les caractéristiques de haut niveau et non linéaires [START_REF] Yang | Learning high-level features for satellite image classification with limited labeled samples[END_REF].

Certains modèles d'apprentissage profond sont utilisés à la classification en imagerie hyperspectrale et quelques résultats sont obtenus, tels que l'autoencodeur (AE) [START_REF] Teffahi | A novel spectral-spatial classification technique for multispectral images using extended multi-attribute profiles and sparse autoencoder[END_REF], les réseaux de croyances profondes (DBN) [START_REF] Chen | Spectral-spatial classification of hyperspectral data based on deep belief network[END_REF], les réseau de neurones récurrents [START_REF] Hang | Cascaded recurrent neural networks for hyperspectral image classification[END_REF][START_REF] Zhou | Hyperspectral image classification using spectral-spatial lstms[END_REF], les réseaux de neurones résiduels (ResNet) [START_REF] Jiang | Hyperspectral image classification based on 3-d separable resnet and transfer learning[END_REF] et les réseaux de neurones convolutifs (CNN) [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF].

Objectif et contributions de ce travail de thèse

Compte tenu de l'énorme potentiel de l'apprentissage en profondeur en traitement d'images et de nombreux modèles pouvant gérer des données multidimensionnelles de manière flexible, les modèles d'apprentissage en profondeur sont principalement utilisés pour le traitement des HSI dans cette thèse. Les cibles sont débruitées, classées et détectées en extrayant entièrement les caractéristiques spectrales et spatiales et en analysant les données hyperspectrales.

Étant donné que les opérations basées sur la convolution peuvent gérer des données multidimensionnelles de manière flexible, 2D-CNN et 3D-CNN sont respectivement utilisés pour l'extraction et la classification des caractéristiques. 2D-CNN montre un grand potentiel dans la préservation de la structure spatiale des cibles. 3D-CNN peut prendre directement des données 3D en entrée, ce qui permet d'exploiter les informations spectrales et spatiales en même temps. De plus, une méthode de réglage des paramètres est proposée pour sélectionner les paramètres à leur tour selon le principe de la variable unique. Le réseau avec des paramètres optimaux nous aide à obtenir de meilleurs résultats. Cependant, qu'il soit 2D-CNN ou 3D-CNN, le réseau est optimisé en minimisant l'erreur entre la sortie et l'étiquette, ce qui signifie que CNN nécessite un grand nombre d'échantillons étiquetés pour garantir ses performances. Malheureusement, les échantillons étiquetés dans les HSI sont limités et la collecte d'échantillons étiquetés prend du temps et demande beaucoup de travail. Pour résoudre ce problème, les méthodes du transfert de l'apprentissage et de la génération d'échantillons virtuels sont introduites. Le transfert de l'apprentissage permet à un système de reconnaître et d'appliquer les connaissances et les compétences acquises dans de précédents domaines/tâches à de nouveaux domaines/tâches [START_REF] Sinno | A survey on transfer learning[END_REF]. Si on dispose d'une HSI (données sources) avec suffisamment d'échantillons étiquetés et avec les mêmes caractéristiques spatiales que la HSI à classer (données cibles), alors l'apprentissage par transfert peut être utilisé pour réduire le besoin d'échantillons étiquetés de données cibles. Si les données sources ne sont disponibles, on peut générer des échantillons virtuels à partir des échantillons originaux des données cibles pour résoudre le problème d'insuffisance d'échantillons pour l'apprentissage. Afin de mieux comparer les résultats de classification obtenus par différents modèles, les valeurs de précision globale (OA) sont principalement utilisées pour évaluer la précision de la classification. Pour une meilleure comparaison visuelle, les cartes de classification de l'ensemble des données de l'Université de Pavie obtenues par 2D-CNN, 3D-CNN et 3D-CNN avec le transfert d'apprentissage et en utilisant des échantillons virtuels (3D-CNN-TV) sont illustrées dans la figure 2 .

On peut voir à partir de la figure 2 que la carte de classification de 2D-CNN a plus de pixels mal classés par rapport aux cartes de classification obtenues avec 3D-CNN, ce qui montre que 3D-CNN a de plus grandes capacités à exploiter pleinement les informations . En outre, la méthode proposée 3D-CNN-TV permet d'obtenir la carte la plus claire et la valeur la plus élevée de OA, ce qui montre que cette méthode 3D-CNN-TV a un grand potentiel en imagerie hyperspectrale pour la classification. L'extraction non supervisée de caractéristiques qui n'implique pas d'échantillons étiquetés est un autre moyen pour résoudre le problème de manque de données étiquetées. Le réseau antagoniste génératif (GAN) est entraîné de manière antagoniste

Contributions du travail de cette thèse

Organisation de la thèse

Cette thèse est organisée en cinq chapitres.

Introduction

Hyperspectral images (HSIs) are collected from the interested scene by spaceborne and airborne hyperspectral imagers with a lot of narrow electromagnetic bands as shown in Figure 6. Based on two-dimensional (2D) space domain, a third spectral dimension is added for each pixel. Thus, an HSI can be represented as a threedimensional (3D) data block that contains not only spatial information but also spectral characteristics. Due to the characteristics of HSIs, hyperspectral imaging technology has been widely applied in many fields [1,2], such as mineral exploration [3], agriculture [4,5], environment management [START_REF] Vaglio Laurin | Biodiversity mapping in a tropical west african forest with airborne hyperspectral data[END_REF]. Hyperspectral classification and detection are two important techniques for these applications, and feature extraction is one of the most significant steps for classification and detection. Traditional feature extraction methods are unusually based on linear transformation, such as principal component analysis (PCA) [START_REF] Kang | Pcabased edge-preserving features for hyperspectral image classification[END_REF] and independent component analysis (ICA) [START_REF] Wang | Independent component analysis-based dimensionality reduction with applications in hyperspectral image analysis[END_REF], which are not suitable for nonlinear hyperspectral data. Worse, most of the traditional feature extraction methods can only extract features in a shallow manner [START_REF] José | Hyperspectral remote sensing data analysis and future challenges[END_REF]. Therefore, effective feature extraction is one of the key processings to improve HSI classification and detection [START_REF] Hang | Matrix-based discriminant subspace ensemble for hyperspectral image spatial-spectral feature fusion[END_REF][START_REF] Hang | Robust matrix discriminative analysis for feature extraction from hyperspectral images[END_REF][START_REF] Xu | Spectral-spatial unified networks for hyperspectral image classification[END_REF][START_REF] Song | Spatiotemporal satellite image fusion using deep convolutional neural networks[END_REF]. Recently, as an important branch of machine learning [START_REF] Wang | Unified video annotation via multigraph learning[END_REF][START_REF] Yang | Multiview canonical correlation analysis networks for remote sensing image recognition[END_REF][START_REF] Wang | Active learning in multimedia annotation and retrieval: A survey[END_REF][START_REF] Hu | 3d-gabor inspired multiview active learning for spectral-spatial hyperspectral image classification[END_REF], deep learning has attracted wide attention due to its strong capabilities in data analysis and feature extraction [START_REF] Lee | Fast computation of the compressive hyperspectral imaging by using alternating least squares methods[END_REF][START_REF] Wang | Hyperspectral image compression based on lapped transform and tucker decomposition[END_REF]. By extracting features of the input data from the bottom to the top of the network, deep learning models can form the high-level abstract features [START_REF] Yang | Learning high-level features for satellite image classification with limited labeled samples[END_REF]. Some deep learning models have been successfully applied to HSIs, such as autoencoder (AE) [START_REF] Teffahi | A novel spectral-spatial classification technique for multispectral images using extended multi-attribute profiles and sparse autoencoder[END_REF], deep belief network (DBN) [START_REF] Chen | Spectral-spatial classification of hyperspectral data based on deep belief network[END_REF], recurrent neural network (RNN) [START_REF] Hang | Cascaded recurrent neural networks for hyperspectral image classification[END_REF][START_REF] Zhou | Hyperspectral image classification using spectral-spatial lstms[END_REF], convolutional neural network (CNN) [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF][START_REF] Li | Deep learning for hyperspectral image classification: An overview[END_REF][START_REF] Paoletti | A new deep convolutional neural network for fast hyperspectral image classification[END_REF]. Among numerous deep learning models, CNN has attracted widespread attention due to its unique network structure and strong feature extraction capabilities [START_REF] Cvetković | Multi-channel descriptors and ensemble of extreme learning machines for classification of remote sensing images[END_REF].

HSIs are 3D tensor data and multidimensional data can be directly input into CNNs, which helps to preserve the original relevant information of the data and avoids complex data reconstruction [START_REF] Zhao | An efficient macroblock-based diverse and flexible prediction modes selection for hyperspectral images coding[END_REF][START_REF] Vakil | A robust multi-stage informationtheoretic approach for registration of partially overlapped hyperspectral aerial imagery and evaluation in the presence of system noise[END_REF][START_REF] Huang | Transfer learning with deep convolutional neural network for sar target classification with limited labeled data[END_REF]. Therefore, CNN has been introduced

to extract high-level invariant features and improve the classification performance of HSIs [START_REF] Chen | Deep learning-based classification of hyperspectral data[END_REF][START_REF] Mei | Hyperspectral image spatial super-resolution via 3d full convolutional neural network[END_REF]. However, since the HSI usually contains hundreds of bands, the number of corresponding network parameters and the amount of calculation increase.

Besides, sufficient training samples are needed to guarantee the performance of CNN.

Unfortunately, labeled samples in HSIs are always limited [START_REF] Liu | p-laplacian regularized sparse coding for human activity recognition[END_REF][START_REF] Liu | Manifold regularized kernel logistic regression for web image annotation[END_REF][START_REF] Yu | Sar target recognition via local sparse representation of multi-manifold regularized lowrank approximation[END_REF]. Therefore, high dimensionality and insufficient labeled samples are two challenges in HSI processing. Dimensionality reduction can effectively reduce the computational difficulty.

Some representative methods, for example transfer learning [START_REF] Yang | Learning and transferring deep joint spectral-spatial features for hyperspectral classification[END_REF][START_REF] Casale | Transfer learning in body sensor networks using ensembles of randomized trees[END_REF][START_REF] Lin | Structure preserving transfer learning for unsupervised hyperspectral image classification[END_REF], manifold regularization based on semi-supervised leaning [START_REF] Liu | p-laplacian regularized sparse coding for human activity recognition[END_REF][START_REF] Yu | Sar target recognition via local sparse representation of multi-manifold regularized lowrank approximation[END_REF], and so on, have been studied to alleviate the problem of limited samples. The former method is suited for high-dimensional data structures, the latter being more suitable for ordinary images.

Therefore, transfer learning is introduced and applied to HSIs.

Unsupervised feature extraction is another good way to get rid of limited labeled samples. In recent years, some deep learning models have been investigated for unsupervised feature extraction. Generative adversarial network (GAN) is trained in an adversarial way requiring no labeled samples. It has been one of the most promising unsupervised learning representatives [START_REF] Radford | Unsupervised representation learning with deep convolutional generative adversarial networks[END_REF]. In [START_REF] Zhan | Semisupervised hyperspectral image classification based on generative adversarial networks[END_REF], a semi-supervised framework based on GAN is established for hyperspectral classification with a small number of labeled samples. But only spectral features are extracted, which are far from enough for classification. In [START_REF] Zhang | Unsupervised feature extraction in hyperspectral images based on wasserstein generative adversarial network[END_REF], Zhang proposed a novel modified GAN whose generator and discriminator are designed in the form of fully deconvolutional network and fully convolutional network to extract the features without supervision.

Nevertheless, only spatial information is taken as input when the modified GAN is trained, which can be treated as 2D convolution on multiple channels. Hyperspec-tral data is a tensor data, which contains not only spatial information but also the spectral characteristics of the target. Fully mining the spectral-spatial features in HSIs is helpful for classifying the target. Considering 3D convolution operation is performed in space and spectrum, we want to design a framework based on threedimensional generative adversarial network (3D-GAN) in which the generator and discriminator are built on fully 3D convolution and 3D deconvolution subnetworks to fully extract the spectral-spatial features with unsupervised learning for classification. In addition, the AE learns a representation for input data through an encoder and then decodes the representation to reconstruct data [START_REF] Vincent | Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion[END_REF][START_REF] Protopapadakis | Stacked autoencoders driven by semi-supervised learning for building extraction from near infrared remote sensing imagery[END_REF]. The AE can be optimized by minimizing the error between the reconstructed data and the input data, and no labels are involved, which is a typical unsupervised model. Because of these characteristics of AE, some unsupervised feature extraction methods that are based on AE have been introduced in HSIs and achieved some results [START_REF] Mei | Unsupervised spatial-spectral feature learning by 3d convolutional autoencoder for hyperspectral classification[END_REF]. In [START_REF] Tao | Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification[END_REF],

two variants based on stacked sparse AE are designed to the unsupervised spectral features learning and multiscale spatial features learning, respectively. The learned spectral and spatial features are stacked as a long feature vector embedded into a classifier for classification, which is more potential and robust in hyperspectral classification compared to traditional methods. However, the spectral features and spatial features are extracted separately. In [START_REF] Zhang | Recursive autoencoders-based unsupervised feature learning for hyperspectral image classification[END_REF], an unsupervised feature extraction method based on recursive AE is developed to produce high-level features. Some results have been obtained based on variants of AE, but often single-level features are considered, which affects feature performance. Therefore, we want to use AE-based models to further explore multi-scale features in HSIs.

In addition, target detection is also an important task for us. Each pixel in HSIs corresponds to a spectral curve and the spectral signature of the same category is similar, which enables identify the materials present in the pixel. Therefore, target detection can be treated as a binary classification task [START_REF] Liu | Reduction of signaldependent noise from hyperspectral images for target detection[END_REF]. With a known spectral signature can also be called a spectral template, comparing the spectral template to the pixels in a scene can determine whether target is present or not.

There are some detectors which are commonly used for target detection in HSIs [START_REF] Renard | Improvement of target detection methods by multiway filtering[END_REF],

such as adaptive coherence/cosine estimator (ACE), adaptive matched filter, and spectral angle mapper. However, HSIs always suffer from spectral variations caused by noise or environment, which enlarges within-class variation and degrades the performance of detectors. It is essential to obtain high detection accuracy even targets in noisy scenes. Thus, we want to improve the target detection results using existing detectors by improving the quality of spectral signature and mining the invariant features of the spectrum. To improve target detection results, denoising usually be done as a preprocessing step for noise removal, and then target detection is performed. Traditional denoising methods, such as PCA [START_REF] Chen | Denoising and dimensionality reduction of hyperspectral imagery using wavelet packets, neighbour shrinking and principal component analysis[END_REF] and models based on Wiener filter (WF) [START_REF] Zha | Noise reduction in interferograms using the wavelet packet transform and wiener filtering[END_REF], and block-matching and 3D filtering (BM3D) [START_REF] Dabov | Image denoising by sparse 3-d transform-domain collaborative filtering[END_REF], have been successfully applied in image processing. However, the traditional denoising method is easy to face the problem of single task [START_REF] Tian | Deep learning on image denoising: An overview[END_REF] or preserving small targets [START_REF] Lin | Small target detection improvement in hyperspectral image[END_REF].

With the development of deep learning, some methods based on deep learning have been proposed for image denoising [START_REF] Zhang | Deep spatio-spectral bayesian posterior for hyperspectral image non-iid noise removal[END_REF]. In [START_REF] Gondara | Medical image denoising using convolutional denoising autoencoders[END_REF][START_REF] Cho | Simple sparsification improves sparse denoising autoencoders in denoising highly corrupted images[END_REF]

Thesis objective and contributions

This dissertation is devoted to using deep learning methods to process HSIs.

The targets are denoised, classified and detected by analyzing hyperspectral data and fully extracting spectral-spatial characteristics with deep learning models. More specifically, the main contributions are summarized as follows:

•

Thesis outline

This thesis is organized in five chapters.

Introduction

As the main branch of machine learning, deep learning has shown considerable potential in the field of remote sensing classification [START_REF] Ghamisi | Hyperspectral and lidar data fusion using extinction profiles and deep convolutional neural network[END_REF][START_REF] Zhou | Deep learning with grouped features for spatial spectral classification of hyperspectral images[END_REF] owing to its strong capability for big data analysis, which enables it to extract the inherent laws and characteristics of the data [START_REF] Amrani | Deep feature extraction and combination for synthetic aperture radar target classification[END_REF]. CNN has a unique network structure with local connection and weight sharing, which reduces the number of parameters significantly [START_REF] Ghamisi | A self-improving convolution neural network for the classification of hyperspectral data[END_REF] and it have been successfully used for HSI classification [START_REF] Mou | Deep recurrent neural networks for hyperspectral image classification[END_REF][START_REF] Li | Hyperspectral image classification using deep pixel-pair features[END_REF][START_REF] Liu | Hyperspectral image classification based on parameter-optimized 3d-cnns combined with transfer learning and virtual samples[END_REF]. However, the CNN performance is greatly affected by the parameter settings and the loss function may reach a local minimum owing to inappropriate weights. To improve the CNN performance for HSIs, a classification method based on a CNN with parameter tuning is proposed in this Chapter.

Overview of 2D-CNN

A 2D-CNN mainly includes convolutional layers and pooling layers as shown in It can be seen from Figure 1.2 that 2D convolution is performed on 2D input data. When the convolution kernel slides over the input, compute the product of the mutually overlapping pixels and calculate their sum, then a 2D output is obtained.

When 2D convolution operation is done with the stride being 1 × 1, its output O x, y at position (x, y) is defined as:

O x, y = W 1 -1 p=0 W 2 -1 q=0 W p, q I x+p, y+q + b (1.1)
where I ∈ R I 1 ×I 2 represents the input with dimension of

I 1 × I 2 , W ∈ R W 1 ×W 2 is
the convolution kernel, and b is the bias.

In the pooling layers, data can be subsampled by reducing the resolution of the feature maps while the number of feature maps is unchanged. Max-pooling operation calculates the maximum value for patches, and meanpooling (also called average-pooling) operation calculates the average value for patches.

Both of them can be used to progressively reduce the spatial size of the representation, and reduce the amount of parameters and computation in the network.

Hyperspectral classification based on 2D-CNN

To explore the rich information in HSIs, a 2D-CNN is introduced for feature extraction and classification in this subsection. Although CNN reduces network parameters through local connection and weight sharing, its overall performance is influenced by the network parameters, such as the input size, network structure, pooling method, activation function and so on. In the existing literature, the network parameters are generally set by default. Hence, appropriate parameter selection methods are worth studying in order to improve the results.

Optimal parameter selection based on 2D-CNN

To obtain the optimal CNN parameters for HSI classification, a classification method based on a 2D-CNN with parameter tuning (2D-CNN-PT) is proposed. Firstly, considering that HSIs are high-dimensional data composed of numerous spectral bands and they include a significant amount of redundancy. PCA can represent the original data with a set of linearly uncorrelated variables through orthogonal transformation. Therefore, PCA is used to reduce the number of dimensions [START_REF] Zheng | Generalized synthetic aperture radar automatic target recognition by convolutional neural network with joint use of two-dimensional principal component analysis and support vector machine[END_REF][START_REF] Agarwal | Efficient hierarchical-pca dimension reduction for hyperspectral imagery[END_REF].

It's known that HSIs are 3D data and each pixel corresponds to a spectral vector. Then, the network parameters are tuned in turn according to the unique variable principle based on the classification results during classification, which means that only one parameter is changed while the others are fixed to explore the effect of the changed parameter on the experimental results. A set of appropriate parameters can be obtained according to parameter tuning.

Finally, the 2D-CNN with optimal parameters can be used to extract features for hyperspectral classification.

Data set description and assessment criteria 1.3.2.1 Data set description

There are some commonly used hyperspectral datasets used to test the performance of methods and models. Different data sets may have different characteristics in resolution, land-cover classes, image quality and so on. Pavia University dataset and Indian Pines data set are used as our target data in the following simulations. Stone-stel-towers 93

Assessment criteria

In order to evaluate and compare the performance of different methods, overall If there are N classes in a data set and the number of samples in the nth class is λ n . Thus, the total number of samples is λ (λ = N n=1 λ n ). C nn denotes the number of test samples that actually belong to the nth class, and are also classified into nth class. The OA, AA, and κ values can be defined as [START_REF] Liu | Denoising of hyperspectral images using the parafac model and statistical performance analysis[END_REF]:

OA = N n=1 C nn N n=1 λ n × 100% (1.3) AA = 1 N N n=1 C nn λ n × 100% (1.4) κ = N n=1 Cnn λ - N n=1 λnCnn λ 2 1 - N n=1 λnCnn λ 2 × 100% (1.5)

Experimental results

In the experiment, two seven-layer 2D-CNNs are built for Pavia University and Indian Pines, respectively. The initial network structure is listed in Table 1. 3, where Besides, the network weights are randomly initialized by a normal distribution with a mean and standard deviation of 0 and 0.5, the network weights are updated by the Adadelta algorithm with a learning rate of 1, and the number of epochs is set to 100 for the two 2D-CNNs. The following parameters are considered in this research: input size, network structure, number of units in the fully connected layer, activation function, pooling method, optimization method, batch size, number of convolutional kernels and number of epochs. Next, we will explore the influence of these parameters on the results.

k 1 × k 2 × n in convolutional

Input size

Considering the input size, i.e., S × S, has a great influence on the design of the whole network structure, it is optimized first in the experiment. Moreover, the proportion of the training data also has an impact on the classification results.

Therefore, the training data ratio which reflects the proportion between the number of training samples and total samples and input size are considered simultaneously. training data ratio of two HSIs is selected as 0.1, and 27 × 27 is finally chosen as the input size because the corresponding OA value is higher and this size has been widely used in the literature which makes the experimental results more contrasting [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF].

For the subsequent simulations, the input size is fixed at 27 × 27 for both two data sets.

Network structure

The network structure is mainly affected by input size and the number of layers, and convolution kernel size, etc. Under the condition of the input size is 27 × 27 and the number of units in fully connected layer is 128, the layer number and the size of the convolutional kernel are considered together in the experiment. Five 2D-CNNs with different structures are established as shown in Table 1.4.

Table 1.4: Network parameters of different networks.

Net C1 P1 C2 P2 C3 P3 Net 1 4 × 4 × 16 2 × 2 3 × 3 × 32 2 × 2 - - Net 2 4 × 4 × 16 2 × 2 5 × 5 × 32 2 × 2 - - Net 3 4 × 4 × 16 2 × 2 3 × 3 × 32 2 × 2 4 × 4 × 48 - Net 4 4 × 4 × 16 2 × 2 5 × 5 × 32 2 × 2 4 × 4 × 48 - Net 5 4 × 4 × 16 2 × 2 3 × 3 × 32 2 × 2 4 × 4 × 48 2 × 2
The OA values of the two data sets under different network structures are listed in Table 1.5. According to the experimental results in Table 1.5, the highest OA value of Pavia University is obtained under Net 3. For Indian Pines data set, Net 1 and Net 3 show better classification performance. However, the dimension of the feature vector before the fully connected layer of Net 3 is much smaller than that of Net 1 and low-dimensional features help reduce the amount of calculation and storage space. Therefore, Net 3 could be a good candidate for the Pavia University and Indian Pines. We can find from Table 1.6 that the OA value does not always increase as the number of units increases. A high number of units only increases the number of parameters, but may also introduce interference information. From the perspective of computational efficiency and accuracy, the optimal number of units in the fully connected layer for both the Pavia University and the Indian Pines is set to 128 which is more commonly used.

Activiation function and pooling method

A neural network without an activation function becomes a linear system. Hence, the activation function allows the inclusion of nonlinear factors. An activation function is differentiable nearly everywhere [START_REF] Gulcehre | Noisy activation functions[END_REF]. Several commonly used activation functions (f (•)) are shown in Figure 1.7. A real number can be mapped to (0, 1) through the sigmoid function. However, the vanishing gradient problem occurs during back propagation. The hyperbolic tangent (tanh) function is suitable for various obvious features. But, it also suffers from the vanishing gradient problem. ReLU has been widely used in CNNs owing to its efficient computation [START_REF] Glorot | Deep sparse rectifier neural networks[END_REF]. Furthermore, it does not suffer from the vanishing gradient or exploding gradient problem. Through experiments we found that the OA values of the Pavia Unicersity and the Indian Pines are higher when ReLU is selected as activation function. Therefore, ReLU is a good choice for activation function.

The pooling layer is usually employed between the convolutional layers. Pooling operation can reduce the resolution of feature maps and prevent over-fitting to a certain degree. In CNNs, the commonly used pooling methods are max-pooling and mean-pooling [START_REF] Boureau | Learning midlevel features for recognition[END_REF]. Max-pooling effectively retains the texture information of images, whereas mean-pooling effectively preserves the background information of images. In theory, texture information is more useful for image classification. In the experiments, the OA values are also higher for both HSIs under max-pooling.

Therefore, max-pooling is selected as the pooling method for the Pavia University and Indian Pines.

Optimization method

Some optimization methods [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF][START_REF] Diederik | Adam: A method for stochastic optimization[END_REF]: Adadelta, stochastic gradient descent (SGD), adaptive gradient algorithm (Adagrad), root mean square prop (RMSprop), adaptive moment estimation (Adam) are compared in this part. It can be seen from Table 1.7 that the highest OA value is obtained when Adam is used as the optimizer. Therefore, Adam is chosen as the optimizer for the two data sets. As seen from the Figure 1.9, for Pavia University, the OA values are lower when the number of kernels in the first convolution layer is 8 and 96, while for other kernel A set of optimal parameters can be finally obtained after parameter tuning. However, overfitting is still a problem need to be faced. To prevent complex coadaptations on the training data, dropout can be used to reduce overfitting by randomly omitting some hidden units from the network [START_REF] Hinton | Improving neural networks by preventing coadaptation of feature detectors[END_REF]. Therefore, dropout is introduced into the optimized 2D-CNN to further improve the classification performance of the network.

Comparison of classification results

For better visual observation of the effectiveness of the proposed method, feature extraction methods based on factor analysis (FA) and DBN are considered for comparison to better evaluate the performance of the proposed method. FA is a linear statistical method that uses fewer numbers of factors to replace original data [START_REF] Attias | Independent factor analysis[END_REF]. DBN is composed of multiple layers of latent variables and it usually takes a 1D vector as input, which learns deep features via pretraining in a hierarchal manner [START_REF] Liu | Discriminative deep belief networks for visual data classification[END_REF][START_REF] Eldeib | Breast cancer classification using deep belief networks[END_REF][START_REF] Li | Hyperspectral classification based on texture feature enhancement and deep belief networks[END_REF].

The For Indian Pines, we can find that the classification maps in 2D-CNN-PT is the clearest. Although the optimal parameters are obtained under a limited set of explored parameters, these results can provide a reference for parameter initialization settings of other models, and it will save time for parameter tuning of models.

Conclusion

Chapter 2

Supervised feature extraction based on 3D-CNN for hyperspectral classification

Introduction

In the Chapter 1, 2D-CNN is introduced to mine information in HSIs. 2D-CNN can directly take 2D data as input and has great potential in preserving spatial structure of the target. However, HSIs that usually contain hundreds of spectral channels not only contain spatial information but also provide abundant spectral information. Therefore, 2D-CNN has limitations in retaining the spectral information. In [START_REF] Yang | Hyperspectral image classification using two-channel deep convolutional neural network[END_REF], a two channels deep CNN composed of a 1D-CNN and a 2D-CNN is proposed to learn jointly spectral-spatial information, but the characteristics of the spectral domain and the spatial domain are extracted separately. Considering that the 3D data can be directly input into the 3D-CNNs, which helps to fully exploit the spectral and spatial information at the same time and avoids complex data reconstruction [START_REF] Zhao | An efficient macroblock-based diverse and flexible prediction modes selection for hyperspectral images coding[END_REF][START_REF] Vakil | A robust multi-stage informationtheoretic approach for registration of partially overlapped hyperspectral aerial imagery and evaluation in the presence of system noise[END_REF][START_REF] Huang | Transfer learning with deep convolutional neural network for sar target classification with limited labeled data[END_REF][START_REF] Chen | Deep learning-based classification of hyperspectral data[END_REF][START_REF] Mei | Hyperspectral image spatial super-resolution via 3d full convolutional neural network[END_REF], 3D-CNN is developed to fully exploit the spectral-spatial information of HSIs in this chapter.

Hyperspectral classification based on 3D-CNN

2D-CNNs mainly capture features from the spatial domain, but 3D-CNNs could help to obtain spatial-spectral features of tensors [START_REF] Zhong | Spectralspatial residual network for hyperspectral image classification: A 3-d deep learning framework[END_REF]. Therefore, a 3D-CNN is considered to fully exploit the information among hyperspectral data. The flow chart of a conventional 3D-CNN for hyperspectral classification is illustrated in When 3D convolution operation is performed with stride of 1 × 1 × 1, the output at position (x, y, z) can be calculated by:

O x, y, z = W 1 -1 p=0 W 2 -1 q=0 W 3 -1 r=0 W p, q, r I x+p, y+q, z+r + b (2.1)
where

I ∈ R I 1 ×I 2 ×I 3 represents the input with dimension of I 1 × I 2 × I 3 , W ∈ R W 1 ×W 2 ×W 3
is the convolution kernel, and b is the bias. In order to preserve as much spatial and spectral information of the target as possible, a 3D data block centered on the target can be used as the input of 3D-CNN. However, HSIs usually contain hundreds of spectral bands and the information of adjacent bands is highly correlated. If the input data contains all the bands, not only the amount of calculation is increased but curse of dimensionality may be caused [START_REF] Hughes | On the mean accuracy of statistical pattern recognizers[END_REF]. Therefore, dimensionality reduction is an important step in hyperspectral classification based on 3D-CNN.

In subsection 1.3, PCA is used to reduce the dimensionality of HSIs. In addition to the commonly used PCA, band selection is one of the alternative ways to reduce the dimensionality of HSIs.

Band selection

When band selection is implemented unsupervised, a commonly-used approach is to combine all possible subsets to find the most satisfactory objective value under some criterion, such as signal-to-noise ratio (SNR), optimum index factor (OIF), which may result in excessive computational complexity and cost. Moreover, the spectral signatures which are important to differentiate the materials are easy to be ignored or destroyed [START_REF] Ghamisi | Advanced spectral classifiers for hyperspectral images: A review[END_REF].

Selecting the bands with more invariant features and low correlation can help us improve the training efficiency and obtain better network performance. In order to efficiently select the band of HSIs, a fast band selection method based on a modified Douglas-Peucker algorithm named FMDP is proposed for hyperspectral classification. In the FMDP method, the number of invariant features calculated by scale invariant feature transformation (SIFT) algorithm and the spectral values are taken into account as evaluation criteria to ensure band information and spectral characteristics. The bands are simplified by limiting the distance between the selected adjacent bands, which not only reduces the band correlation but also reduces the number of iterations.

Introduction of SIFT and DP algorithms

The SIFT algorithm is proposed by David [START_REF] David G Lowe | Object recognition from local scale-invariant features[END_REF][START_REF] David G Lowe | Distinctive image features from scale-invariant keypoints[END_REF] which can be used to transform an image into local feature vectors. The features are invariant to image scaling, translation and rotation, which is of great help to image processing [START_REF] Sharif | Scene analysis and search using local features and support vector machine for effective content-based image retrieval[END_REF].

To find the distinctive features, a difference-of-Gaussian pyramid needs to be constructed at first to search for potential interest points that are brighter or darker than its surroundings. Next, the location and scale of candidate keypoints are determined by performing a detailed model fit to the nearby data and they are selected according to their stability. Then, each keypoint is assigned one or more orientations based on its local image patch. Finally, the descriptor is built for each keypoint in its local neighborhood, which is measured by the local image gradients.

The SIFT keypoints are useful for object recognition and matching due to their invariance and distinctiveness.

DP algorithm also known as the Ramer DP algorithm, is proposed by Urs Ramer [START_REF] Ramer | An iterative procedure for the polygonal approximation of plane curves[END_REF], David Douglas and Thomas Peucker [START_REF] David | Algorithms for the reduction of the number of points required to represent a digitized line or its caricature[END_REF]. It is perfected by other scholars in the following decades, which has been widely used to compress the redundant graphical points and extract keypoints. to become the end point of the two new segments AC and BC. For each segment created, repeat the previous process until the maximum distance not satisfies the threshold value, and a polyline which can be used as the approximation of curve is eventually obtained.

Proposed FMDP method for band selection

The goal of selecting the band is to keep the bands containing more information and low correlation. Considering the distinctive features from scale-invariant keypoints obtained by SIFT are invariant to image scale and rotation, which is of immense benefit to image processing, the number of keypoints can be introduced to estimate the information of band. Since an HSI is a 3D data cube and each channel collects geometrical information of the same scene in the spatial domain [START_REF] Ma | Hyperspectral image classification based on deep deconvolution network with skip architecture[END_REF],

keypoints detection can be individually performed on the 2D image corresponding to each band to estimate the number of keypoints. Besides, different dimensions of HSI correspond to different energies represented by the square of Frobenius norm (F-norm). The smaller the corresponding F-norm value of the image, the lower the corresponding energy and the less useful information contained. In order to better estimate the information contained in the band, an evaluation criteria (Q(h 3 ), h 3 = 1, 2, ...H 3 ) of the h 3 th band considering the aforementioned keypoint number and F-norm is defined as follows:

Q f n (h 3 ) = H(:, :, h 3 2 F = H 1 h 1 =1 H 2 h 2 =1 |H(h 1 , h 2 , h 3 )| 2 (2.
2)

Q(h 3 ) = Q kp (h 3 ) × Q f n (h 3 ) (2.3)
where the raw HSI with H 1 rows, H 2 columns and

H 3 spectral bands, Q f n (h 3 )
represents the F-norm value of the h 3 th band, Q kp (h 3 ) represents the number of the keypoints of the image corresponding to the h 3 th band. Both Q kp (h 3 ) and

Q f n (h 3 )
are normalized to [0 1]. The larger the Q(h 3 ) value of a band, the greater the possibility that it has high quality.

In addition to the selected bands with high quality and abundant information, low correlation between the selected bands is also essential. Besides, each pixel in HSIs corresponds a spectral curve, which can be used to distinguish land-cover classes. Therefore, we expect the spectral characteristics can be reserved as much as possible and the corresponding polylines of selected bands are able to distinguish the target. To get rid of the computational complexity caused by combining all possible sub-bands to choose the least relevant one, we are looking for a new way to reduce the band correlation. It can be known that the neighboring bands are more correlative through calculating the correlation coefficient between the bands.

Therefore, a modified DP algorithm is used to select bands and reduce the band correlation by controlling the distance between adjacent selected bands. For each HSI, the evaluation criteria Q kp (h 3 ) can be calculated. Given two threshods (T 1 , T 2 ) and the number of bands to select N , we can get a set of candidate bands based on the modified DP algorithm. Sort all the candidate bands according to Q(h 3 ) value and the top N bands with the largest Q(h 3 ) value are the final selection. Besides, to reduce the number of iterations and increase the difference of the selected bands, the set of the two thresholds try to make the number of candidate bands is near to N toward 0 and T 2 tries to be about twice as large as

As shown in

T 1 .
When the proposed FMDP is utilized, only a few iterations are needed and the spectral information can be preserved as much as possible, which not only greatly improves the computational efficiency but also lays a solid foundation for subsequent classification.

Experimental results of band selection

To verify the effectiveness of the proposed band selection method, comparative experiments on Pavia University and Indian Pines data sets are carried out in this subsection.

When the band to be selected is 10, for Pavia University, T 1 and T 2 are set to 9 It can be observed from Figure 2.6 and 2.7 that the selected bands try to fill the band range instead of focusing on a narrow spectral band, which helps reduce data redundancy and data correlation. In addition, most of the selected bands are key locations that affect the contour of the spectral curve and the fitted spectral curves are closer to the original curves, which allows the selected band can provide more In order to better verify the performance of the proposed method, four other unsupervised band selection methods, maximum-variance principal component analysis (MVPCA), adaptive band selection (ABS), minimum noise band selection method (MNBS), and band column selection (BCS) are considered for comparison. MVPCA [START_REF] Chang | A joint band prioritization and band-decorrelation approach to band selection for hyperspectral image classification[END_REF] prioritizes bands using loading-factors matrix via the corresponding eigenvalues and eigenvectors to achieve band reduction. ABS is developed in [START_REF] Liu | A new method of hyperspectral remote sensing image dimensional reduction[END_REF] based on OIF [START_REF] Chavez | Statistical method for selecting landsat mss[END_REF] and the high correlation between adjacent bands is also take into account.

In [START_REF] Sun | A new band selection method for hyperspectral image based on data quality[END_REF], a MNBS is proposed based on data quality integrating both SNRs and correlation of bands. Because each HSI can be represented as a tensor data and selecting the most desirable column subsets is an analogy to band selection for HSIs, BCS is employed in [START_REF] Wang | Unsupervised hyperspectral image band selection via column subset selection[END_REF] Besides, when the number of selected bands is small, the proposed FMDP achieves higher accuracy compared with other mentioned methods. It's because the band selected by the proposed FMDP has a wide distribution and retains good spectral characteristics. When the number of selected bands exceeds 30, the classification results obtained by BCS, MNBS, MVPCA and the proposed FMDP are better than the results obtained with all bands, which proves that the appropriate number of selected bands can help to obtain better results compared to all bands being considered.

For Indian Pines (Figure 2.10), FMDP, BCS, MNBS and MVPCA have similar relationship between OA values and the number of selected bands. The OA values increase with the number of selected bands at the beginning, and then tend to be stable. By comparing with the OA values of all bands, it can be inferred that OA values decrease after a certain range. The OA values obtained based on ABS increase with the number of selected bands, but the OA values are low compared with the results got by other methods. When the selected band is fixed at 10, the proposed FMDP helps obtain the highest OA value. In addition, when the selected band is In general, the classification results of Pavia University and Indian Pines demonstrate that choosing appropriate number of bands instead of all bands for classifications helps obtain better results and improves efficiency. In particular, the proposed method exhibits best performance with a few iterations when the number of selected bands is small.

Solutions with limited labeled samples

After the HSI has been reduced in dimensionality, it can be input into 3D-CNN for feature extraction and classification. The designed framework is showed in According to the unique variable principle described in subsection 1.3, the parameters of the 3D-CNN can be also tuned one by one to get a set of optimal parameters, which helps improve network performance. However, although the dimensionality of HSI is reduced, there are still a large number of parameters need to be optimized in a 3D-CNN, which requires sufficient labeled samples. Unfortunately, the labeled samples in hyperspectral data is limited and the collection of labeled samples is labor-consuming and time-consuming, which has a negative impact on the classification results.

Transfer learning

Transfer learning (or knowledge transfer) has attracted widespread attention in recent years and has been widely applied in computer vision [START_REF] Gopalakrishnan | Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection[END_REF] and natural language processing [START_REF] Houlsby | Parameter-efficient transfer learning for nlp[END_REF], etc. To better understand transfer learning, the definitions of a "domain" and a "task" are given, respectively [START_REF] Sinno | A survey on transfer learning[END_REF].

A domain D consists of a feature space X and a marginal probability distribution P (X), where X = {x 1 , x 2 , ..., x n } ∈ X and x i represents the ith fea- can be used to predict the corresponding label f (x). From a probabilistic viewpoint, f (x) can be written as P (y|x).

For simplicity, we only consider the condition where there is one source domain D S , and one target domain D T . The source data is denoted as

D S = {(x S 1 , y S 1 ), (x S 2 , y S 2 ),
..., (x Sn , y Sn )}, where x S i ∈ X S is the data instance and y S i ∈ Y S is the corresponding label. Similarly, the target domain data can be denoted as

D T = {(x T 1 , y T 1 ), (x T 2 , y T 2 ), ..., (x Tn , y Tn )}, where x T i ∈ X T is the input and y T i ∈ Y T
is the corresponding output. Then, the definition of transfer learning can be defined From the definition of transfer learning, we can see that transfer learning has the ability of a system to recognize and apply knowledge and skills learned in previous domains/tasks to novel domains/tasks. There are some commonly used approaches such as instance-based transfer learning (or instance-transfer) approach, featurerepresentation-transfer approach, parameter-transfer approach, etc.

If there is another HSI (source data) with enough labeled samples and the same feature space as the HSI to be classified (target data), then transfer learning can be used to help us reduce the need for labeled samples of target data. The designed framework of 3D-CNN with transfer learning (3D-CNN-TL) is illustrated in Figure 2.12, where DR means dimensionality reduction.

Figure 2.12: Proposed 3D-CNN-TL framwork.

The training procedure of designed 3D-CNN-TL can be divided into five steps:

Step 1: the source data and target data are reduced to the same dimension.

Step 2: two 3D-CNNs (3D-CNN I and 3D-CNN II) with same network structure except for the output layer are established, and the number of units in output layer is equal to the number of classes contained in the corresponding data set.

Step 3: the 3D-CNN I with source data as input is trained and optimized by sufficient training samples.

Step 4: knowledge transfer can be made: the weights in convolutional layers and pooling layers in the 3D-CNN II can be transferred from the same layers of the 3D-CNN I in Step 3, and the weights of other layers are initialized randomly.

Step 5: the 3D-CNN II can be further fine-tuned by the training samples from the target data to obtain better classification performance. Besides, since the weights of the middle layers of 3D-CNN II are transferred from 3D-CNN I, the optimization of 3D-CNN II has lower requirements on the number of samples and the number of iterations.

Virtual samples

When the source data is available, knowledge transfer can be made from the source data to the target domain to improve the network performance by avoiding rather expensive data labeling efforts [START_REF] Fielding | Spiral ct in the evaluation of flank pain: overall accuracy and feature analysis[END_REF]. However, not all target data have corresponding source data. Without the help of source data, transfer learning cannot be implemented. If the source data is absent, as a pseudo-sample transformed from the original sample of the target data, virtual samples are also a solution to make up for the lack of HSI samples [START_REF] Chen | Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[END_REF][START_REF] Radford | Unsupervised representation learning with deep convolutional generative adversarial networks[END_REF].

If ϕ o represents an original sample in the HSI with a size of S × S × H dr , then the virtual sample ϕ v can be defined as:

ϕ v = ηϕ o + N (µ, σ 2 ) (2.4)
where η is the correlation coefficient, and N (µ, σ 2 ) denotes the Gaussian noise with a mean of µ and a variance of σ 2 , which is used to simulate the interference of the external environment to the samples. After mixing the virtual samples with the original ones, the overall number of training samples can be greatly increased.

3D-CNN with transfer learning and virtual samples

Since both transfer learning and virtual samples can make contributions to solve the problem of limited training samples, a hybrid method named 3D-CNN-TV which combines 3D-CNN, transfer learning, and virtual samples, is proposed in order to further improve HSI classification. At last, the training samples consisting of the original and the virtual ones can help to further fine-tune the 3D-CNN-TV model. In addition, due to the introduction of virtual samples, not only the number of training samples is greatly increased, but the robustness of the model can be improved.

Experimental results

In order to compare and verify the effectiveness of the proposed models, the classification results based on single 3D-CNN, 3D-CNN-TL, 3D-CNN-VS, and 3D-CNN-TV are analyzed and discussed in this subsection. In order to improve efficiency and reduce the amount of calculation, the dimensionality of the data sets involved in this subsection is reduced, and both PCA and proposed FMDP are used for comparison. Specifically, the original HSI (

H ∈ R H 1 ×H 2 ×H 3 ) is reduced to a lower dimensional image (H dr ∈ R H 1 ×H 2 ×H dr , (H dr < H 3 )).
For each pixel, a 3D tensor with a size of S × S × H dr is selected as the input of network. In the experiments, we choose Pavia University and Indian Pines as the target data to study and analyze the network performance in hyperspectral classification. The dimensionality of both two data sets is reduced to 10 by PCA and FMDP, respectively. The input size is fixed at 27 × 27 × 10. In the following, the parameter settings of different models are introduced in detail.

Details of 3D-CNN

As described in Figure 2.11, the network structure of 3D-CNN is listed in Table 2.1 with input size being 27 × 27 × 10. 128 is chosen as the batch size and Adam is chosen as the optimizer.

Details of 3D-CNN-TL

The center of Pavia city (Pavia Centre) shown in Figure 2.14 (a) is acquired by the ROSIS sensor, the same as the sensor that obtained Pavia University. After discarding the pixels containing no information, there are 1096 × 715 pixels with Pavia Centre and Salinas are used as the corresponding source data for Pavia University and Indian Pines in the transfer learning experiment, respectively. We assume that both Pavia Centre and Salinas contain sufficient labeled samples.

To ensure the stability of the network performance, 70% of samples of each

Details of 3D-CNN-VS

Virtual samples are introduced to the 3D-CNN model according to Eq. (2.4).

To reduce the difference between the virtual samples and real samples, the value of correlation coefficient η should be closed to 1. Therefore, η is set to a uniformly distributed random number in [0.9, 1.1] in the experiment. Considering that the number of virtual samples and the interference N can also influence the network performance, a sensitivity analysis is conducted in this part to achieve better network performance.

If the number of original training samples selected from among the target data is n t , then the number of virtual samples will be n v = r × n t where r represents the ratio between the number of virtual samples and the number of original samples.

The mean value µ of noise N is set to 0, and the variance σ 2 is set to 0.01 at the beginning. In the experiment, the virtual samples and the original samples are mixed together to form the training data set. When the value of the ratio r is different, i.e., when the number of virtual samples is different, for Indian Pines.

It can be seen from Figure 2.15 that for Pavia University and Indian Pines, the OA values increase first and then decrease with r. For both data sets, the highest value is arrived when the number of virtual samples is 1 × n t , i.e., the number of virtual samples is equal to the number of original samples. Therefore, the number of virtual samples is set to n v = n t for the 3D-CNN-VS model in the classification.

When introducing virtual samples, the noise N can also affect the classification performance. Keeping the number of virtual samples fixed at n v = n t and the mean value of N at 0, and changing the variance value σ

Details of 3D-CNN-TV

As mentioned in Section 2.2.2.3, a 3D-CNN-TV model combined with transfer learning and virtual samples can be constructed for the classification. Meanwhile, the virtual samples with zero mean and noise variance of 0.001 could be generated from the original samples. Then, the virtual samples are mixed with the original ones to fine-tune the 3D-CNN-TL model with transferred weights. When the 3D-CNN-TV is well optimized, input the target data into the optimized 3D-CNN, and the category prediction can be obtained.

Comparison of classification results

In order to make a visual comparison, the classifications maps of Pavia University and Indian Pines obtained from 3D-CNN-based models are illustrated in Figure 2.16 and Figure 2.17. For Indian Pines, we can find from Figure 2.17 that the misclassified pixels are mainly concentrated in the upper left area. The misclassification of pixels is greatly reduced in Figure 2.17

Conclusion

When GAN is trained in an adversarial way requiring no labeled samples. It has been one of the most promising unsupervised learning representatives [START_REF] Radford | Unsupervised representation learning with deep convolutional generative adversarial networks[END_REF]. In [START_REF] Zhan | Semisupervised hyperspectral image classification based on generative adversarial networks[END_REF], a semi-supervised framework based on 1D-GAN is established for hyperspectral classification with a small number of labeled samples. But only spectral features are extracted, which are far from enough for classification. In [START_REF] Zhang | Unsupervised feature extraction in hyperspectral images based on wasserstein generative adversarial network[END_REF], Zhang proposes a novel modified GAN whose generator and discriminator are designed in the form of fully deconvolutional network and fully convolutional network to extract the features without supervision. Nevertheless, only spatial information is taken as input when the modified GAN is trained, which can be treated as 2D convolution on multiple channels. Hyperspectral data is a tensor data, which contains not only spatial information but also the spectral characteristics of the target. Fully mining the spectral-spatial features in HSIs is helpful for classifying the target. Considering 3D convolution operation is performed in space and spectrum, we want to design a framework based on 3D-GAN in which the generator and discriminator are built on fully 3D convolution and 3D deconvolution subnetworks to fully extract the spectral-spatial features with unsupervised learning for classification. Even worse, gradient explode or gradient vanish may be caused. To solve the aforementioned problems, an improved WGAN adding gradient penalty to enforce the Lipschitz constraint, named WGAN-GP, has been developed and has been demonstrated that gradient penalty is an effective way to solve exploding and vanishing gradients. The new objective is [START_REF] Gulrajani | Improved training of wasserstein gans[END_REF]:

Overview of GAN

L = E x∼pg [D(x)] -E x∼pr [D(x)]
Original critic loss in WGAN

+ λE x∼p x [( ∇ xD(x) 2 -1) 2 ]
Gradient penalty

(3.2)
where λ is a gradient penalty coefficient, and p x samples uniformly along straight lines between pairs of points sampled from the p g and p r . It has been demonstrated

that the training of WGAN-GP is more stable, and gradient penalty is an effective way to solve exploding and vanishing gradients.

Proposed unsupervised feature extraction method based on 3D-WGAN-GP

We have learned that HSI is a high-dimensional data which contains hundreds of spectral bands and it's difficult for generator to produce high-dimensional data, which makes the training of GAN is difficult. In Chapter 2, a band selection method is proposed to reduce the dimension of HSIs. In this Chapter, a new dimensionality reduction method is designed based on 1×1 convolution and transfer learning, which can provide an alternative way for data dimensionality reduction.

Proposed dimensionality reduction method

When dimensionality reduction is conducted, the spatial size (height and width) of HSI is unchanged and only the dimension corresponding to the band (depth) is reduced. Considering this characteristic of dimensionality reduction and being inspired in [START_REF] Lin | Network in network[END_REF], we propose to use 1 × 1 convolutions to obtain lower-dimensional and more abstract features. Furthermore, we want to reduce the dimension of target data through transfer learning with an unsupervised process, which means labeled samples of target data are not required during the dimensionality reduction process.

The designed framework of dimensionality reduction is depicted in Figure 3.4.

Take Pavia University as an example, we give a specific explanation of the dimensionality reduction framework. Since the number of spectral bands is 103 for Pavia University and 102 for Pavia Centre, to make the target data and the corre- When the 3D-CNN is well-trained, the first convolutional layer and pooling layer can be transferred as the dimensionality reduction module. When the target data is input to this module, the lower-dimensional data can be obtained.

Details of proposed unsupervised feature extraction method

To extract the spectral-spatial features in HSIs with an unsupervised process, a 3D convolution model based on WGAN-GP is designed and the framework of the proposed method is shown in

Experimental results

To verify the performance of the proposed method, Pavia University and Indian

Pines are chosen as target data, and Pavia Centre and Salinas are used as source data as Section 2.3.2. According to the previously proposed dimensionality reduction method, the dimension of the target data is reduced to 10 with the help of source data and transfer learning. For each pixel, a 3D block with size being 27 × 27 × 10.

According to the size of input data, a 3D-WGAN-GP is built and some details are described in Table 3.2, where Af represents activation function. It can be seen from Table 3.2 that all activation functions of layers in discriminator are leaky rectified linear unit (leakyReLU). ReLU is chosen as the activation function of layers in generator, except the output layer, which uses hyperbolic tangent function (Tanh). Besides, in convolution-related operations, the stride is set as 1 × 1 × 1 and the padding is set as valid which means there is no zero padding operation on the boundary data. The generated samples obtained from the uniformly distributed noise with size being 100 × 1 through generator are mixed with the real samples of source data, and then are fed into discriminator. In the training procedure, dropout is introduced to avoid overfitting and a mini-batch based on the root mean square prop (RMSProp) algorithm [START_REF] Tieleman | Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude[END_REF] which performs well even on very nonstationary problems is employed and the size of batch is set as 32.

To evaluate the performance of the proposed method, the unsupervised feature extraction methods based on standard GAN, 3D-WGAN [START_REF] Zhang | Unsupervised feature extraction in hyperspectral images based on wasserstein generative adversarial network[END_REF] and the proposed 3D-WGAN-GP are compared. However, it's difficult to directly evaluate the extracted features. Therefore, the classification performance based on the extracted features is used to estimate the quality of the extracted features. Better classification results reflect that the corresponding methods have stronger feature extraction ability. In From Table 3.3, it can be seen that the classification accuracy of meadows, bare soil and bitumen is relatively low compared to other categories, which may be caused by large intra-class variation and small interclass variation. The features obtained by 3D-WGAN-GP help obtain higher OA, AA and κ values compared with the features obtained by GAN and 3D-WGAN. Besides, SVM performs better than softmax on Pavia University.

From Table 3.4, it can be seen that when the features obtained by GAN are used for classification, the classification accuracy of many land-cover classes is less than 90%, which is unsatisfactory. When the features obtained by 3D-WGAN and 3D-WGAN-GP are used for classification, the classification results have been greatly which proves the ability and potential of 3D convolution in feature extraction.

Conclusion

High dimensionality and limited labeled samples are two issues we have to face when we classify hyperspectral data. In order to reduce the dimension of HSIs, a novel dimensionality reduction method based on 1 × 1 convolutions and 1 × 1 pooling is proposed to obtain lower dimensionality data containing more abstract and high-level features. In order to get the rid of the limitation of labeled samples, an The optimized discriminator is transferred and utilized as an unsupervised feature extractor to help solve the problem of insufficient labeled samples. Experimental results prove that the performance of the proposed method is better than that of GAN and 3D-WGAN. In addition, the proposed method provides an alternative way for dimensionality reduction and unsupervised feature extraction of HSIs.

Overview of AE

Traditional AE [START_REF] Bengio | Greedy layer-wise training of deep networks[END_REF] as shown in Then this representation Y can be decoded to a reconstruction one O. The output size of AE is the same as its input size. These two steps can be expressed by the formula as: Traditional AE usually takes the form of a 1D vector as input, which has limi-tations on retaining the spatial information of the original data. Convolution-based operation can be flexibly performed on tensor data and it has been widely used in image processing. CAEs replacing fully connected layers by convolutional layers, which can not only train the network unsupervised, but also process multi-dimensional data more flexibly. Taking 3D convolution as an example, when the input is I with size of

Y = f (WI + b) (4.1)
I 1 × I 2 × I 3 (I ∈ R I 1 ×I 2 ×I 3 ), the convolution kernel is W with size of W 1 × W 2 × W 3 (W ∈ R W 1 ×W 2 ×W 3 )
, and the stride is 1 × 1 × 1, its output is defined as:

O x, y, z = W 1 -1 p=0 W 2 -1 q=0 W 3 -1 r=0 W p, q, r I x+p, y+q, z+r + b (4.3)
where O x, y, z means the output at position (x, y, z), W p, q, r denotes the kernel value of position (p, q, r), and I x+p, y+q, z+r represents the input value at position (x + p, y + q, z + r). Then, the MSE value can be calculated by Eq. (4.4). and input (I x, y, z ), as described in Eq. (4.4). When the network can reconstruct the input data well, we believe that the 3D-CAE has a strong ability to mine the useful information in the data.

E(I, Y)

= 1 I 1 × I 2 × I 3 I 1 -1 x=0 I 2 -1 y=0 I 3 -1 z=0 (I x, y, z -O x, y, z ) 2
Thirdly, multi-level features from the optimized encoder are obtained. The hierarchical structure of the encoder from the bottom to top provides us with features of different levels and different scales. Max-pooling is introduced to reduce the feature dimension and increase feature invariance [START_REF] Zuo | Learning contextual dependence with convolutional hierarchical recurrent neural networks[END_REF]. The filter size of max-pooling is set to equal to the size of the corresponding feature map. Through pooling operations, each layer can get a feature vector containing different information. The final features are concatenated by these feature vectors from multiple layers of encoder to make them contain more information and have high scale robustness. It is worth noting that the proposed multi-level features come from a single network. Compared with training multiple networks to obtain multi-level features, the proposed method is more effective and greatly saving training time. The goal of the proposed method is to make full use of the well-trained network to obtain as much information as possible, and then help to improve the subsequent classification accuracy.

Experimental results

In order to compare and study the performance of the proposed feature extraction method, experiments are performed on Pavia University and Indian Pines. Based on the experimental results of the relationship between band and accuracy in Chapter 2, the bands of the two data sets are reduced to 10 by PCA in order to reduce the amount of calculation and improve the efficiency of network training [START_REF] Agarwal | Efficient hierarchical-pca dimension reduction for hyperspectral imagery[END_REF][START_REF] Zhu | Generative adversarial networks for hyperspectral image classification[END_REF].

Network Construction

For each pixel in HSIs, a 3D block with a size of S × S × H dr centered on the observed pixel is selected as the input to construct the network, where S × S represents the spatial neighborhood window around the observed pixel and H dr means the dimension after dimensionality reduction. Taking 13 × 13 × 10 as an example, the corresponding 3D-CAE structure is given in

× 5 × 3 × 32 5 × 5 × 5 × 32 Conv-3 5 × 5 × 5 × 32 3 × 3 × 3 × 64 3 × 3 × 3 × 64 Conv-4 3 × 3 × 3 × 64 3 × 3 × 3 × 128 1 × 1 × 1 × 128
In Table 4.1, Conv-n represents the nth convolutional layer and kernel of k 1 × k 2 × k 3 × k 4 means that there are k 4 convolution kernels with kernel size being

k 1 × k 2 × k 3 in the current layer.
Besides, the stride is set to 1 × 1 × 1 during the convolution operation. ReLU is mainly used as an activation function to introduce nonlinear mapping into the network, except for the last deconvolution layer with sigmoid. Adam [START_REF] Diederik | Adam: A method for stochastic optimization[END_REF] is selected as the optimizer to update the weights. For Indian Pines data set, when single-level features are used for classification, it can be found from Table 4.3 that the performance of single-level features of Prediction IV and Prediction III is not as good as Prediction II. Prediction II is the best among classification results based on single-level features, but the classification accuracy of the Grass-pasture-mowed, Oats, Wheat and Buildings-grass-trees is less than 80%. When multi-level features are used for classification, the classification accuracy of the Grass-pasture-mowed, Oats, Wheat and Buildings-grass-trees is increased by 14%, 25%, 7%, and 7% compared with Prediction II. In addition, the highest OA, AA, and κ values are achieved when multi-level features are used. Pre- Moreover, even the peak value of a single-level features is about 2% lower than that of multi-level features.

Comparison and analysis of experimental results

Classification

In general, the results that are based on multi-level features are better than those of single-level features for both data sets, which proves that comprehensive consideration of the feature information of different layers can further improve the results of hyperspectral classification. For the Pavia University data set, it can be seen that there are many pixels in the green area that are incorrectly classified into the yellow, and some pixels in the sienna region are misclassified into the red in Figure 4.9 (c) -(e). Besides, the misclassified pixels in the green and sienna region are greatly reduced in Figure 4.9 (f) and (g), but some pixels in the purple region are still not correctly classified, especially in Figure 4.9 (e). Overall, the classification map in Figure 4.9 (h) is the clearest. For the Indian Pines data set, there are many misclassified pixels in

Proposed 3D-M 2 CAE framework for small target feature extraction and classification

In Subsection 4.3, multi-level features from different layers of the same input data are studied. Generally, the input size of different targets in the model is the same. However, the relationship between different targets and input size may be different. Therefore, it's necessary to improve the feature robustness of the target to the input size, especially when there are small targets. In order to achieve this goal, a 3D-M 2 CAE framework with multi-size and multi-model is proposed in this subsection.

Details of proposed framework

In the proposed framework, three data blocks of different input sizes centered on the observed pixel are selected as inputs to obtain features as shown in Figure 4.11. Compared with training multiple 3D-CAEs separately to obtain features, the proposed framework is trained in a progressive way with the help of transfer learning, which is more efficient and greatly saves training time. Besides, benefiting from progressive training, the quality of the reconstructed image can be also improved [START_REF] Karras | Progressive growing of gans for improved quality, stability, and variation[END_REF],

especially when large-size or high-dimensional images need to be reconstructed.

Experimental results

Data set description

In order to verify the performance of the proposed method, two data sets are used as our target data. The first data set named HYDICE is collected by Hyperspectral Digital Imagery Collection Experiment (HYDICE). The original image has 1280 × 320 pixels with 220 spectral bands. Since it's not convenient and efficient to directly process the entire image, 148 spectral bands are retained after removing low-information and noise bands [START_REF] Su | Multisensor fusion with hyperspectral imaging data: detection and classification[END_REF]. In addition, we select part of the image that contains the target of interest as the new data set (Figure 4.12), which is denoted as HSI a . The second data is Pavia University, which is described in detail in In order to evaluate the quality of learned features by the proposed method, the classification results based on learned features are used to measure their effectiveness.

SVM with linear kernel is selected as the classifier. OA, AA and κ values are mainly used to evaluate the classification results.

Result analysis of HSI a data set

In order to study the effect of the input size on the feature performance, we firstly focus on the results of small targets (Target 1, Target 2 and Target 3 in For Target 2, we can find from Figure 4.17 In general, when single 3D-CAEs is used to obtain features, the feature performance of different targets respond differently to the input size. The input size has a great influence on the classification results. However, the proposed 3D-M 2 CAE can help us obtain better results than any single 3D-CAEs without limitation of input size. Besides, the classification accuracy is also greatly improved, which proves that the proposed method can effectively help small targets improve the classification accuracy. From Figure 4.20, we can find that when single 3D-CAE is used to extract features, the AA values are low especially when the input size is small. The AA value got by proposed 3D-M 2 CAE is about 98%, which is 4% higher than 3D-CAEs I and II, and 2% higher than 3D-CAE III when the corresponding H × H × H dr is In general, the proposed method not only helpful for improving the classification of small targets, but also applicable to other targets.

Result analysis of

Conclusion

In this chapter, two frameworks based on 3D-CAE are designed to get rid of lim- The results of Pavia University demonstrate that the proposed framework is not only helpful for improving the classification of small targets, but also applicable to other targets.

Chapter 5

Deep learning models for improvement of target detection

Introduction

In the previous chapters, supervised feature extraction and unsupervised feature extraction have been studied for hyperspectral classification. In this chapter, we focus on target detection of HSIs. With a known spectral signature can also be called a spectral template, comparing the spectral template with the pixels in a scene can determine whether target is present or not. There are some detectors which are commonly used for target detection [START_REF] Renard | Improvement of target detection methods by multiway filtering[END_REF], such as adaptive coherence/cosine estimator (ACE), adaptive matched filter (AMF), and spectral angle mapper (SAM). However, HSIs always suffer from spectral variations caused by noise or environment, which enlarges within-class variation and degrades the performance of detectors. It is essential to obtain high detection accuracy even targets in noisy scenes. Thus, we want to improve the target detection results using existing detectors by improving the quality of spectral signature and mining the invariant features of the spectrum.

To achieve this goal, denoising is usually done as a preprocessing step for noise removal and then target detection is performed. Traditional denoising methods, such as PCA [START_REF] Chen | Denoising and dimensionality reduction of hyperspectral imagery using wavelet packets, neighbour shrinking and principal component analysis[END_REF], models based on Wiener filter (WF) [START_REF] Zha | Noise reduction in interferograms using the wavelet packet transform and wiener filtering[END_REF], and block-matching and 3D filtering (BM3D) [START_REF] Dabov | Image denoising by sparse 3-d transform-domain collaborative filtering[END_REF], have been successfully applied in image processing.

However, the traditional denoising methods are easy to face the problem of single task [START_REF] Tian | Deep learning on image denoising: An overview[END_REF] or preserving small targets [START_REF] Lin | Small target detection improvement in hyperspectral image[END_REF].

With the development of deep learning, some methods based on deep networks have been proposed for image denoising [START_REF] Zhang | Deep spatio-spectral bayesian posterior for hyperspectral image non-iid noise removal[END_REF]. In [START_REF] Gondara | Medical image denoising using convolutional denoising autoencoders[END_REF][START_REF] Cho | Simple sparsification improves sparse denoising autoencoders in denoising highly corrupted images[END_REF], models based on denoising autoencoder (DAE) model are established for image denoising, which uses encoder to get the latent representation and then reconstructs it into the clean data through the decoder. In [START_REF] Zhang | Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising[END_REF], feed-forward denoising convolutional neural network (DnCNN) is designed for image denoising and obtained effective results. In [START_REF] Yuan | Hyperspectral image denoising employing a spatial-spectral deep residual convolutional neural network[END_REF], deep residual convolutional neural network (ResCNN) is introduced to learn a non-linear map between noisy and clean image for HSI denoising. In [START_REF] Chen | Image blind denoising with generative adversarial network based noise modeling[END_REF], GAN is used for estimating the noise distribution and constructing a paired training dataset to train CNN for image blind denoising. Compared with conventional denoising methods, deep learning-based methods are usually not limited to specific denoising tasks and the parameters are automatically updated according to the input.

In addition, the target of interest in practice is usually small compared to the background (target of non-interest). For example, when aircraft wreckage needs to be detected, its background is usually on challenging terrains, such as mountains, forests or seas [START_REF] Risnumawan | Towards an automatic aircraft wreckage detection using a monocular camera of uav[END_REF]. In these cases, the target to be detected can be treated as small target compared to backgrounds. Based on this situation, when small target detection is focused on, we want to segment the HSI to obtain the region of interests (ROIs) and narrow the detection range. In other words, the detection process is divided into two stages. The first stage is selecting ROIs according to the result of segmentation. Some models have been developed for segmentation, such as fully convolutional network (FCN) [START_REF] Long | Fully convolutional networks for semantic segmentation[END_REF], U-net [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF], SegNet [START_REF] Badrinarayanan | Segnet: A deep convolutional encoder-decoder architecture for image segmentation[END_REF]. However, segmentation ground truth is usually required when these models are optimized, which is inconsistent with our goal of unsupervised image segmentation. Therefore, we want to find a new way to segment HSIs unsupervised and select ROIs. Then, the second stage can be executed to detect whether the selected ROIs contain targets.

Spectral reconstruction for denoising and target detection

Methodology

In this section, we describe some basic knowledge which will be used in the proposed method.

n mode unfolding

We have known that an HSI can be represented as a 3D block denoted by H ∈ R H 1 ×H 2 ×H 3 , where the HSI has H 1 rows, H 2 columns, H 3 spectral bands and R is the real manifold. Then the HSI H can be flattened to be a n mode matrix H n ∈ R Hn×Mn as shown in Figure 5.1, where M n = H p × H q (p, q = n).

Target detection combined a multiscale denoising autoencoder

Target detection can be treated as a binary classification task [START_REF] Liu | Reduction of signaldependent noise from hyperspectral images for target detection[END_REF]. According to the spectral characteristics, pixels can be classified as target or background. To improve the results of target detection, DAE is introduced to reconstruct spectrums and increase spectral robustness.

Spectral reconstruction by denoising autoencoder

Each pixel in HSI can be represented as a 1D vector. If there is a noisy HSI H with size of H 1 × H 2 × H 3 . According to the 3 mode unfolding, 3D tensor data H can be unfolded to be a 2D matrix H 3 with size of

H 3 × M 3 (M 3 = H 1 × H 2 ).

Proposed model for target detection

The reconstructed spectrums removing noise can replace the original spectrums for target detection. In a DAE network, the dimensions of input layer and output layer are the same, but the dimension of hidden layer can affect the performance of reconstruction which has impact on subsequent target detection. In order to make the reconstructed spectrums contain as much information as possible, a multiscale In the following experiments, the size of batch is setting as 128 and the number of epochs is 500. We can see from Table 5.1 that results obtained by DAE_200 are better for simulated a while DAE_100 are better for simulated b with WGN when only one DAE is used. However, DAE_100 performs better for both simulated image under the condition of MPN noise. Therefore, if the spectrum is reconstructed by a single DAE, the detection results are unstable and easily affected by network structure.

Experiments on simulated data

The proposed MSDAE helps us yield best detection results than any sub-denoising autoencoder whether the image is destroyed by WGN or MPN. This is because the final reconstructed spectrum is restored from features of different scales, which provides more information for the subsequent target detection. PFA values are small.

Experiments on real-world data

Since the simulated HSIs are generated in an idealistic scene, two real-world HSIs, referred to as HSI b (Figure 8 For HSI c , the spatial size of each column of targets is 5 × 3, 3 × 3 and 1 × 1.

Small target detection

To verify whether the proposed method has a good ability in preserving the small targets, the detection maps of simulated b and HSI c with the more commenly used The results of simulated images and real-world HSIs highlight the prospects of the proposed MSDAE for small target detection in the presence of WGN interference.

Unsupervised segmentation for small target detection

Image segmentation can be treated as a pixel classification problem. In order to get rid of the segmentation ground truth, unsupervised segmentation is an effective way. K-mean clustering is one of the main unsupervised segmentation methods [START_REF] Shan | Image segmentation method based on k-mean algorithm[END_REF],

but it has limitations in mining the spectral-spatial information compared with deep learning models [START_REF] Garcia-Garcia | A review on deep learning techniques applied to semantic segmentation[END_REF]. Therefore, unsupervised segmentation based on deep learning model is investigated for improvement of small target detection.

Unsupervised segmentation

CNN has strong feature extraction abilities and has been widely used in image processing. However, label samples are unusually required to optimize the network.

In [START_REF] Kanezaki | Unsupervised image segmentation by backpropagation[END_REF][START_REF] Kim | Unsupervised learning of image segmentation based on differentiable feature clustering[END_REF], an unsupervised image segementation method is developed by assigning (1) Pixels of similar features are designed with the same label.

(2) Pixels of spatial continuity are designed with the same label.

(3) The number of unique cluster labels is designed to be large.

Criteria (1) and ( 2) are dedicated to merging neighboring pixels with similar characteristics into the same class. But there may be extreme situations where all pixels are merged into one class. In order to avoid this situation, criteria (3) is necessary.

Experimental results

In the experiment, we choose HSI a as the target data to test the effectiveness of unsupervised segmentation for small target detection.

We have known that HSIs contain hundreds of bands. 2D convolution for multichannel is used in 2D-CNN for unsupervised segmentation and the number of channels is equal to the number of bands. The corresponding network structure used for segmentation is listed in Table 5.3 and SGD is used to update weights. The maximum number of segmentation classes is set to 8. It can be find from Table 5. We can see from Figure 5.16 (b) that the target image is segmented into 5 classes.

Although the segmented classes are smaller than the actual classes, the neighboring targets of different classes can be distinguished well. Pixels of the same color are considered to have the same pixel value. According to the pixel value, a set of connected components can be obtained. Given a preset threshold, we select the regions where the number of pixels contained in the connected components is lower than this threshold as the ROIs. As shown in Figure 5.16 (c), the white part is the area that may contain small targets.

Taking target 3 in HSI a as the target to be detected and ACE as the detector, the detection maps are shown in Figure 5.17 when we directly detect the target in the entire image.

As described in Eq. (5.1), by comparing the calculated P d value with the threshold γ, we can determine whether the current pixel is the target. It can be seen from But the pixels are scattered, which is difficult to locate the target. It can be found that the detection maps in Figure 5.18 help detect and locate the target more accurately compared to the detection maps in Figure 5.17. In particular, when the γ value is set to 0.40, only a few pixels are misdetected. According to the detection map in Figure 5.18 (b), these small targets can be well located.

Experimental results prove that the first stage of unsupervised segmentation can effectively reduce the detection range and remove interfering pixels. Besides, the detection results can be further improved. Considering that in practical applications, the background area is generally much larger than the target area, unsupervised segmentation to select ROIs for target detection is very promising.

Conclusion

In this Chapter, we introduce DAE to HSIs to reconstruct spectrum for removing noise interference and improving performance of target detection. In order to further improve the target detection results, we design a MSDAE model to mine spectral characteristics as much as possible. The original spectrum is compressed, represented and expanded and then they are decoded into different reconstructed spectrums. These reconstructed spectrums from different scales features are finally merged to one spectrum for target detection, which effectively exploits the spectral characteristics and invariant features. Experimental results of simulated and realworld HSIs with WGN and MPN demonstrate the effectiveness. Compared with a single DAE and other mentioned methods, the performance of proposed MSDAE is more stable and gets higher P d values. Besides, the proposed method shows great ability in preserving small targets. The promising results show significant potential of the proposed MSDAE model for target detection. In addition, in order to further improve the detection of small targets, a two-stage method including segmentation and detection is proposed. Unsupervised segmentation is used to obtain ROIs and narrow the detection range and then target detection is performed in the ROIs. Experimental results prove that the proposed two-stage method is effective and promising for detecting small targets.

Conclusion and future works Conclusion

This dissertation aims at the study of HSI processing and its applications. Target In addition to feature extraction and classification, target detection is also studied. Target detection can be treated as a binary classification task. According to the spectral characteristics, pixels can be classified as target or background. Due to spectral variations caused by noise or environment, the within-class variation is enlarged which degrades the performance of detectors, especially when the target size is small. We design a MSDAE model to denoise and mine the invariant features.

The final spectrum input to the detector is fused by reconstructed spectrums from different scales representations, which provides more complementary information and more robust features for subsequent detection. Experiments on simulated and real-world data demonstrate that the proposed MSDAE can not only improve the target detection but also has great potential for preserving small targets in denoising.

In addition, to further improve the detection of small targets, a two-stage method including segmentation and detection is proposed. Unsupervised segmentation is used to obtain ROIs and narrow the detection range, and then target detection is performed in the ROIs. Experimental results demonstrate the effectiveness of the proposed method and have promising prospects in small target detection.

Future works

In this dissertation, we successfully apply the deep learning models to HSIs to extract features, classify and detect targets. But there are still some aspects that need to be further study. We propose here some research directions for a future extension of this work.

• In the process of feature extraction and classification, we don't separately consider and analyze the effect of noise on the results. So, the effect of different types of noise on the results and whether the noise is removed during feature extraction need to be further studied.

• In this thesis, a parameter selection method based on unique variable principle is proposed and the parameters are adjusted one by one. Taking into account the correlation between the parameters, a new algorithm needs to be developed in order to achieve the best overall performance of network.

• As described in Chapter 5, we design a MSDAE model based on DAE to reconstruct the spectrum and remove noise. The experiments show that the noise is well removed, but only spectral information is considered as input. In the future, a denoising model that consider both spectral and spatial information need to be studied.

• When a small target is a research object, its size and number of samples are much smaller compared to the background, making the processing of small targets more difficult and challenging. More work needs to be focused on small targets in the future.

• In previous research, we mainly focus on classification and detection. In the future, we want to use deep learning to analyze features for spectral unmixing.

Figure 2 :

 2 Figure 2: Cartes de classification de l'Université de Pavie obtenues par différentes méthodes: (a) Vérité terrain, (b) 2D-CNN, (c) 3D-CNN, (d) 3D-CNN-TV.

Figure 3 :

 3 Figure 3: Cartes de classification de l'Université de Pavie obtenues par différentes méthodes: (a) Vérité terrain, (b) 3D-WGAN-GP, (c) 3D-CAE avec fonctionnalités à un seul niveau, (d) 3D-CAE avec fonctionnalités multi-niveaux.

Figure 4 :Figure 5 :

 45 Figure 4: Cartes de classification du HSI a obtenues par différentes méthodes: (a) Vérité terrain, (b) FA, (c) DBN, (d) Proposé 3D-M 2 CAE.

Figure 6 :

 6 Figure 6: Schematic diagram of hyperspectral imaging.

Figure 1 . 1 ,

 11 Figure 1.1, where Ci, Pi and F represent the ith convolutional layer, the ith pooling layer, and fully connected layer respectively, and n @ feature map means there are n feature maps in current layer. Each convolution kernel corresponds to an output (feature map), and different convolution kernels can extract different features.

Figure 1 . 1 :

 11 Figure 1.1: A conventional CNN structure.

Figure 1 . 2 :

 12 Figure 1.2: 2D convolution operation.

Figure 1 . 3

 13 shows examples of max-pooling and mean-pooling.

Figure 1 . 3 :

 13 Figure 1.3: 2D pooling operation: (a) Max-pooling, (b) Mean-pooling.

Figure 1 . 4 :

 14 Figure 1.4: Hyperspectral classification based on a 2D-CNN with parameter tuning.

Figure 1 . 5 :

 15 Figure 1.5: Data sets: (a) False-color image of Pavia University. (b) Ground truth of Pavia University. (c) False-color image of Indian Pines. (d) Ground truth of Indian Pines.

  accuracy (OA), average accuracy (AA), and kappa coefficient (κ) are introduced to represent the classification results. All the values used in the experiments are average values obtained from multiple experiments.

  Odd input size (S × S) from 13 × 13 to 31 × 31 under the training ratio from 0.1 to 0.5 are tested in the experiment, while the other parameters are fixed. The experimental results are shown in Figure 1.6.

Figure 1 . 6 :

 16 Figure 1.6: OA values under different input sizes and training set ratio.

Figure 1 . 7 :

 17 Figure 1.7: Different activation functions.

Figure 1 . 8 : 1 . 3 . 3 . 6 1 . 3 . 3 . 7

 1813361337 Figure 1.8: OA values and computation times under different batch sizes.

Figure 1 . 9 : 1 . 3 . 3 . 8

 191338 Figure 1.9: OA values under different number of convolutional kernels.

  classification maps of Pavia University and Indian Pines obtained by different methods are shown in Figure 1.10 and Figure 1.11, respectively. It can be seen from Figure 1.10 that classification map of FA has the most misclassified pixels, especially a large number of pixels in the green are misclassified as pixels in the yellow area in the upper left and lower left of the HSI. Besides many pixels in the yellow are mistakenly classified as green in the central area of the HSI. In Figure 1.10 (c), the number of misclassified pixels in the upper left and lower left is greatly reduced compared with Figure 1.10 (b), but there are still lots of pixels in the central area. Overall, the classification map of Figure 1.10 (d) is the clearest and the misclassified pixels are the least.

Figure 1 .Figure 1 . 10 :

 1110 Figure 1.10: Classification maps of Pavia University under different methods: (a) Ground truth, (b) FA, (c) DBN, (d) 2D-CNN-PT.

Figure 1 . 11 :

 111 Figure 1.11: Classification maps of Indian Pines under different methods: (a) Ground truth, (b) FA, (c) DBN, (d) 2D-CNN-PT.

A

  2D-CNN-PT method is proposed in this Chapter to improve the classification results of HSIs. First, PCA is introduced to reduce the HSI dimension. Second, a 2D-CNN is constructed and the parameters of the 2D-CNN are tuned in turn according to the unique variable principle on the basis of experimental results and effectiveness. Finally, classification is performed with the optimized 2D-CNN. The experimental results on two real-world HSIs show that the proposed 2D-CNN-PT method achieves better HSI classification performance compared to the other two commonly used methods.

Figure 2

 2 Figure 2.1.

Figure 2 . 1 :

 21 Figure 2.1: A conventional 3D-CNN for hyperspectral classification.

Figure 2 . 2 :

 22 Figure 2.2: 3D convolution operation.

Figure 2 . 3 :

 23 Figure 2.3: DP algorithm for line simplification.

Figure 2 . 4 ,

 24 if curve AB is plotted according to Q(h 3 ) value calculated by the Eq. (2.3) where A and B correspond to the first and last bands of an HSI. C is the point with maximum Q(h 3 ) value on the curve, corresponding to the Dth band.Instead of setting a threshold for d (the distance from point C to the straight-line segment AB), two thresholds (T 1 and T 2 , T 1 <T 2 ) are set to limit the distance (X 1 ) between point A and point D, and the distance (X 2 ) between point D and point B. X 1 and X 2 are estimated by the difference of the corresponding band number, for

Figure 2 . 4 :

 24 Figure 2.4: The modified DP algorithm.

Figure 2 . 5 :

 25 Figure 2.5: The flowchart of proposed FMDP for band selection.

and 18 ;

 18 For Indian Pines, T 1 and T 2 are set to 16 and 32. The original spectral curves and fitted spectral curves of different classes in Pavia University and Indian Pines are depicted in Figure 2.6 and 2.7, respectively. The selected bands are corresponding to the red asterisk in Figure 2.6 (b) and 2.7 (b).

Figure 2 . 6 :

 26 Figure 2.6: Pavia University: (a) Original spectral curve, (b) Fitted spectral curve with selected bands.

Figure 2 . 7 :

 27 Figure 2.7: Indian Pines: (a) Original spectral curve, (b) Fitted spectral curve with selected bands.

Figure 2 . 8 :

 28 Figure 2.8: OA values under different number of selected bands.

  by transforming the original hyperspectral data into a matrix and then selecting the most informative and least correlative column subset. The OA values with different number of selected bands of Pavia University and Indian Pines are shown in Figure 2.9 and Figure 2.10, respectively.

Figure 2 . 9 :

 29 Figure 2.9: OA values of Pavia University based on different number of bands selected by different methods.

Figure 2 .

 2 Figure 2.10: OA values of Indian Pines based on different number of bands selected by different methods.

Figure 2 .

 2 Figure 2.11 where DR represents dimensionality reduction.

Figure 2 . 11 :

 211 Figure 2.11: Proposed hyperspectral classification based on a 3D-CNN.

  ture. In general, if two domains are different, then they may have different feature spaces or different marginal probability distributions. Given a specific domain, D = {X , P (X)}, a task (T = {Y , f (•)}) is composed of a label space Y and an objective function f (•) which can be learned from the training data. The training data consists of pairs {x i , y i }, where x i ∈ X and y i ∈ Y . For a new instance x, f (•)

  domain D S and learning task T S , a target domain D T and learning task T T , transfer learning aims to help improve the learning of the target predictive function f T (•) in D T using the knowledge in D S and T S , where D S = D T , or T S = T T .

Figure 2 .

 2 [START_REF] Lee | Fast computation of the compressive hyperspectral imaging by using alternating least squares methods[END_REF] shows the procedure of the proposed 3D-CNN-TV method, where a stadium box indicates the beginning and ending of a process, a parallelogram box denotes the process of inputting and outputting data, and a rectangular box represents a processing step.The training procedure of designed 3D-CNN-TV can be devided into three steps: First of all, 3D-CNN I is trained and optimized by sufficient training samples from the source data to obtain optimized weights. At the same time, virtual samples are generated from the original samples of target data according to Eq. (2.4). The

Figure 2 . 13 :

 213 Figure 2.13: Flow chart of proposed 3D-CNN-TV framwork.

Figure 2 . 14 :

 214 Figure 2.14: Source data sets: (a) Pavia Centre, (b) Salinas.

  class in the Pavia Centre and Salinas are randomly chosen as the training set and the remaining 30% belong to the testing set. The network structures and parameter settings of 3D-CNNs are the same as in Table 2.1. When the two 3D-CNNs are welltrained by sufficient labeled samples from Pavia Centre and Salinas, respectively, the weights of the convolutional layers and the pooling layers are transferred to the corresponding 3D-CNN-TL (3D-CNN II) model. After transfer learning, 5% of samples of each class in target data are used to fine-tune the 3D-CNN-TL networks.

Figure 2 .

 2 15 (a) shows the relationship between r and OA values for Pavia University and Figure2.15 (b)

Figure 2 . 15 :

 215 Figure 2.15: Relationship between r and OA values.

Figure 2 . 16 :

 216 Figure 2.16: Classification maps of Pavia University under different methods: (a) 3D-CNN, (b) 3D-CNN-TL, (c) 3D-CNN-VS, (d) 3D-CNN-TV.

  (b) -(d), which demonstrate that transfer learning and virtual samples have great potential in further improve the network performance.

Figure 2 . 17 :

 217 Figure 2.17: Classification maps of Indian Pines under different methods: (a) 3D-CNN, (b) 3D-CNN-TL, (c) 3D-CNN-VS, (d) 3D-CNN-TV.

  In the previous Chapter, transfer learning and virtual samples are investigated to alleviate the problem of limited labeled samples in HSIs. However, the models mentioned above are all supervised feature extraction, which means that the training process still requires the participation of labeled samples. Unsupervised feature extraction which doesn't involve labeled samples is another good way to help us get rid of labeled data. Considering the powerful data mining capabilities of deep learning models, unsupervised feature extractors based on deep learning models can be designed to fully exploit the nonlinear and spectral-spatial features of HSIs without labeled samples.

  GAN is proposed by Goodfellow et al.[START_REF] Goodfellow | Generative adversarial networks[END_REF], mainly including a generator and a discriminator. Generator can capture the probability distributions of real data x, by producing synthetic data from given some noise source z; Discriminator estimates whether the sample is real or generated. The architecture of standard GAN is shown in Figure3.1, where G represents generator and D represents discriminator.

Figure 3 . 1 :

 31 Figure 3.1: Standard GAN.

Figure 3 . 2 :

 32 Figure 3.2: Weight distribution with weights clipping.

Figure 3 . 3 :

 33 Figure 3.3: Weight distribution with gradient penalty.

Figure 3 . 4 :

 34 Figure 3.4: Framewrok of proposed dimensionality reduction method.

Figure 3 . 5 ,

 35 where deconv represents fractionalstrided convolutions (or deconvolution operation), conv means strided convolution operation, G and D represent generator and discriminator, respectively.

Figure 3 . 5 :

 35 Figure 3.5: Proposed unsupervised feature extraction method based on 3D-WGAN-GP.

  the experiment, two widely used classifiers, support vector machine (SVM) and softmax, are used to classify the features. 10% of the samples of each class are randomly chosen to train the classifier and the remaining 90% are for testing. Classification accuracy of a single land-cover class and OA values are used to assess the classification performance. The comparison results of Pavia University and Indian Pines are listed in the

Figure 3 . 7 .

 37 Figure 3.6 (g), some samples of the yellow area are incorrectly classified as green or blue. In Figure 3.6 (h), although there are still some points of misclassification, it's the closest to the ground truth image, which shows that the proposed method based on 3D-WGAN-GP is more promising in unsupervised feature extraction. The classifications maps of Indian Pines obtained by combining the proposed dimensionality reduction method and different GAN-based models are illustrated in Figure 3.7. It can be seen that the classification maps in Figure 3.7 (c) and (d) have many misclassified pixels in the upper part of the image. The classification maps of Figure 3.7 (e) -(h) have fewer misclassified pixels compared to Figure 3.7 (c) and (d),

Figure 3 . 6 :

 36 Figure 3.6: Classification maps of Pavia University under different methods: (a) False-color image, (b) Ground truth, (c) GAN-softmax, (d) GAN-SVM, (e) 3D-WGAN-softmax, (f) 3D-WGAN-SVM, (g) 3D-WGAN-GP-softmax, (h) 3D-WGAN-GP-SVM.

Figure 3 . 7 :

 37 Figure 3.7: Classification maps of Indian Pines under different methods: (a) Falsecolor image, (b) Ground truth, (c) GAN-softmax, (d) GAN-SVM, (e) 3D-WGANsoftmax, (f) 3D-WGAN-SVM, (g) 3D-WGAN-GP-softmax, (h) 3D-WGAN-GP-SVM.

  To get rid of the limitation of labeled samples, unsupervised feature extraction method based on GAN is designed in Chapter 3. However, since GAN is trained in a confrontational manner, the optimization of the GAN is more complicated and more challenging. The AE learns a representation for input data through an encoder and then decodes the representation to reconstruct data[START_REF] Vincent | Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion[END_REF][START_REF] Protopapadakis | Stacked autoencoders driven by semi-supervised learning for building extraction from near infrared remote sensing imagery[END_REF]. The AE can be optimized by minimizing the error between the reconstructed data and the input data, and no labels are involved, which is a typical unsupervised model. Because of these characteristics of AE, unsupervised feature extraction methods based on AE have been introduced in HSIs and achieved some results[START_REF] Mei | Unsupervised spatial-spectral feature learning by 3d convolutional autoencoder for hyperspectral classification[END_REF][START_REF] Tao | Unsupervised spectral-spatial feature learning with stacked sparse autoencoder for hyperspectral imagery classification[END_REF][START_REF] Zhang | Recursive autoencoders-based unsupervised feature learning for hyperspectral image classification[END_REF][START_REF] Zhou | Learning compact and discriminative stacked autoencoder for hyperspectral image classification[END_REF]. Unfortunately, when AE-based models are developed for unsupervised feature extraction, features from the single layer are usually considered, which can lose some useful information[START_REF] Lin | Feature pyramid networks for object detection[END_REF]. The image pyramid framework, which uses different-scale images to independently train multiple networks to obtain multi-level features is one of the solutions[START_REF] Edward H Adelson | Pyramid methods in image processing[END_REF], but training multiple networks increases the time and computational cost, which is unsatisfactory.The encoder of a AE is a hierarchical structure from bottom to top, and it's like a feature pyramid. The bottom layer mainly corresponds to information, such as edges, texture, and contours, and the top layer mainly corresponds to semantic information[START_REF] Jawahar | What does bert learn about the structure of language?[END_REF]. Considering the construction and training of AE is easier than GAN, an unsupervised multi-level feature extraction method based on a 3D-CAE is proposed in this Chapter. The designed 3D-CAE is composed of 3D convolutional layers and 3D deconvoluional layers, combining the advantages of CNN and AE. The 3D-CAE can not only fully mine the spectral-spatial information with 3D data as input, but it also does not require the participation of labeled samples in the training process. Besides, multi-level features are directly obtained from different encoded layers of the optimized encoder, which is more efficient when compared to training multiple networks. The full use of the detail information at the bottom layer and semantic information at the top layer can achieve complementary advantages and improve the classification results.In addition, the input size for different targets is always same while different targets often perform differently with the same input size, especially when there are small targets. In order to solve this problem and balance different targets, a novel multi-size and multi-model framework based on three-dimensional convolutional autoencoder, called 3D-M 2 CAE, is proposed. Three 3D-CAEs with different input sizes centered on the observed pixel are used to build the framework and extract features. Moreover, in order to save training time, the framework is established and trained in a progressive way with the help of transfer learning[START_REF] Sinno | A survey on transfer learning[END_REF][START_REF] Weiss | A survey of transfer learning[END_REF][START_REF] Yang | Learning and transferring deep joint spectral-spatial features for hyperspectral classification[END_REF]. The weights of the middle layers of the latter 3D-CAE are transferred from the former optimized 3D-CAE, which speeds up and facilitates network training. Benefiting from this training method, the features of the same target from different sizes are obtained in a more efficient way.

Figure 4 .

 4 1 consists of fully connected layers, and it unusually contains an input layer, a hidden layer and an output layer, which constitute an encoder and a decoder. If there is an input I ∈ R I 1 and it's first mapped to a latent representation Y by encoder during the training procedure.

Figure 4 . 1 :

 41 Figure 4.1: Conventional AE architecture.

Figure 4 . 2 :

 42 Figure 4.2: CAE architecture.

1

 1 Details of the proposed frameworkConsidering convolution-based operation has high flexibility in processing multidimensional data and has a strong ability in feature extraction, a 3D-CAE is introduced to extract features unsupervised. In order to better preserve the spatial and spectral characteristics of HSIs, the designed 3D-CAE is established by fully 3D convolutional layers and 3D deconvolutional layers (see Figure4.3), where Conv-n and Deconv-n mean the nth convolutional layer and the nth deconvolutional layer, respectively. As in the previous two chapters, a 3D block centered on the current observed pixel is used as the input of 3D-CAE to learn its invariant characteristics.The proposed framework based on 3D-CAE for multi-level feature learning is mainly divided into three steps: Firstly, a 3D-CAE is constructed. The 3D-CAE is designed as a symmetrical structure composed of 3D convolutional layers and deconvolutional layers, as shown in Figure4.3. The size of feature map is gradually reduced, and the number of convolution kernels is gradually increased. The size of output is the same as the size of input.

Figure 4 . 3 :

 43 Figure 4.3: Proposed framework for multi-level feature extraction.

  results based on different single-level features are considered for comparison to better evaluate the effectiveness of the multi-level features. The better the classification result, the better the corresponding features. In the experiment, SVM is selected as the classifier. OA, AA, and κ values are introduced to evaluate the classification results. For each class in data sets, approximately 10% is used to train the classifier and the rest is used for testing. At first, single-level features and multi-level features from three encoded layers are compared under the condition of input size being 13 × 13 × 10. Since the number of encoded layers used to form multi-level features may also affect the classification results, we will study the influence of this parameter on the results later. As shown in Figure 4.4, the feature map size in top three layers (the third, fourth, and fifth layers) of encoder is 5 × 5 × 5, 3 × 3 × 3, and 1 × 1 × 1, respectively. Therefore, the filter size of max-pooling in the third and fourth layers is correspondingly set as 5 × 5 × 5 and 3 × 3 × 3. The feature map size of the fifth layer is already 1 × 1 × 1, so we directly flatten the feature maps into a 1D vector. After max-pooling operation, three feature vectors are obtained with sizes of 1 × 32, 1 × 64, and 1 × 128. The three feature vectors are concatenated to obtain a final feature vector with the size being 1 × 224. These features are fed into the classifier, and the prediction results can be obtained, where Prediction I represents the predicted classification results based on the final multi-level features with a size of 1 × 224, Prediction II represents the results of single-level features 1 × 128 from the fifth layer, Prediction III corresponds to the single-level features with a size of 1 × 64, and Prediction IV corresponds to the single-level features with a size of 1 × 32.Tables 4.2 and 4.3 list the classification results that are based on different features of Pavia University and Indian Pines, respectively.

Figure 4 . 4 :

 44 Figure 4.4: Proposed framework for multi-level feature extraction.

  3 are obtained under the condition that the input size is 13 × 13 × 10. When the input size changes from 13 × 13 × 10 to 19 × 19 × 10, the classification accuracy based on single-level features from top encoded layer (Prediction II) and multi-level features (Prediction I) are compared. The comparison results of Pavia University and Indian Pines are depicted in Figure 4.5 and Figure 4.6, respectively.

Figure 4 . 5 :

 45 Figure 4.5: Classification accuracy of Pavia University under different input sizes.

Figure 4 . 6 :

 46 Figure 4.6: Classification accuracy of Indian Pines under different input sizes.

  In the previous experiments, the multi-level features are obtained by concatenating the information of three encoded layers. In order to observe the impact of the number of encoded layers on the classification results, the multi-level features obtained from two, three, and four encoded layers are compared with input size being 17 × 17 × 10.

Figure 4 .

 4 Figure 4.8 is the comparison results of Indian Pines.

Figure 4 . 7 :

 47 Figure 4.7: Classification accuracy of Pavia University based on multi-level features with different numbers of encoded layers.

Figure 4 . 8 : 3 (

 483 Figure 4.8: Classification accuracy of Indian Pines based on multi-level features with different numbers of encoded layers.

For better visual

  comparison, classification maps of Pavia University and Indian Pines obtained by different methods are depicted in Figure 4.9 and Figure 4.10, respectively.

Figure 4 . 9 :

 49 Figure 4.9: Pavia University: (a) Composite image, (b) Ground truth, (c) FA, (d) DBN, (e) 2D-CNN, (f) SAE, (g) 3D-CAE (single-level features), and (h) 3D-CAE (multi-level features).

Figure 4 . 10 :

 410 Figure 4.10: Indian Pines: (a) Composite image, (b) Ground truth, (c) FA, (d) DBN, (e) 2D-CNN, (f) SAE, (g) 3D-CAE (single-level features), and (h) 3D-CAE (multi-level features).

Figure 4 . 10 (

 410 c), (d) and (f), especially the upper left corner area. The classification maps in

Figure 4 .

 4 Figure 4.10 (e) and (g) are better. Among all of the classification maps, Figure 4.10 (h) has the least number of misclassified pixels, which demonstrates the effectiveness of the proposed method.

Figure 4 . 11 :

 411 Figure 4.11: Proposed 3D-M 2 CAE framework for unsupervised feature extraction.

Chapter 1 . 3 × 3 × 3 3 × 3 × 3 3 × 3 × 9 - 5 × 5 × 3 3 × 3 × 3 3 × 3 × 3 3 × 3 ×

 1333333955333333 Both two data sets are normalized to [0, 1] in the experiment. HSI a data set contains 316 × 216 pixels covering 148 bands, which divided into 7 land-cover classes. In the subsequent experiment, 10% samples of each class are used for training the classifier and the remaining samples for testing. The details of Table 4.7: Encoder structures of three 3D-CAEs in proposed 3D-M 2 CAE framework. 3D-CAE II 11 × 11 × 3D-CAE III 15 × 15 × 11 5 × 5 × 3 5 × 5 × 3 3 × 3 × 3 3 × 3 × 3 3 × 3 × example, after convolving input II with the kernel of 5 × 5 × 3, the feature map size is (11 -5 + 1) × (11 -5 + 1) × (9 -3 + 1) = 7 × 7 × 7, which is the same as the input size of the 3D-CAE I. Besides, ReLU is utilized to introduce nonlinearity in all convolutional layers and deconvolutional layers except the last layer that uses sigmoid activation. Batch normalization is introduced to normalize the features and Adam is selected for optimizing the network parameters. After the experimental test, the number of training epochs of the three 3D-CAEs are set to 20, 5 and 5 respectively, with batch size being 512. Through progressive training, three feature vectors from different input sizes are finally obtained. In order to make full use of the extracted features, these three feature vectors are concatenated into one feature vector for subsequent classification.

  HSI a ), which are more challenging in HSI processing. We gradually increase the input size and the OA values of these three small targets based on features obtained by 3D-CAE I, 3D-CAE II and 3D-CAE III are plotted in Figure 4.13 -Figure 4.15. We can find from Figure 4.13 that when 3D-CAE I is used for feature extraction, the OA values of Target 1 is slightly affected by input size, the OA values of Target 2 increase as the input size increases and then the OA values decrease, and the OA values of Target 3 show an upward trend with the input size in the range of 7 × 7 × 7 to 13 × 13 × 7. Overall, 7 × 7 × 7 may be more appropriate for Target 1, while 9 × 9 × 7 is more appropriate for Target 2 and 13 × 13 × 7 is more appropriate for Target 3.

Figure 4 .

 4 Figure 4.13: OA values of three small targets based on 3D-CAE I under different input sizes.

Figure 4 .

 4 Figure 4.14: OA values of three small targets based on 3D-CAE II under different input sizes.

Figure 4 .Figure 4 .

 44 Figure 4.15: OA values of three small targets based on 3D-CAE III under different input sizes.

Figure 4 .

 4 Figure 4.16: OA values of Target 1 based on features obtained from different networks.

Figure 4 .

 4 Figure 4.17: OA values of Target 2 based on features obtained from different networks.

  that the OA values based on 3D-CAE I and 3D-CAE II rise to the peak and then decreases. When the corresponding H × H × H dr being 11 × 11 × 7, the OA values based on 3D-CAE I and 3D-CAE II are higher, while 9 × 9 × 7 helps 3D-CAE III get the highest OA value. When proposed 3D-M 2 CAE is used for feature extraction, we can find the OA values are greatly improved compared with single 3D-CAEs, especially when the corresponding H × H × H dr is small.

Figure 4 .

 4 Figure 4.18: OA values of Target 3 based on features obtained from different networks.

  Pavia University data set In the previous experiments, we mainly focus on small targets. Next, we treat Pavia University as target data to further verify the effectiveness and generalization of the proposed 3D-M 2 CAE framework. The OA, AA and κ values of Pavia University are given in Figure 4.19 -Figure 4.21.

Figure 4 .

 4 Figure 4.19: OA values of Pavia University based on features obtained from different networks.

7 × 7 × 7 .

 77 The low AA value reflects the large difference in classification accuracy of different classes. The proposed 3D-M 2 CAE can not only help us further improve the classification accuracy but also narrow the difference between the results of different classes.

Figure 4 .

 4 Figure 4.21 shows the κ values of different models under different sizes. It can be seen that the κ values of proposed 3D-M 2 CAE far exceed those of 3D-CAEs, which

Figure 4 .

 4 Figure 4.20: AA values of Pavia University based on features obtained from different networks.

Figure 4 . 4 . 4 . 2 . 5

 44425 Figure 4.21: κ values of Pavia University based on features obtained from different networks.

Figure 4 .

 4 Figure 4.22 and Figure 4.23.

Figure 4 . 22 :

 422 Figure 4.22: Classification maps of HSI a obtained by different methods: (a) FA, (b) SAE, (c) DBN, (d) 2D-CNN, (e) 3D-CAE I, (f) 3D-CAE II, (g) 3D-CAE III, (h) Proposed 3D-M 2 CAE.

Figure 4 .

 4 Figure 4.22 (a). The classification results in Figure 4.22 (b) -(d) are improved compared with Figure 4.22 (a), but there are still a lot of yellow pixels are not correctly classified. When 3D-CAE is used to obtain features, the corresponding classification maps are clearer. Although the OA values obtained by single 3D-CAE and 3D-M 2 CAE are not very much different, the classification results of small targets are quite different. This is because the given OA values are for the whole image, and the number of samples for small targets is much smaller than the total number of samples. The classification results of small targets have little effect on the OA value compared with other targets. From the classification map, there are few misclassified pixels in red, blue and yellow region in Figure 4.22 (h), which demonstrates that the proposed method has great potential in feature extraction and classification of small targets.

Figure 4 . 23 :

 423 Figure 4.23: Classification maps of Pavia University obtained by different methods: (a) FA, (b) SAE (c) DBN, (d) 2D-CNN, (e) 3D-CAE I, (f) 3D-CAE II, (g) 3D-CAE III, (h) Proposed 3D-M 2 CAE.

  itations of labeled samples and further improve classification accuracy. Considering that the convolution-based operations can handle multi-dimensional data flexibly and has a strong ability in feature extraction, the designed 3D-CAEs are stacked by fully 3D convolutional and 3D deconvolutional layers, which helps us exploit the spectral-spatial characteristics among hyperspectral data. In the first framework, multi-level features are proposed to contain detail information and semantic information at the same time. The proposed multi-level features are directly obtained from different encoded layers of the optimized encoder, which helps us to make full use of the well-trained network and further improve feature quality. Experimental results of Pavia University and Indian Pines show that single-level features from the top encoded layer perform better when compared to single-level features from other encoded layers. The performance of the proposed multi-level features exceeds any single-level features under different input sizes. The OA, AA, and κ values based on proposed multi-level features increased by about 2% to 3% for Pavia University and 2% to 5% for Indian Pines compared with single-level features from top encoded layer. Besides, we find that the number of layers used to form multi-level features also affects the feature performance. The more encoded layers are selected, the larger the dimension of the multi-level features. Our goal is to use low-dimensional features to obtain high accuracy. Based on the experimental results, we choose three encoded layers for multi-layer features in the experiment. Moreover, the proposed multi-level features are compared with the features obtained by supervised DBN and 2D-CNN, as well as unsupervised FA and SAE. The experimental results show that the proposed method outperforms the considered methods. The proposed multi-level features help us to obtain the highest classification accuracy, which demonstrates that they have huge potential in hyperspectral classification. In addition, another framework named 3D-M 2 CAE is proposed to balance different targets and improve classification results of small targets. The proposed 3D-M 2 CAE consists of three 3D-CAEs with different input size. The input size and network layers of the three 3D-CAEs are gradually increasing. Since the three inputs of the same target have high relevance and similarity, the weights of the middle layers of the second and third 3D-CAEs are transferred from the previously optimized network. Benefiting from the progressive training methodology and transfer learning, we can facilitate the training and save time of 3D-M 2 CAE. In addition, features from different input sizes can be obtained during the progressive training,which helps us improve the feature robustness to size variations and provide more information to better analyze targets. Since small targets are more sensitive to the input size and have fewer samples, the analysis of small target is more challenging in HSIs. In the experiment, we first focus on small targets to observe the performance of the proposed 3D-M 2 CAE. Experimental results of HSI a show that the proposed method can greatly improve the classification results of small targets compared with single-input model. Then, the experiment is executed on Pavia University data set.

Figure 5 . 1 : 5 . 2 . 1 . 2

 515212 Figure 5.1: A DAE architecture.

Figure 5 . 2 :

 52 Figure 5.2: A DAE architecture.

Each row vector in H 3

 3 corresponding to a spectral curve can be used as the input of DAE network as shown in Figure5.3. By minimizing the error between output and input, the DAE is trained and fine-tuned. After the network is well-trained, the reconstructed spectrum that contains the useful information as much as possible can be obtained. Due to the DAE tries to recover the original one from the corrupted one, the spectrum reconstructed by DAE can remove noise while retaining the invariant features.

Figure 5 . 3 :

 53 Figure 5.3: Spectral reconstruction with DAE.

  denoising autoencoder (MSDAE) is designed for improvements of target detection in HSIs. The input spectrum is encoded to different scales to get a set of representations of input, and then they are decoded to fuse into the final reconstructed spectrum. The flowchart of proposed MSDAE model for target detection is depicted in Figure 5.4.

Figure 5 . 4 :

 54 Figure 5.4: Target detection with the proposed MSDAE.

Figure 5 . 5 :Figure 5 . 6 :

 5556 Figure 5.5: Simulated scenes: (a) simulated a (the 80 th band), (b) simulated b (the 80 th band), (c) spectral signatures.

Figure 5 .

 5 Figure 5.6 reflects that reconstructed spectrums have largely removed noise and are similar to the corresponding original spectrum. When the image is disturbed by WGN, DAE_200 performs better than the other two single DAEs. When the image is disturbed by MPN, DAE_100 and DAE_200 perform better than DAE_50. In general, the spectrum represented by MSDAE under two kinds of noise is closer to the original signature than the spectrum reconstructed any single DAE. In the experiment, we range ζ from 10 dB to 80 dB to compare the performance of the proposed MSDAE with a single DAE. We find when the SNR exceeds 30 dB for simulated images, the P d values are always one. Other relevant P d values under PFA = 10 -3 of Target 2 in two simulated HSIs are given in Table 5.1.

  curves of Target 2 for simulated a and simulated b denoised by different methods are depicted in Figure 5.7 and in Figure 5.8, respectively.

Figure 5 . 7 :Figure 5 . 8 :

 5758 Figure 5.7: ROC curves of simulated a under different noises with SNR being 20 dB: (a) WGN, (b) MPN.

  (a)) and HSI c (Figure 8 (b)), taken from the entire HYDICE are considered in this part to test the performance of the proposed model. Both HSI b and HSI c have 148 spectral bands with spatial size of HSI b being 141 × 126 and HSI c being 140 × 140. For HSI b , the target size is 12 × 12.

Figure 5 . 9 :Figure 5 Figure 5 . 10 :

 595510 Figure 5.9: Real-world scenes: (a) HSI b (the 60 th band), (b) HSI c (the 60 th band), (c) spectral signatures.

Figure 5 . 11 :

 511 Figure 5.11: ROC curves of HSI b under different noises with SNR being 20 dB: (a) WGN, (b) MPN.

Figure 5 . 12 :

 512 Figure 5.12: ROC curves of HSI c under different noises with SNR being 20 dB: (a) WGN, (b) MPN.

Figure 5 .

 5 Figure 5.12 shows the performance of the proposed MSDAE for HSI c is far superior to other methods, which proves the potential of the proposed method in target detection.

  WGN and SNR being 20dB are compared. All the detection maps are obtained under PFA = 10 -3 after denoising or reconstructing by different models, which are shown in Figure 5.13 and Figure 5.14, respectively.

Figure 5 . 13 :From Figure 5 .

 5135 Figure 5.13: Detection maps of simulated b : (a) ground truth, (b) denoised by WF, (c) reconstructed by BM3D, (d) reconstructed by DnCNN, (e) reconstructed by MSDAE.

Figure 5 . 14 :

 514 Figure 5.14: Detection maps of HSI c : (a) ground truth, (b) denoised by WF, (c) denoised by BM3D, (d) denoised by DnCNN, (e) reconstructed by MSDAE.

  labels to pixels based on features. Label prediction and network parameter learning are alternately iteratively trained and optimized. The illustration of unsupervised segmentation based on CNN is shown in Figure 5.15.

Figure 5 . 15 :

 515 Figure 5.15: Unsupervised segmentation based on 2D-CNN.

  3 that the length and width of the output have not changed and are the same as the size of the input image. After 100 epochs of training, the segmentation map is shown in Figure 5.16 (b).

Figure 5 . 16 :

 516 Figure 5.16: HSI c : (a) Composite image, (b) Segmentation map, (c) Small target regions of interest.

Figure 5 . 17 :

 517 Figure 5.17: Detection maps of HSI a without unsupervised segmentation: (a) Ground truth. (b) Detection map with γ=0.45, (c) Detection map with γ=0.50, (d) Detection map with γ=0.55.

Figure 5 .

 5 Figure 5.17, when the γ value is 0.45 and 0.50, many pixels are mistakenly detected as targets. When the γ value is 0.55, most of the detected pixels are target pixels.

Figure 5 .

 5 Figure 5.18 shows the detection maps of Target 3 in HSI a after selecting ROIs with unsupervised segmentation.

Figure 5 . 18 :

 518 Figure 5.18: Detection maps of HSI a with unsupervised segmentation: (a) Ground truth. (b) Detection map with γ=0.40, (c) Detection map with γ=0.45, (d) Detection map with γ=0.5.

  classification and target detection are key techniques for hyperspectral applications, and feature extraction is the most significant step of classification and detection. Therefore, feature extraction, classification and detection are the main research topics investigated. Since deep learning has strong capabilities in data mining and feature extraction, we are devoted to processing HSIs with deep learning models. Considering that HSIs are 3D tensor data and CNN can handle multi-dimensional data flexibly, hyperspectral classification models based on 2D-CNN and 3D-CNN are studied. Due to the model performance is greatly influenced by the parameter settings, a parameter tuning method (2D-CNN-PT) based on 2D-CNN with unique variable principle is proposed for hyperspectral classification, and the optimal parameters are selected mainly based on classification results, which helps us further improve network performance. The parameters of the 3D-CNN or other models can also be selected as proposed 2D-CNN-PT. Experimental results on real-world HSIs demonstrate that appropriate parameter settings can help to obtain better classification results. Besides, 3D-CNN shows greater potential in fully exploiting the spectral-spatial information and helps us obtain higher classification accuracy compared to 2D-CNN. However, there are more parameters in 3D-CNN and the optimization of CNN is supervised, which means a large number of labeled samples are required to guarantte the network performance. Unfortunately, the labeled samples are limited in HSIs. To get rid of the limitation of labeled samples, a 3D-CNN-TV method based on 3D-CNNs combined with transfer learning and virtual samples is proposed. Transfer learning helps to apply knowledge learned in source data with sufficient labeled samples to novel target data. The weights of the 3D-CNN with target data are transferred from another 3D-CNN which has same network struc-ture as the previous 3D-CNN and is optimized by source data, so that the 3D-CNN corresponing to target data has a strong feature extraction ability at the beginning. Virtual samples generated from the original samples can greatly increase the number of labeled samples. The introduction of transfer learning and virtual samples effectively alleviates the problem of insufficient labeled samples. Experimental results on real-world HSIs show either transfer learning or virtual samples can effectively alleviates the problem of insufficient labeled samples, and the combination of transfer learning and virtual samples helps to yield highest classification results. Unsupervised learning is a type of algorithm that learns patterns from unlabeled data. If we can learn the features in the hyperspectral data in an unsupervised way, the problem of insufficient labeled samples can be solved. GAN is trained in an adversarial way requiring no labeled samples and it has been one of the most promising unsupervised learning representatives. AE can be optimized by minimizing the error between the reconstructed data and the input data, and no labels are involved. GAN and AE are typical unsupervised training networks. Therefore, unsupervised feature extractors based on GAN and AE are investigated in this thesis. Firstly, unsupervised feature extraction methods based on 3D-WGAN-GP is proposed. With the help of transfer learning, the discriminator of the optimized 3D-GAN-GP is transferred as the feature extractor. Then, a multi-level feature extraction method based on 3D-CAE is proposed. The proposed multi-level features are directly obtained from different encoded layers of the optimized encoder, which is more efficient when compared to training multiple networks and makes full use of the information at the bottom and top layers. Finally, a 3D-M 2 CAE framework is designed to balance different targets and improve classification results of small targets. Three 3D-CAEs with different input sizes centered on the observed pixel are used to build the framework and extract features. The framework is established and trained in a progressive way with the help of transfer learning to save training time. Benefiting from this training method, the features of the same target from different sizes are obtained in a more efficient way. Features from the same target and different sizes can be obtained, which can greatly improve the robustness of features to size changes. The experimental results verify the effectiveness of three proposed unsupervised feature extractors and show great application prospects without labeled samples.

4. 2

 2 The classification accuracy of Pavia University based on different features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Classification accuracy of Indian Pines based on different features. . . 4.4 Classification accuracy of Pavia University based on different feature extraction methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Classification accuracy of Indian Pines based on different feature extraction methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.6 Land-cover classes and color coding in HSI a . . . . . . . . . . . . . . . 4.7 Encoder structures of three 3D-CAEs in proposed 3D-M 2 CAE framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 The P d values of different models for simulated a and simulated b . . . 5.2 The P d values of different models for HSI b and HSI c . . . . . . . . . . 5.3 Network structures of 2D-CNN. . . . . . . . . . . . . . . . . . . . . .

  Cependant, le manque d'échantillons étiquetés de HSI limite les performances des 3D-CNN. Pour résoudre ce problème, une méthode améliorée basée sur des 3D-CNNs combinés à un transfert d'apprentissage ou à des échantillons virtuels est proposée. Les poids dans les couches inférieures du réseau cible En raison des variations spectrales causées par le bruit ou l'environnement, la variation intra-classe est importante, ce qui dégrade les performances des détecteurs, en particulier lorsque la taille de la cible est petite. Compte tenu de la grande capac-

	sont transférés d'un autre 3D-CNN bien entrainé sur une HSI (données sources) ité d'extraction de caractéristiques et de représentation des modèles d'apprentissage
	avec suffisamment d'échantillons et avec les mêmes caractéristiques spatiales que en profondeur, le DAE est introduit pour réduire le bruit et exploiter les informa-
	les données cibles. De plus, des échantillons virtuels générés à partir des échan-tions invariantes pour la détection de cibles. De plus, pour extraire entièrement
	tillons originaux des données cibles sont utilisés pour augmenter encore le nombre les caractéristiques des spectres d'origine, un modèle MSDAE est conçu pour in-
	d'échantillons. Le transfert d'apprentissage ou les échantillons virtuels peuvent at-corporer des informations complémentaires dans le spectre final en fusionnant les
	ténuer le problème posé par la limitation des échantillons étiquetés rencontré dans spectres reconstruits à partir de représentations à différentes échelles, ce qui four-
	de nombreuses situations. nit des informations plus complexes et des caractéristiques plus robustes pour une
	identification spectrale.
	• Les HSI sont représentées par des tableaux ou tenseurs 3D et comme les données
	multidimensionnelles peuvent être directement appliquées à l'entrée de 3D-CNN.

Cette thèse est consacrée à l'utilisation de méthodes basées sur les tenseurs, en se concentrant sur les méthodes d'apprentissage profond, pour traiter les HSI. Les principaux apports se résument comme suit :

• Considérant que CNN a un grand potentiel dans l'extraction de caractéristiques et peut bien préserver la structure spatiale de la cible, 2D-CNN est introduit pour extraire les caractéristiques parmi les données pour la classification hyperspectrale. Cependant, les performances de CNN sont toujours influencées par les réglages des paramètres. Pour obtenir les paramètres optimaux pour la classification HSI, nous proposons une méthode de classification basée sur un 2D-CNN avec réglage des paramètres (2D-CNN-PT). Les paramètres du réseau sont réglés à leur tour selon le principe de la variable unique et un ensemble de paramètres optimaux peut enfin être obtenu pour améliorer les performances du réseau. Par conséquent, un 3D-CNN est introduit pour exploiter pleinement les informations spectrales et spatiales des données hyperspectrales et améliorer la précision de la classification. • Le GAN est entraîné de manière antagoniste ne nécessitant aucun échantillon étiqueté, ce qui est un modèle d'entraînement non supervisé. Pour pallier la limitation des échantillons étiquetés, un extracteur de caractéristiques non supervisé est conçu sur la base d'un transfert d'apprentissage et d'un 3D-WGAN-GP. Le 3D-WGAN-GP ajoute une pénalité de gradient pour appliquer la contrainte de Lipschitz, ce qui peut résoudre le problème de l'explosion et de la disparition des gradients. De plus, compte tenu des caractéristiques spectrales et spatiales des HSI, le réseau est conçu sous forme 3D, ce qui permet d'exploiter pleinement les informations contenues dans les données hyperspectrales. • L'AE peut être optimisé en minimisant l'erreur entre les données reconstruites et les données d'entrée sans utilisation de données étiquetées, c'est un modèle non supervisé typique. Pour apprendre simultanément les informations spectrales et spatiales des cibles et ne pas dépendre des échantillons étiquetés qui sont souvent limités, un nouveau réseau d'extraction de caractéristiques à plusieurs niveaux non supervisé basé sur un 3D-CAE est proposé. En outre, l'encodeur d'un 3D-CAE est une structure hiérarchique du bas en haut, et les caractéristiques extraites sont en forme d'une pyramide. Pour exploiter pleinement les avantages du réseau optimisé, les caractéristiques multi-niveaux obtenues à partir des couches codées avec différentes échelles et résolutions sont proposées, ce qui est plus efficace que d'utiliser plusieurs réseaux pour les obtenir. De plus, un réseau multi-taille et multi-modèle basé sur 3D-CAE, appelé 3D-M 2 CAE, est proposé pour l'extraction et la classification de petites cibles. La conception et l'optimisation du réseau reposent sur une croissance progressive et un transfert d'apprentissage. Bénéficiant de cette méthode d'entraînement, les caractéristiques d'une même cible sont obtenues de manière efficace aux différentes tailles et le réseau proposé présente une grande robustesse vis-à-vis des différentes cibles.

•

Table 1

 1 Indiana, USA. There are 224 spectral bands ranging from 0.4 to 2.5 µm. The number of spectral bands is reduced to 200 after removing bands covering the region of water absorption. The Indian Pines scene contains two-thirds agriculture, and one-third forest or other natural perennial vegetation. As shown in Figure1.5 (d), there are 16 land-cover classes and black represents the unlabeled area. The details of land-cover classes and number of samples are listed in Table1.2.

	.1.

Table 1 .

 1 2: Details of land-cover classes in Indian Pines data set.

	Class No.	Color coding	Class name	Number of samples
	1		Alfalfa	46
	2		Corn-notil	1428
	3		Corn-min	830
	4		Corn	237
	5		Grass-pasture	483
	6		Grass-trees	730
	7		Grass-pasture-mowed	28
	8		Hay-windrowed	478
	9		Oats	20
	10		Soybean-notill	972
	11		Soybean-mintill	2455
	12		Soybean-clean	593
	13		Wheat	205
	14		Woods	1265
	15		Buildings-grass-trees	386
	16

Table 1 .

 1

	Layer	C1	P1	C2	P2	F
	Parameter	4 × 4 × 16	2 × 2	3 × 3 × 32	2 × 2	128
	Activation function	ReLU	Max-pooling	ReLU	Max-pooling -

3: Initial network structure of 2D-CNN.

Table 1 .

 1

	OA (%) Net
		Net1	Net 2	Net 3	Net 4	Net 5
	Data set
	Pavia University	92.88	92.86	93.45	93.39	93.27
	Indian Pines	90.46	88.96	89.76	89.65	89.48
	1.3.3.

5: OA values of Pavia University and Indian Pines under different networks.

3 Number of units in the fully connected layer

  The OA values under different number of units in fully connected layer are shown in Table1.6, where n f represents the number of units.

For the classification model, the fully connected layer plays the role of converting features into 1D vector form. The larger the number of units in the fully connected layer, the larger the feature dimension and the greater number of weights that needs to be trained.

Table 1 .

 1 6: OA values of Pavia University and Indian Pines under different number of units in fully connected layer.

	OA (%) n f
		32	64	128	256	512	1024	2048
	Data set
	Pavia University	92.74	93.27	93.45	93.49	93.50	93.95	93.16
	Indian Pines	88.78	89.39	89.76	89.72	89.55	90.55	90.82

Table 1 .

 1

	OA (%)	Optimizer
			SGD	Adagrad Adadelta RMSprop Adam
	Data set
	Pavia University	92.87	93.03	93.45	93.95	94.14
	Indian Pines	86.58	87.38	90.45	90.53	90.61

7: OA values of Pavia University and Indian Pines under different optimizers.

Table 1 . 8

 18

. It can be found from Table 1.8 that OA values slowly increase as the number of epochs increases, but the training time increases greatly. Considering OA values and training time, the number of epochs is set to 100 for both two data sets.

Table 1 .

 1 8: OA values of Pavia University and Indian Pines under different number of epochs.

	OA (%) n e
		100	200	300	400	500	600
	Data set
	Pavia University	95.03	95.10	95.14	95.14	95.36	95.37
	Indian Pines	90.88	90.78	90.84	90.83	90.85	90.87

Table 2 .

 2

		1: Network structure of 3D-CNN.
	Network layer Convolutional layer ReLU Pooling laler Dropout
	1	4 × 4 × 3 × 16	yes	2 × 2 × 1	-
	2	5 × 5 × 3 × 32	yes	2 × 2 × 1	0.2
	3	4 × 4 × 3 × 64	yes	-	0.2

  102 spectral bands. Nine land-cover classes are contained in this data set and the details are listed in Table 2.2.

	Table 2.2: Comparison of land-cover classes and number of samples in Pavia Centre
	and Pavia University.
		Pavia Centre	Pavia University
	Class No.	Name	Number	Name	Number
	1	Water	65971	Asphalt	6631
	2	Trees	7598	Meadows	18649
	3	Asphalt	3090	Gravel	2099
	4	Bricks	2685	Trees	3064
	5	Bitumen	6584	Metal sheets	1345
	6	Tiles	9248	Bare soil	5029
	7	Shadows	7287	Bitumen	1330
	8	Meadows	42826	Bricks	3682
	9	Bare Soil	2863	Shadows	947

It can be found from Table

2

.2 that Pavia Centre and Pavia University have seven common classes, such as Trees, Asphalt, Bricks, Bitumen, etc.

Salinas shown in

Figure 2.14 (b) is collected by the 224-band AVIRIS sensor over Salinas Valley, California, the same as the sensor that obtained Indian Pines. The scene covers comprises 512 lines by 217 samples, including 16 land-cover classes. As with Indian Pines scene, 20 water absorption bands is discarded. It can be observed from Table

2

.3 that although the land-over classes in Indian Pines are different from Salinas, the two data sets mainly contain agriculture, forest and vegetation.

Table 2 .

 2 3: Comparison of land-cover classes and number of samples in Indian Pines and Salinas.

		Indian Pines		Salinas
	Class No.	Name	Number	Name	Number
	1	Weeds1	2009	Alfalfa	46
	2	Weeds2	3726	Corn-notil	1428
	3	Fallow	1976	Corn-min	830
	4	Fallow-rough-plow	1394	Corn	237
	5	Fallow-smooth	2678	Grass-pasture	483
	6	Stubble	3959	Grass-trees	730
	7	Celery	3579	Grass-pasture-mowed	28
	8	Grapes-untrained	11271	Hay-windrowed	478
	9	Soil-vinyard-develop	6203	Oats	20
	10	Corn	3278	Soybean-notill	972
	11	Lettuce-4wk	1068	Soybean-mintill	2455
	12	Lettuce-5wk	1927	Soybean-clean	593
	13	Lettuce-6wk	916	Wheat	205
	14	Lettuce-7wk	1070	Woods	1265
	15	Vinyard-untrained	7268	Buildings-grass-trees	386
	16	Vinyard_vertical_trellis	1807	Stone-stel-towers	93

Table 2 .

 2 4: OA values under different noise variances of the virtual samples.As presented in Table2.4, the OA value of Pavia University is relatively high when the value of σ 2 is less than 0.001. Besides, the highest OA values of Pavia University and Indian Pines are both obtained with σ 2 being 0.001, which means that the virtual samples are more similar to the original samples at this point.

	σ 2	0.00001	0.0001	0.001	0.01	0.1	1
	OA 1	98.31	98.52	99.15	98.15	98.55	98.22
	OA 2	96.60	96.64	97.75	96.73	96.30	95.81

2 , the resulting OA values are shown in Table 2.4, where OA 1 represents the OA values of Pavia University and OA 2 represents the OA values of Indian Pines.

Table 3 .

 3

		1: Network structure of 3D-CNN.
	Network layer Convolutional layer ReLU Pooling laler	Output
	1	1 × 1 × 83 × 1	yes	1 × 1 × 2	27 × 27 × 10 × 1
	2	4 × 4 × 3 × 16	yes	2 × 2 × 2	12 × 12 × 4 × 16
	3	5 × 5 × 3 × 32	yes	2 × 2 × 2	4 × 4 × 1 × 32
	4	4 × 4 × 1 × 64	yes	-	1 × 1 × 1 × 64
	corresponding feature maps is 27×27×10. It can be found that the height and width
	of the input data are not changed after the first layer of convolution and pooling
	operations, and only the depth (spectral dimension) is reduced. The 3D-CNN is
	trained by source data which is assumed to have sufficient labeled samples. During
	the training process, the parameters of the network are continuously optimized.

The input size S × S × H 3 is set to 27 × 27 × 102 at the beginning and the stride is 1. The output size of feature map in the first convolutional layer is (27 -1 + 1) × (27 -1 + 1) × (102 -83 + 1) = 27 × 27 × 20. After pooling operation, the size of the

Table 3 .

 3

				2: Architectures of the 3D-WGAN-GP.
	Net Layer	Conv	Af	Net Layer	Deconv	Af
		1	4 × 4 × 3 × 16 leakyReLU		1	5 × 5 × 3 × 128 ReLU
	D	2 3	5 × 5 × 3 × 32 leakyReLU 5 × 5 × 3 × 64 leakyReLU	G	2 3	5 × 5 × 3 × 64 ReLU 5 × 5 × 3 × 32 ReLU
		4	5 × 5 × 3 × 128 leakyReLU		4	4 × 4 × 3 × 16 ReLU

Table 3 .

 3 3 and Table 3.4, respectively.

Table 3 .

 3 3: Classification accuracy of Pavia University under different methods.

	Model	GAN	GAN	WGAN	WGAN	WGAN	WGAN
	Class	-softmax	-SVM	-softmax	-SVM	-GP-softmax	-GP-SVM
	Asphalt	92.83	98.08	96.67	98.49	96.89	98.73
	Meadows	82.52	83.04	93.09	82.70	93.76	92.14
	Gravel	98.24	99.66	98.90	99.65	99.57	99.59
	Trees	95.40	93.51	98.01	98.10	98.73	99.05
	Meltal sheets	99.48	99.90	99.92	99.85	97.99	99.78
	Bare Soil	90.91	90.43	82.16	93.77	93.77	99.54
	Bitumen	85.56	84.06	77.29	66.70	92.71	90.45
	Bricks	88.73	95.27	90.79	98.75	95.30	98.61
	Shadow	92.93	70.33	96.20	95.99	97.46	95.78
	OA (%)	94.27	95.57	94.84	96.67	97.45	98.60
	AA (%)	91.84	90.48	92.56	92.70	96.24	97.08
	κ (%)	92.40	94.10	93.11	95.57	96.62	98.15

Table 3 .

 3 4: Classification accuracy of Indian Pines under different methods.

	Model	GAN	GAN	WGAN	WGAN	WGAN	WGAN
	Class	-softmax	-SVM	-softmax	-SVM	-GP-softmax	-GP-SVM
	Weeds1	93.47	84.78	91.30	86.96	91.30	93.48
	Weeds2	85.92	82.49	85.29	89.36	91.94	94.26
	Fallow	80.36	91.32	90.36	89.28	94.58	98.07
	Fallow-rough	77.63	69.62	78.06	81.43	89.87	87.34
	Fallow-smooth	92.34	92.96	96.27	92.13	87.99	92.75
	Stubble	95.62	97.67	98.77	98.22	97.67	96.44
	Celery	60.71	64.29	75.00	71.43	75.00	100.00
	Grapes-untrained	98.12	98.54	97.49	96.86	100.00	100.00
	Soil-vinyard	60.00	85.00	80.00	40.00	100.00	75.00
	Corn	86.32	91.77	95.16	92.28	90.95	96.29
	Lettuce-4wk	91.32	94.70	95.64	94.91	95.93	96.01
	Lettuce-5wk	81.96	79.59	87.02	89.04	89.71	95.28
	Lettuce-6wk	98.05	98.54	99.89	99.89	98.05	97.56
	Lettuce-7wk	97.71	95.65	96.21	98.74	98.33	98.26
	Vinyard-untrained	88.86	86.01	99.48	95.60	100.00	98.45
	Vinyard-vertical	91.40	79.57	78.49	77.42	93.54	97.85
	OA (%)	89.72	90.89	93.20	93.21	94.63	96.16
	AA (%)	86.23	87.03	90.28	87.10	93.43	94.82
	κ (%)	88.25	89.59	92.23	92.24	93.87	95.61

Table 3 .

 3

	OA (%)	Pavia University	Indian Pines
	Model	OA(%) AA(%) κ(%) OA(%) AA(%) κ(%)
	DR+GAN	95.57	90.48 94.10 90.89	87.03 92.24
	DR+WGAN	96.67	92.70 95.57 93.21	87.10 92.24
	DR+WGAN-GP	98.60	97.08 98.15 96.16	94.82 95.61
	PCA+GAN	95.26	93.57 93.37 88.74	83.40 87.05
	PCA+WGAN	95.73	95.69 94.73 92.35	89.74 91.18
	PCA+WGAN-GP 95.96	95.74 95.19 93.74	88.45 93.31
	From Table 3.3.3, we can see the differences between the proposed method
	(DR+WGAN-GP) and others. The proposed method improves about 2.3%, 2%,
	2.4% in OA, AA and κ values for Pavia University and 3.5%, 5%, and 3.4% for In-
	dian Pines compared with PCA+WGAN. When the proposed method is compared
	with PCA+GAN, the differences are larger, which are 2.8%, 3.8%, 3.8% for Pavia
	University and 7%, 12%, 7.5% for Indian Pines. In addition, since the proposed

3.3

, where "A + B" indicates the combination of dimensionality reduction method "A" and model "B", DR represents the proposed method. Table 3.5: Classification results of different GANs combining PCA or proposed dimensionality reduction method.

Table 4

 4

	.1. Considering

Table 4 .

 4 2: The classification accuracy of Pavia University based on different features.

	Prediction		Single-Level		Multi-Level
	Class	IV	III	II	I
	Asphalt	95.17	96.47	97.68	98.28
	Meadows	78.94	89.47	93.14	94.14
	Gravel	97.45	98.69	98.47	99.46
	Trees	95.98	96.96	97.98	97.75
	Metal sheets	99.86	99.98	100.00	100.00
	Bare soil	78.43	84.81	88.67	96.60
	Bitumen	77.44	79.70	79.92	91.43
	Bricks	90.71	93.45	96.06	96.79
	Shadows	98.83	99.36	99.78	99.79
	OA (%)	92.76	95.11	96.19	98.10
	AA (%)	90.33	93.20	94.65	97.14
	κ (%)	90.33	93.49	94.93	97.48

For the Pavia University data set, it can be observed from Table

4

.2 that Predic-

Table 4 .

 4 3: Classification accuracy of Indian Pines based on different features. any result based on single-level features, which proves that multi-level features allow us to obtain more useful information. It can be seen from Table 4.2 and Table 4.3 that the classification accuracy of some classes is always lower than other classes under different features, such as Bitumen in Pavia University, and Grass-pasture-mowed and Wheat in Indian Pines. From Table 1.1 and Table1.2 in Chapter 1, we can find that the number of Bitumen in Pavia University is the second smallest, and the number of Grasspasture-mowed and Wheat in Indian Pines is less than the average. Besides, the within-class variation and inter-class similarity may also reduce the classification accuracy, such as Asphalt and Bitumen, and Wheat and Grass-trees. But in general, the proposed multi-level features obtain the highest OA, AA, and κ values for the two data sets and the classification accuracy of most land-cover classes is improved when compared to the results that are obtained by single-level features.Both of the results shown in Table4.2 and Table4.

	Prediction		Single-Level		Multi-Level
	Class	IV	III	II	I
	Alfalfa	80.43	82.61	84.78	89.13
	Corn-notil	56.63	70.36	92.05	96.14
	Corn-min	58.89	74.58	82.28	87.04
	Corn	53.16	72.99	80.17	87.34
	Grass-pasture	84.06	95.24	97.31	98.75
	Grass-trees	93.84	96.85	97.81	98.90
	Grass-pasture-mowed	82.14	75.00	53.57	67.85
	Hay-windrowed	97.28	98.54	100.00	100.00
	Oats	95.00	90.00	75.00	100.00
	Soybean-notill	54.22	75.21	86.93	91.04
	Soybean-mintill	76.86	75.89	83.29	88.39
	Soybean-clean	85.36	94.63	97.07	97.56
	Wheat	54.97	66.61	76.73	83.31
	Woods	94.23	98.10	97.94	99.53
	Buildings-grass-trees	76.69	83.68	78.76	86.27
	Stone-stel-towers	92.47	95.70	96.77	97.85
	OA (%)	73.77	81.70	88.17	92.08
	AA (%)	77.27	84.12	86.28	91.83
	κ (%)	69.95	79.16	86.54	90.98

Table 4 .

 4 4: Classification accuracy of Pavia University based on different feature extraction methods.

	Method	Supervised FE		Unsupervised FE
	Class No.	DBN 2D-CNN	FA	SAE	3D-CAE Single-Level	3D-CAE Multi-Level
	1	95.85	96.74	95.88	96.26	97.48	98.58
	2	75.51	73.03	79.56	73.70	93.14	94.76
	3	97.88	99.63	86.47	97.55	98.76	99.68
	4	96.87	96.80	95.14	95.07	98.07	97.78
	5	99.78	99.14	99.03 100.00	100.00	100.00
	6	76.60	88.03	94.55	66.91	92.32	97.71
	7	72.93	89.02	81.65	82.78	86.99	95.49
	8	95.11	89.19	69.61	90.82	96.77	98.07
	9	99.79	96.72	95.88	97.88	100.00	100.00
	OA (%)	92.97	95.03	88.16	91.45	97.01	98.65
	AA (%)	90.03	92.13	88.64	89.00	95.94	98.01
	κ (%)	90.60	93.36	84.62	88.50	96.03	98.21

90%, but the introduction of multi-level features reaches 95%. Overall, the highest OA, AA, and κ values are obtained by the proposed multi-level features.

For Indian Pines data set (Table

4.

5), the classification results of FA are not good and the classification accuracy of most classes is less than 90%. DBN and SAE help us to improve the classification accuracy to a certain extent, but it's still not satisfactory. The OA and κ values based on 2D-CNN and CAE-based models exceed 90%, which demonstrates that convolution-based operations are more flexible and have strong feature extraction capabilities. Besides, the OA, AA, and κ values that are based on multi-level features improved by about 3%, 1%, and 3% compared with single-level features.

Table 4 .

 4 5: Classification accuracy of Indian Pines based on different feature extraction methods.

	Method	Supervised FE		Unsupervised FE
	Class No.	DBN	2D-CNN	FA	SAE	3D-CAE Single-Level	3D-CAE Multi-Level
	1	89.13	84.78	89.13	65.22	84.78	91.30
	2	92.77	82.49	61.81	86.14	93.49	94.61
	3	92.36	91.32	61.90	84.59	91.25	96.98
	4	87.76	69.62	43.88	83.12	91.14	94.93
	5	75.77	92.96	87.78	83.85	97.10	97.51
	6	92.33	97.676	81.51	95.21	99.17	99.45
	7	92.86	64.28	89.29	50.00	75.00	85.71
	8	98.12	98.53	93.10	94.35	99.58	100.00
	9	90.00	85.00	65.00	65.00	95.00	100.00
	10	77.77	87.97	53.60	88.37	88.78	94.15
	11	81.02	94.70	88.96	93.60	92.67	95.47
	12	98.54	83.02	99.99	88.29	94.63	91.39
	13	85.67	98.53	34.23	71.50	90.05	90.73
	14	98.74	95.65	95.65	92.89	98.33	99.92
	15	95.34	86.15	70.47	74.35	92.75	96.89
	16	46.23	79.56	68.82	55.91	99.97	95.69
	OA (%)	87.87	90.88	75.16	87.85	93.71	96.17
	AA (%)	87.15	87.03	74.07	79.53	92.73	95.29
	κ (%)	86.24	89.59	71.16	86.12	92.83	95.63

Table 5 .

 5 1: The P d values of different models for simulated a and simulated b .To evaluate the performance of the proposed model, when SNR is 20 dB, the ROC

	Noise	Model	Simulated a 10 dB 20 dB 25 dB 30 dB 10 dB 20 dB 25 dB 30 dB Simulated b
		DAE_50	0.60	0.80	0.96	1.00	0.66	0.86	0.97	1.00
	WGN	DAE_100 0.54 DAE_200 0.64	0.58 0.73	0.96 0.99	1.00 1.00	0.71 0.60	0.80 0.66	1.00 0.80	1.00 0.97
		MSDAE	0.85	0.96	1.00	1.00	0.86	0.97	1.00	1.00
		DAE_50	0.74	0.96	1.00	1.00	0.77	0.94	1.00	1.00
	MPN	DAE_100 0.78 DAE_200 0.71	0.99 0.84	1.00 0.99	1.00 1.00	0.83 0.71	0.97 0.86	1.00 1.00	1.00 1.00
		MSDAE	0.83	1.00	1.00	1.00	0.89	1.00	1.00	1.00

Table 5 .

 5 2: The P d values of different models for HSI b and HSI c .

	Noise	Model	HSI b 10 dB 20 dB 30 dB 40 dB 10 dB 20 dB 30 dB 40 dB HSI c
		DAE_74	0.63	0.94	1.00	1.00	0.60	0.74	0.88	1.00
	WGN	DAE_148 0.69 DAE_296 0.73	0.71 0.96	0.86 1.00	0.95 1.00	0.74 0.88	0.91 0.99	0.97 1.00	1.00 1.00
		MSDAE	0.83	1.00	1.00	1.00	0.91	1.00	1.00	1.00
		DAE_74	0.72	0.94	0.96	1.00	0.69	0.89	0.94	1.00
	MPN	DAE_148 0.51 DAE_296 0.65	0.88 0.99	0.94 1.00	0.99 1.00	0.69 0.86	0.91 0.99	1.00 1.00	1.00 1.00
		MSDAE	0.90	1.00	1.00	1.00	0.94	1.00	1.00	1.00
	We can see from Table 5.2 that among all single DAE models, DAE_296 performs
	better under two different types of noise in real experiment. But the proposed
	MSDAE still has a significant improvement compared with DAE_296, especially

When the ζ is increased from 10 dB to 80 dB, we find when the SNR exceeds 40 dB for HYDICE images, the P d values are always one, so we don't list them in the table. Other relevant P d values under PFA = 10 -3 for different reconstructed images are listed in Table 5.2. when the SNR value is small. Overall, the proposed MSDAE achieves the best results in different SNR values.

Table 5 .

 5 3: Network structures of 2D-CNN. 216 × 148 × 1 3 × 3 × 148 × 24 316 × 216 × 24 × Conv-2 316 × 216 × 24 × 1 3 × 3 × 24 × 24 316 × 216 × 24 × Conv-3 316 × 216 × 24 × 1 1 × 1 × 24 × 24 316 × 216 × 24 ×

	Layer	Input Size	Kernel	Output
	Conv-1	316 ×

Acknowledgments

I would like to express my deepest appreciation to my supervisor, Prof. Salah Bourennane, for receiving me to start my PhD journey in France, and for his vast

List of Tables

(a) (b)

Network construction

The dimensionality of spectral dimension of HSI a is reduced by PCA to reduce the amount of calculation. The three 3D-CAEs are designed into a symmetrical structure in the experiment as shown in

Experimental results

To verify the performance of the proposed method, experiments on simulated and real-world HSIs are done and analyzed in this Section. In addition, the denoising performance of the proposed method is tested on noisy image R which is obtained by adding random noise N to the image H, i.e. R = H + N. Gaussian noise and multiplicative noise (MPN) are often encountered in hyperspectral imagery [START_REF] Acito | Signal-dependent noise modeling and model parameter estimation in hyperspectral images[END_REF][START_REF] Mikhail L Uss | Local signal-dependent noise variance estimation from hyperspectral textural images[END_REF]. Thus, we use zero-mean white Gaussian noise (WGN) and MPN uniformly distributed with zero-mean are introduced to model the random noise.

In the experiment, we will discuss and analyze the removal effects of two kinds of noise with different signal-to-noise ratio (SNR) values. The SNR is estimated

Moreover, three commonly used denoising algorithms WF, BM3D and DnCNN are served as the contrastive methods.

The most commonly used ACE detector is selected to detect the target, which can be expressed as follows:

where k is the pixel spectrum and s is the target template spectrum. Γ is the covariance matrix estimated by 1 T and k i is the i th column of the matrix R with k = 1

List of Tables