Sahar Ghannay

Antoine Caubrière

Salima Mdhaffar

Gaëlle Laperrière

Bassam Jabaian

Yannick Estève

Where are we in semantic concept extraction for Spoken Language Understanding? ⋆

Keywords: Spoken language understanding, End-to-end approach, Cascade approach, Self supervised training

Spoken language understanding (SLU) topic has seen a lot of progress these last three years, with the emergence of end-to-end neural approaches. Spoken language understanding refers to natural language processing tasks related to semantic extraction from speech signal, like named entity recognition from speech or slot filling task in a context of human-machine dialogue. Classically, SLU tasks were processed through a cascade approach that consists in applying, firstly, an automatic speech recognition process, followed by a natural language processing module applied to the automatic transcriptions. These three last years, end-toend neural approaches, based on deep neural networks, have been proposed in order to directly extract the semantics from speech signal, by using a single neural model. More recent works on self-supervised training with unlabeled data open new perspectives in term of performance for automatic speech recognition and natural language processing. In this paper, we present a brief overview of the recent advances on the French MEDIA benchmark dataset for SLU, with or without the use of additional data. We also present our last results that significantly outperform the current state-of-the-art with a Concept Error Rate (CER) of 11.2%, instead of 13.6% for the last state-of-the-art system presented this year.

Introduction

Spoken language understanding (SLU) refers to natural language processing tasks related to semantic extraction from the speech signal [START_REF] Tur | Spoken language understanding: Systems for extracting semantic information from speech[END_REF], like named entity recognition from speech, call routing, slot filling task in a context of humanmachine dialogue. . . Usually, SLU tasks were processed through a cascade approach that consists in applying first an automatic speech recognition (ASR) process, followed by a natural language processing module applied to the automatic transcription [START_REF] De Mori | Spoken language understanding: a survey[END_REF]. For both automatic speech recognition and natural language processing, deep neural networks (DNN) have made great advances possible, leading to impressive improvements of qualitative performance for final SLU tasks [START_REF] Amodei | Deep speech 2: Endto-end speech recognition in english and mandarin[END_REF][START_REF] Collobert | Natural language processing (almost) from scratch[END_REF][START_REF] Vaswani | Attention is all you need[END_REF].

These three last years, end-to-end neural approaches, based on deep neural networks, have been proposed in order to directly extract the semantics from speech signal, by using a single neural model [START_REF] Serdyuk | Towards endto-end spoken language understanding[END_REF][START_REF] Ghannay | End-to-end named entity and semantic concept extraction from speech[END_REF]. A first advantage of such approaches consists on a joint optimization of the ASR and NLP part, since the unique neural model is optimized only for the final SLU task. Another advantage is the limitation of the error propagation: when using a cascade approach, an error in the first treatment implies errors in the following ones. In a neural end-to-end approach, the model decision is delayed to the output layer: all the information uncertainty is handled until the final decision.

Very recently, works on self-supervised training with unlabeled data open new perspectives in term of performance for automatic speech recognition and natural language processing [START_REF] Baevski | wav2vec 2.0: A framework for self-supervised learning of speech representations[END_REF][START_REF] Devlin | BERT: Pretraining of deep bidirectional transformers for language understanding[END_REF]. They can be applied to SLU task.

This study presents experimental results on the French MEDIA benchmark dataset. This benchmark dataset is one of the most challenging benchmarks for SLU task. In this paper, we present a brief overview of the performance evolution of state-of-the-art systems on this benchmark dataset. We also present an approach that takes benefit from acoustic-based and linguistic-based models pre-trained on unlabelled data: this approach represents the next milestone to be surpassed.

MEDIA dataset

The French MEDIA corpus [START_REF] Bonneau-Maynard | Semantic annotation of the french media dialog corpus[END_REF], is dedicated to semantic extraction from speech in a context of human-machine dialogues for a hotel booking task. This dataset was created as a part of the Technolangue project of the French government in 2002. Its main objective is to set up an infrastructure for the production and dissemination of language resources, the evaluation of written and oral language technologies, the participation in national and international standardisation bodies and an information monitoring in the field.

The MEDIA dataset is made of telephone dialogue recordings with their manual transcriptions and semantic annotations. It is composed of 1257 dialogues from 250 different speakers, collected with a Wizard-of-Oz setting between two humans: one plays a computer, the other plays the user. The dataset is split into three parts (train, dev, test) as described in table 1. In this work, we used the user part of MEDIA, since it has both speech and semantic annotations.

The semantic domain of this corpus is represented by 76 semantic concept tags such as room number, hotel name, location, etc. Some more complex linguistic tags, like co-references, are also used in this corpus.

The following sentence (translated from French) is an example of the MEDIA content: "I would like to book one double room in Paris up to one hundred and thirty euros". It will be annotated as (I would like to book, reservation), (one, number-room), (double room, room-type), (up to, comparative-payment), (one hundred and thirty, amount-payment), (euros, currency-payment).

In [START_REF] Béchet | Benchmarking benchmarks: introducing new automatic indicators for benchmarking spoken language understanding corpora[END_REF], Béchet and Raymond showed why the MEDIA task can be considered as the most challenging SLU benchmark available, in comparison to other wellknown benchmarks such as ATIS [START_REF] Dahl | Expanding the scope of the atis task: The atis-3 corpus[END_REF], SNIPS [START_REF] Coucke | Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces[END_REF], and M2M [START_REF] Shah | Building a conversational agent overnight with dialogue self-play[END_REF].

3 Overview of approaches proposed for the MEDIA benchmark

Cascade approach

Conventional SLU systems are designed as a cascade of components. Each of them solves separately a specific problem. First, an ASR module, trained on a large amount of data, maps speech signals to automatic transcriptions. This is then passed on to a natural language understanding (NLU) module that predicts semantic information from the automatic transcriptions. In this approach, error propagation is unavoidable, despite the performance of current ASR and NLU systems. In addition, those modules are optimized separately under different criteria. The ASR system is trained to minimize the word error rate (WER), while the NLU module is trained to minimize the concept error rate (CER) in case of slot filling task. This separate optimization suggests that a cascade SLU system is suboptimal. Working on automatic transcriptions, for an SLU task on MEDIA corpus, is highly challenging. Many approaches have been proposed. Early NLU approaches were based on generative models such as Stochastic finite state transducers (FST), on discriminative or conditional models such as conditional random fields (CRFs) and support vector machines(SVMs) [START_REF] Hahn | Comparing stochastic approaches to spoken language understanding in multiple languages[END_REF]. In the light of the success of neural approaches in different fields, some studies developed neural architectures for SLU task. In [START_REF] Mesnil | Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding[END_REF], the author presents the first recurrent neural architecture dedicated to SLU for the ATIS benchmark corpus. This neural model was applied to transcriptions despite speech signals directly. In [START_REF] Simonnet | ASR error management for improving spoken language understanding[END_REF][START_REF] Simonnet | Simulating ASR errors for training SLU systems[END_REF], for the first time, an encoder-decoder neural network structure with attention mechanism [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF] was proposed for this task. This time, it was on manual and automatic transcriptions from the MEDIA corpus. In order to reduce the unavoidable SLU performance decline due to ASR errors, the authors in [START_REF] Simonnet | ASR error management for improving spoken language understanding[END_REF] have proposed ASR confidence measures to localize ASR errors. These confidence measures have been used as additional SLU features to be combined with lexical and syntactic features, useful for characterizing concept mentions. In [START_REF] Simonnet | Simulating ASR errors for training SLU systems[END_REF], the authors proposed an approach to simulate ASR errors from manual transcriptions, to improve the performance of SLU systems. The use of the resulting corpus prepares the SLU system to ASR errors during their training and makes it more robust to ASR errors.

End-to-end approach

As seen in the previous section, one problem with cascaded approaches is the propagation of errors through the components. The intermediate transcription is noisy due to speech recognition errors, and the NLU component has to deal with these errors. The other problem comes from the separate optimzations of the different modules.

To tackle these issues, end-to-end approaches were proposed in order not to use an intermediate speech transcriptions. This kind of approach aims to develop a single system directly optimized to extract semantic concepts from speech.

SLU end-to-end systems are usually trained to generate both recognized words and semantic tags [START_REF] Ghannay | End-to-end named entity and semantic concept extraction from speech[END_REF][START_REF] Desot | Towards end-to-end spoken intent recognition in smart home[END_REF].

Until now, mainly two kinds of neural architectures have been proposed on the MEDIA benchmark. The first one is based on the use of the Connectionist Temporal Classification (CTC) loss function [START_REF] Graves | Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks[END_REF], while the other one is based on the use of an encoder-decoder architecture with attention mechanism [START_REF] Bahdanau | Neural machine translation by jointly learning to align and translate[END_REF].

CTC approach

In this work, we call CTC approach the neural architecture trained by using the CTC loss function. This loss function allows the system to learn an alignment between the input speech and the word and concept sequences to produce.

To our knowledge, the best-published results with a CTC approach on ME-DIA were obtained by [START_REF] Caubrière | Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability[END_REF]. In this study, the authors proposed a neural architecture largely inspired by the DeepSpeech 2 speech recognition system. The neural architecture is a stack of two 2D-invariant convolutional layers (CNN), followed by five bidirectional long short term memory (bLSTM) layers with sequence-wise batch normalization, a classical fully connected layer, and the softmax output layer. As input features, we used spectrograms of power to normalize audio clips, calculated on 20ms windows. This system was trained following the curriculumbased transfer learning approach, which consists in training the same model through successive stages, with different tasks ranked from the most generic one to the most specific one. The authors used speech recognition tasks, then named entity extraction and finally semantic concept extraction tasks.

Encoder-decoder approach with attention mechanism

The encoder-decoder architecture was initially implemented in the machine translation context. This approach quickly showed its benefits for the speech recognition task [START_REF] Chorowski | End-to-end continuous speech recognition using attention-based recurrent nn: First results[END_REF][START_REF] Chan | Listen, attend and spell: A neural network for large vocabulary conversational speech recognition[END_REF][START_REF] Chiu | State-of-the-art speech recognition with sequence-to-sequence models[END_REF], and more recently for SLU tasks [START_REF] Serdyuk | Towards endto-end spoken language understanding[END_REF][START_REF] Pelloin | End2end acoustic to semantic transduction[END_REF].

The encoder-decoder architecture is divided into two main parts. First, an encoder receives the speech features as input, and provides its hidden states to build a high-level representation of the features. This high-level representation is then passed on to an attention module. It identifies the parts of these representations that are relevant for each step of the decoding process. Next, the attention module computes a context vector from these representations to feed the decoder. Finally, the decoder processes the input context vectors to predict the transcription of speech, enriched with semantic concepts. At each decoding time step, a new context vector is computed from the encoded speech representations. Unlike CTC approaches, the output sequence size of an encoder-decoder approach does not depend on the input sequence size.

A recent study [START_REF] Pelloin | End2end acoustic to semantic transduction[END_REF] used a similar architecture and obtained the state-ofthe-art performance for the MEDIA task. The encoder part is composed of four 2-dimensional convolution layers followed by four bLSTM layers. Each convolution layer is followed by a batch normalization. The decoder part is a stack of four bLSTM layers, two fully connected layers, and a softmax layer. The input features of the network are 40-dimensional MelFBanks with a Hamming window of 25ms and 10ms strides.

This encoder-decoder system is trained following the curriculum-based transfer learning, with the same data used for the CTC approach presented in section 3.2.1, except for the named entity extraction task which was not used.

System performance

SLU systems can be evaluated with different metrics. Historically, on the MEDIA corpus, two metrics are jointly used: the Concept Error Rate (CER) and the Concept/Value Error Rate (CVER). The CER is computed similarly to the Word Error Rate, by only taking into account the concepts occurrences in both the reference and the hypothesis files. The CVER metrics is an extension of the CER. It considers the correctness of the complete concept/value pair. In the example in section 2, both "one hundred and thirty" and amount-payment have to be correct to consider the concept/value pair (one hundred and thirty,amount-payment) as correct. Errors on the value component can come from a bad segmentation (missing or additional words in the value) or from ASR errors.

Table 2 presents the best results obtained on the official MEDIA benchmark dataset, by the main families of approaches presented in the two previous sections. By computing the 95% confidence interval, we observe a 0.7 confidence margin for CER and 0.8 for CVER, when the CER is 13.6% and the CVER 18.5%. Until now, the best result was reached by an end-to-end encoderdecoder architecture with attention mechanism, trained by following a curriculum transfer-learning approach [START_REF] Pelloin | End2end acoustic to semantic transduction[END_REF].

Improving the state of the art

In the previous section we present state-of-the-art performances. Recently, unsupervised learning on huge amount of data have been successfully proposed to pre-train Transformers-based models [START_REF] Devlin | BERT: Pretraining of deep bidirectional transformers for language understanding[END_REF][START_REF] Baevski | wav2vec 2.0: A framework for self-supervised learning of speech representations[END_REF]. Thanks to these models, ASR state-of-the-art performance [START_REF] Baevski | wav2vec 2.0: A framework for self-supervised learning of speech representations[END_REF] and NLP state-of-the-art performance [START_REF] Devlin | BERT: Pretraining of deep bidirectional transformers for language understanding[END_REF] have been outperformed, with respectively wav2vec and BERT models. In this section, we present a cascade system using both BERT and wav2vec optimized on the MEDIA task.

BERT and CamemBERT models

For the NLU module, we propose to use the one that achieved the state-ofthe-art result on manual transcriptions of MEDIA corpus [START_REF] Ghannay | Neural networks approaches focused on French spoken language understanding: application to the MEDIA evaluation task[END_REF]. This system is based on a fine-tuning of BERT [START_REF] Devlin | BERT: Pretraining of deep bidirectional transformers for language understanding[END_REF] on MEDIA SLU task using the French CamemBERT [START_REF] Martin | Camembert: a tasty french language model[END_REF] model.

BERT [START_REF] Devlin | BERT: Pretraining of deep bidirectional transformers for language understanding[END_REF] is a deeply bidirectional, unsupervised language representation model, which stands for Bidirectional Encoder Representations from Transformers. It is designed to pre-train deep bidirectional representations from unlabeled text, taking into account both left and right context in all layers. The resulting pre-trained BERT model can be fine-tuned with just one additional output layer, to create state-of-the-art models for a wide range of NLP tasks. BERT is pretrained using a combination of masked language modeling objective and next sentence prediction on a large corpus which include the Toronto Book Corpus and Wikipedia.

The French CamemBERT model is based on RoBERTa (Robustly Optimized BERT Pre-training Approach) [START_REF] Liu | Roberta: A robustly optimized bert pretraining approach[END_REF] which is based on BERT. CamemBERT is similar to RoBERTa, which dynamically change the masking pattern applied to the training data, and remove the next sentence prediction task. In addition, it uses the whole word masking and the SentencePiece tokenization [START_REF] Kudo | SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing[END_REF]. The CamemBERT model is trained on the French CCNet corpus composed of 135GB of raw text.

Wav2vec models

Wav2vec 2.0 [START_REF] Baevski | wav2vec 2.0: A framework for self-supervised learning of speech representations[END_REF] is a model pre-trained through self-supervision. It takes raw audio as input and computes contextual representations that can be used as input for speech recognition systems. It contains three main components: a convolutional feature encoder, a context network and a quantization block. The convolutional feature encoder converts the audio signal into a latent representation. This representation is given to the context network which takes care of the context. The context network architecture consists of a succession of several transformer encoder blocks. The quantization network is used to map the latent representation to quantized representation.

In [START_REF] Evain | Lebenchmark: A reproducible framework for assessing self-supervised representation learning from speech[END_REF], the authors released French pre-trained wav2vec 2.0 models. Two models have been released for public use3 , a large one and a base one. In this study, we use the large configuration which encodes raw audio into frames of 1024-dimensional vectors. The models are pre-trained in a unsupervised way with 3K hours of unlabeled speech. Details about data used to train the wav2vec models can be found in [START_REF] Evain | Lebenchmark: A reproducible framework for assessing self-supervised representation learning from speech[END_REF]. The trained model is composed of about 300M parameters.

To get better ASR results than the ones we could reach by fine-tuning the French wav2vec 2.0 model, on the MEDIA training data only, we suggest to ,first, fine-tune on external audio data, as proposed in [START_REF] Caubrière | Curriculum-Based Transfer Learning for an Effective End-to-End Spoken Language Understanding and Domain Portability[END_REF] or [START_REF] Pelloin | End2end acoustic to semantic transduction[END_REF]. To make the experiments reproducible, instead of using the Broadcast News data used in these works, we used the CommonVoice French dataset4 (version 6.1), collected by the Mozilla Foundation, and much easily accessible. The train set consists of 425.5 hours of speech, while the validation and test sets contain around 24 hours of speech.

Cascade approach with pre-trained models

As written before, we propose in this work to use a cascade approach, with pretrained models for each component. The ASR system is composed of the large pre-trained French wav2vec model, a linear layer of 1024 units, and the softmax output layer. First, we optimize the ASR system on the French CommonVoice dataset. Then, we fine-tune it for speech recognition on the French MEDIA corpus, the wav2vec weights being updated at each training stage. The loss function used at each fine-tuning step is the CTC loss function. We call the final ASR model W2V • Common Voice • M ASR .

The NLU system is applied on the automatic transcriptions provided from the ASR system, to obtain semantic annotations. This system is based on the fine-tuning of the French CamemBERT [START_REF] Martin | Camembert: a tasty french language model[END_REF] model, on the manual transcriptions of MEDIA corpus. It achieved state-of-the-art result on manual transcriptions of MEDIA corpus [START_REF] Ghannay | Neural networks approaches focused on French spoken language understanding: application to the MEDIA evaluation task[END_REF], yielding to 7.56 of CER when there is no error in the transcription.

Results and discussion

The experimental result obtained with the proposed cascade approach is presented in table 3. We compare the performance of this cascade system, named W2V • Common Voice • M ASR + CamemBERT, to the E2E encoder-decoder model proposed in [START_REF] Pelloin | End2end acoustic to semantic transduction[END_REF], that reached the best result on this task until now, and other wav2vec-based models. All the wav2vec-based models presented in table 3 were implemented thanks to the SpeechBrain toolkit 5 , including the fine-tuning of the wav2vec models.

Like in section 3.3, the results are evaluated in terms of CER and CVER. Our new system yields to 17.64% of relative CER improvement and 7.02% of relative CVER improvement, by reaching respectively 11.2% of CER and 17.2% of WER. The result shows the effectiveness of unsupervised pre-trained models like wav2vec and BERT in such a scenario. Notice that the W2V • Common Voice • M ASR model allows us to have an effective ASR system that achieved 8.5% of WER.

In system (1), the wav2vec model is fine-tuned directly on MEDIA SLU (M SLU ) task. In system (2), the wav2vec model is first fine-tuned on the Common Voice data then on M SLU task, and a beam search decoding is applied. In systems ( 3) and ( 4) the wav2vec model is first fine-tuned on the Common Voice data, then on MEDIA ASR (M ASR ), and last on M SLU task, using the greedy or the beam search decoding using a 5-gram language model to rescore. This language model is trained on the manual transcriptions of M SLU training data only.

It is worth to mention that even before the generalisation of the use of neural networks for sequential tagging tasks, such as the slot filling task investigated in this paper, several efforts have been made to better take into account the the ASR system errors during the semantic labeling. Many approaches have been proposed for a joint decoding between speech recognition and understanding, considering the n-best recognition hypotheses during the semantic annotation [START_REF] Hakkani-Tü | Beyond ASR 1-best: Using word confusion networks in spoken language understanding[END_REF][START_REF] Servan | Conceptual decoding from word lattices: application to the spoken dialogue corpus ME-DIA[END_REF][START_REF] Jabaian | Error-corrective discriminative joint decoding of automatic spoken language transcription and understanding[END_REF]. When neural networks have become state-of-the-art systems for SLU, end-to-end approaches have gradually replaced cascade approaches and have shown very good performance, allowing the semantic labeling of a speech signal and minimising the impact of transcription errors on the SLU performance. However, these architectures need a large amount of data and often use pre-trained external module that have been trained separately in out-of-context data. The results presented in table 3 show that if such pre-trained models are used in a cascade architecture, the resulting system reaches or even exceeds the performance of the end-to-end based one. In addition, the result of the cascade system reinforces the idea of the use of pre-trained models at the encoder (wav2vec) and decoder (BERT) levels within end-to-end architecture, as proposed in [START_REF] Chung | Splat: Speech-language joint pre-training for spoken language understanding[END_REF]. This leads us to conclude that the two architectures remain valid and competitive and that the choice should be made according to the availability of additional data and the pre-training models.

Conclusions

In this paper, we present a brief overview of the recent advances on the French MEDIA benchmark dataset for SLU. We propose a system based on a cascade 3. Performance on Test MEDIA in terms of CER and CVER scores of the proposed cascade and end-to-end systems using pre-trained models. "•" formalizes a transfer learning step during the training of the E2E system. approach, that takes benefit from acoustic-based and linguistic-based models pretrained on unlabelled data : wav2vec models for the ASR system, and BERTlike model for the NLU system. Experimental results show that our system outperforms significantly the current state of the art with a Concept Error Rate (CER) of 11.2% instead of 13.6% for the last state-of-the-art system presented this year.

This new advance reinforces the idea of the use of pre-trained models at the encoder (wav2vec) and decoder (BERT) levels within an end-to-end architecture. This will explored in our future work.

This study leads us to conclude that the two architectures (cascade vs. end-toend) remain valid and competitive and that the choice should be made according to the availability of additional data and relevant pre-trained models.

  W2V • Common Voice • MSLU (Beam 5g) 15.8 20.4 (3) W2V • Common Voice • MASR • MSLU (greedy) 15.4 20.5 (4) W2V • Common Voice • MASR • MSLU (Beam 5g) 14.5 18.8 Cascade W2V • Common Voice • MASR + CamemBERT 11.2 17.2 Table

Table 1 .

 1 The official MEDIA dataset distribution

	Data Nb	Nb	Nb	Nb
	Words Utterances Concepts Hours
	train 94.2k	13.7k	31.7k	10h46m
	dev 10.7k	1.3k	3.3k	01h13m
	test 26.6k	3.7k	8.8 k	02h59m

Table 2 .

 2 Best results obtained on the official MEDIA benchmark dataset, by the main families of approaches presented in this paper. Results are given in both Concept Error Rate and Concept/Value Error Rate

	Architecture	Model	CER CVER
	Cascade (2018)	HMM/DNN ASR + neural NLU [32]	20.2 26.0
	Cascade (2018)	HMM/DNN ASR + CRF [32]	20.2 25.3
	Cascade (2019)	HMM/TDNN ASR + CRF [6]	16.1 20.4
	End-to-end (2019) E2E CTC [6]	16.4 20.9
	End-to-end (2021) E2E encoder-decoder with attention [28] 13.6 18.5

https://huggingface.co/LeBenchmark

https://commonvoice.mozilla.org/fr/datasets

https://speechbrain.github.io


C Bernard-Michel

S Douté

M Fauvel

L Gardes

MISTIS S Girard

Retrieval of Mars surface physical properties from OMEGA hyperspectral images using Regularized Sliced Inverse Regression

In this paper, a statistical method is proposed to evaluate the physical properties of surface materials on Mars from hyperspectral images collected by the OMEGA instrument aboard the Mars express spacecraft.

The approach is based on the estimation of the functional relationship F between some observed spectra and some physical parameters. To this end, a database of synthetic spectra is generated by a physical radiative transfer model and used to estimate F . The high dimension of spectra is reduced by Gaussian regularized sliced inverse regression (GRSIR) to overcome the curse of dimensionality and consequently the sensitivity of the inversion to noise (ill-conditioned problems). Compared with other approaches, GRSIR has the advantage of being very fast, interpretable and accurate.

Introduction

Visible and near infrared imaging spectroscopy is a key remote sensing technique used to study and monitor planets. It allows the detection, mapping and characterization of minerals, as well as volatile species, that often constitute the first step toward the resolution of key climatic and geological issues [Murchie et al., 2007;Bibring et al., 2004a;[START_REF] D R A F T Brown | The Cassini Visual And Infrared Mapping Spectrometer (Vims) Investigation[END_REF][START_REF] Carlson | Near infrared spectrometer experiment on Galileo[END_REF]. These tasks are carried out through spectral analysis of solar light reflected by the material forming the top few millimeters or centimeters of the ground. Physical properties of the surface, such as chemical composition, granularity, texture, and physical state are some of the most important parameters that characterize the morphology of spectra. Modeling the direct link between these parameters and observable spectra is called the forward problem in classical physics. It can be evaluated numerically by radiative transfer models, simulating the propagation of solar light through the atmosphere and reflected back to the sensor [Douté et al., 2007b;Hapke, 2002;Shkuratov et al., 1999;[START_REF] Douté | A multilayer bidirectional reflectance model for the analysis of planetary surface hyperspectral images at visible and near-infrared wavelengths[END_REF]Hapke, 1993]. Such a model allows the simulation of spectra from given values of the model parameters. Conversely, deducing the physical model parameters from the observed spectra is called an inverse problem. Since it generally cannot be solved analytically, the use of optimization or statistical methods is necessary. Solving inverse problems requires an adequate understanding of the fundamental physics, so that a relation X = G(Y ) may be specified between the spectra X and the parameters Y . Given G, different methods can be used to deduce the parameters Y from the observation X. Current solutions to the inverse problem can be D R A F T November 6, 2008, 2:58pm D R A F T divided into three main categories (for further details and comparisons, see [Kimes et al., 2000;Pragnère et al., 1999]):

1. Optimization algorithms: These methods minimize an objective function that expresses the quality of the estimations, and include numerical optimization techniques such as Powell's method, Simplex method and the quasi-Newton method. Estimation can be unstable since inverse problems are often ill-posed (solutions are not unique, and a small change in the data can lead to large differences in the estimations). A probabilistic formalism can be used to regularize inverse problems by introducing a prior distribution on model parameters [START_REF] Aster | Parameter Estimation and Inverse Problems[END_REF]Tarantola, 2005]. These approaches are computationally expensive since they independently invert new spectra. Therefore, they cannot be used to invert an image with several hundred thousand pixels. Moreover, they can sometimes fall into local minima if the objective function is not convex.

2. Look-up table (LUT) / k-nearest neighbors approach (k-NN): This methodology is currently used by physicists to study planetary bodies [START_REF] Carlson | Distribution of hydrate on Europa: Further evidence for sulfuric acid hydrate[END_REF]Philpot et al., 2004;Weiss et al., 2000;[START_REF] Douté | Mapping SO 2 frost on Io by the modeling of NIMS hyperspectral images[END_REF]. The previous heavy runtime computation is replaced by a simpler look-up operation. A large database (LUT) is generated by radiative transfer for many parameter values. To invert an hyperspectral image, the pixel's spectrum is then compared with the LUT spectra in order to find the best match (the nearest neighbor), according to an objective function minimization. Parameters are then deduced from this best match. The speed gain is significant in comparison to traditional optimization methods, since retrieving a value from memory is often faster than undergoing an expensive computation. The main disadvantages of this approach are the multiplicity of solutions and their instability [START_REF] Bernard-Michel | Estimation of Mars surface physical properties from hyperspectral images using sliced inverse regression[END_REF]. with the same physical model. Among training approaches, neural networks [Hastie et al., 2003, chapter 11] or support vector machines (SVM) [Hastie et al., 2003, chapter 12] seem promising but the underlying learning process remains time consuming [START_REF] Combal | Retrieval of canopy biophysical variables from bidirectional reflectance using prior information to solve the ill-posed inverse problem[END_REF]Durbha et al., 2007;Pragnère et al., 1999].

Hyperspectral images on planets must be inverted with the following constraints: 1.

Working with large data-sets and various models require fast methodologies, 2. When dealing with very high-dimensional data, one is faced with the 'curse of dimensionality' and the associated sparsity issues and, 3. Observed spectra always contain some noise.

In this paper, a new training approach is proposed: The Gaussian Regularized Sliced Inverse Regression (GRSIR) [Bernard-Michel et al., to appear, 2008]. It is based on a dimension reduction technique first proposed by [Li , 1991] and similar to Partial Least Squares regression (PLS), [Hastie et al., 2003, chapter 3]. It has the advantage of being fast, stable, statistically and physically interpretable, and can also help to select an appropriate look-up table for inversion. For the sake of validation, this approach is compared with k-NN, PLS and SVM.

D R A F T November 6, 2008, 2:58pm D R A F T
In Section 2, real and simulated data-sets are presented, then the principle of GRSIR is recalled in Section 3. The choice of an appropriate LUT is discussed in Section 4.

Experimental results are presented on simulated data-sets in Section 5 and on real Mars images in Section 6. Conclusions are drawn in Section 7.

Data

The data-sets used in the next sections can be divided in three categories: The hyperspectral images observed on Mars, a LUT simulated by radiative transfer algorithms according to some physical modeling of these images, and test data in order to quantify the according of the estimation. All data-sets consist of some spectra X ∈ R d with associated parameters Y ∈ R p , d being the number of wavelengths and p the number of parameters.

Hyperspectral images from Mars

The data-sets were collected by the imaging spectrometer OMEGA (Observatoire pour la Minéralogie, l'Eau, la Glace et l'Activité) [Bibring et al., 2004a]. 4.15 (i.e. d = 184). For each spectrum of the image, the atmospheric gaseous contribution has been removed [Douté et al., 2007b]. After a first analysis, these OMEGA observations revealed that the south polar region of Mars exhibits mainly water ice, carbon dioxide ice and dust at the surface [Bibring et al., 2004b]. A detailed qualitative mapping of the ice during the local summer shows that the permanent bright cap is dominated by superficial CO 2 ice. Nevertheless, its spectral signature is slightly contaminated by the contribution of dust and water ice. Among the possible coexistence modes of the latter components, a granular mixture of H 2 O, CO 2 ice and dust is the most appropriate to reproduce the morphology of the spectra [Douté et al., 2007a]. This conclusion is based on numerical experiments that aimed at reproducing the spectra by different surface reflectance models.

The permanent polar cap has been mapped by a classification method based on wavelets [START_REF] Douté | A Comprehensive Numerical Package for the Modeling of Mars Hyperspectral Images[END_REF]. For each image, the CO 2 ice areas contain about 10000 to 20000 spectra. The selected model for the polar cap considers the intimate mixture of H 2 O, CO 2 ice and dust as an optically thick parallel layer without substantial roughness. The transfer of solar photons through such a material is calculated by a modified version of the Hapke semi-empirical formulation [Hapke, 2002;[START_REF] Douté | A multilayer bidirectional reflectance model for the analysis of planetary surface hyperspectral images at visible and near-infrared wavelengths[END_REF]Hapke, 1993].

Look-up table (LUT)

The model implies five spatially varying parameters (i.e. p = 5): the grain size of water and CO 2 ice, the proportion of water ice, CO 2 ice and dust. The proportions are normalized to sum to one. The other parameters involved in the physical model, such as the incidence and emergence angles or the grain size of dust have been fixed to a constant.

D R A F T November 6, 2008, 2:58pm D R A F T
Since a priori knowledge on the parameters range is unavailable, the simulation of a large look-table (LUT) is necessary. Details of the sampling strategy are provided in Table 1.

The notation used in this article is as follows: n is the number of simulated spectra (n = 31500), x i ∈ R 184 denotes a spectrum from the LUT and y i ∈ R 5 its associated parameters, where i ∈ {1, . . . , n}.

Test data

For validation and comparison, the use of a test data-set (Tdata) is required. Since no ground truth is currently available for the physical properties of Mars polar regions, we can only rely on synthetic data. The same physical model is used to simulate the LUT and Tdata. The range of variation for the parameter is detailed in Table 1. To emulate real data, a zero mean multiGaussian noise of dimension 184 is added to all the spectra of the test data-set. The covariance matrix is determined experimentally from a small, spatially homogeneous, portion of the real image. Assuming that much of the variability comes from the noise, the latter is then evaluated using statistics based on a shift difference on the selected portion. As in the previous section, n T is the number of simulated test spectra (n T = 3500), x T i is a spectrum from Tdata and y T i its associated parameters, where i ∈ {1, . . . , n T }.

Proposed approach

Our goal is to estimate the functional relationship F between the spectra X ∈ R d

and each physical parameter Y j = F (X) ∈ R with j ∈ {1, . . . , p}. In this work, each parameter is studied individually, therefore p = 1 and j will be omitted for the sake of simplicity. The originality of the methodology is to split the difficult problem of learning

D R A F T November 6, 2008, 2:58pm D R A F T
a d-variate function into two sub-problems that can be easily solved. First, a dimension reduction technique is applied (Section 3.1). Then, estimation is performed in the lower dimensional subspace (Section 3.2).

Dimension reduction step: Gaussian regularized sliced inverse regression

Dimension reduction methods rely on the assumption that the predictor X can be replaced, without loss of information, by its projection onto a subspace of smaller dimension L, called the effective dimension reduction space (EDR). The basis of the subspace is denoted β 1 , . . . , β L and the functional relationship Y = F (X) can be rewritten as

Y = f (β t 1 X, . . . , β t L X)
, where f is now a L-variate function. In most applications, L is smaller than 3 making the estimation of f tractable.

Principal Component Analysis (PCA) is a classical approach to reduce the dimension [Hastie et al., 2003, chapter 14]. The basic concept of PCA is to define the orthogonal projection of the spectra onto a lower dimensional linear space such that the total variance of the projected spectra is maximized. It amounts to diagonalization of the spectra covariance matrix

Σ = 1 n n i=1 (x i -x)(x i -x) t where x = 1 n n i=1 x i , (2)
or equivalently to calculation of the eigenvectors of Σ. However, in regression problems PCA is generally not satisfactory since only the explanatory variable X is considered while the dependent variable Y is not taken into account. Specific dimension reduction techniques have been developed for regression problems, such as PLS and Sliced Inverse Regression (SIR) [Li , 1991]. SIR consists of sorting the parameter values in increasing order and dividing them into H non-overlapping slices S h , h ∈ {1, . . . , H}. For each slice

D R A F T November 6, 2008, 2:58pm D R A F T
S h , a mean spectrum m h is calculated as the mean value of the parameter values that are varying in the slice. The orthogonal projection of the mean spectra onto a lower dimensional linear space is defined so that the variance of the projected mean spectra is maximized under the constraint that the total projected variance is one. This is equivalent to diagonalization of Σ -1 Γ , where Γ is the mean spectra covariance matrix given in equation ( 3). The SIR principle is illustrated in Figure 1.

Since inverse problems are generally ill-posed [START_REF] Aster | Parameter Estimation and Inverse Problems[END_REF]Tarantola, 2005], Σ is ill-conditioned, making its inversion difficult. To solve this problem, it is proposed to compute a Gaussian Regularized version of Sliced Inverse Regression (GRSIR). Theoretical foundations can be found in [Bernard-Michel et al., to appear, 2008]. The concept of this method is to include some prior information on the projections in order to dampen the effect of noise in the input data. The ill-posed problem is then replaced by a slightly perturbed well-posed problem that depends on a regularization parameter δ. Therefore, GRSIR computes the L eigenvectors corresponding to the L largest eigenvalues of (Σ 2 + δI p ) -1 ΣΓ where I p is the d × d identity matrix. In practice, GRSIR then requires the three following computational steps:

Step 1: Sort y i , i ∈ {1, . . . , n} into increasing order and divide into H non-overlapping slices S h , h = 1, . . . , H. If the LUT has been simulated for random values of Y , then slices are chosen such that each slice contains the same number of observations. If the LUT has been simulated for a fixed number of distinct parameters, the slices are chosen to coincide with these discrete values.

D R A F T November 6, 2008, 2:58pm D R A F T
Step 2: Compute the "between slices" covariance matrix of the means:

Γ = H h=1 n h n (m h -x)(m h -x) t with m h = 1 n h y i ∈S h x i , (3)
where n h denotes the number of observations in the slice S h .

Step 3: Estimate the GRSIR axes β 1 (δ), . . . , β L (δ) by computing the eigenvectors of (Σ 2 + δI p ) -1 ΣΓ. The first GRSIR axis, β 1 (δ), is determined by the eigenvector corresponding to the largest eigenvalue, the second GRSIR axis, β 2 (δ), is determined by the eigenvector corresponding to the second largest eigenvalue, and so on.

In Section 5.4, a criterion to choose the dimension L is given. Interestingly, the first axis can be viewed as a weighted function of the wavelengths, giving some knowledge about the wavelengths that carry information on the parameter (Section 5.5). The projections of the spectra on the axis β 1 (δ) is termed "reduced spectra".

Estimation of the functional relationship

One the spectra have been reduced, the relationship f has to be estimated. In the proposed approach, a piecewise linear interpolation is performed on the set of data points For each new spectrum x with a projection t = β 1 (δ), x , the estimated parameter value ŷ is then given by:

(m proj h , m param h ), h = 1, . . . ,
ŷ =            m param 1 if t ∈ -∞, m proj 1 m param h + (t -m proj h ) m param h+1 -m param h m proj h+1 -m proj h if t ∈ m proj h , m proj h+1 , h = 1, . . . , H m param H if t ∈ m proj H , +∞ (4) D R A F T November 6, 2008, 2:58pm D R A F T
An example of the application of GRSIR to the LUT is given in Figure 1. It shows the relationship between reduced spectra and the grain size of CO 2 ice and its estimation by a piecewise linear interpolation.

Choice of the Look-up table

In the simulated LUT, many spectra could be superfluous to the proper estimation of the parameters. For instance, considering Tdata, spectra with parameter values not included in the Tdata's parameter range should be removed from the LUT. Better estimations can be obtained with the reduced LUT [START_REF] Bernard-Michel | Estimation of Mars surface physical properties from hyperspectral images using sliced inverse regression[END_REF]. The difficulty is that, in practical cases, the range of variation of the parameters is unknown. A good strategy is to assumed a large LUT and to reduce it after the first estimation. However, the effectiveness of this technique in practical situations depends strongly on the quality of the first estimation.

In this paper, another approach is proposed that makes use of PCA to visualize the adequacy of the chosen LUT for a given set of observed spectra. The parameters of the observed spectra can be estimated only if their projections onto the first m PCA axes (deduced from the application of PCA to the LUT), coincides with the projection of the

Analyzing results on simulations

In this section, the competing methods are first presented. Then, the validation criteria and the parameters selection are detailed. Finally results on simulated data-sets are given.

Competing methods

k-NN: From the LUT one finds the k nearest spectra and fixes the estimated y as the mean parameter value of k nearest spectra parameter. The distance between two spectra is taken as the Euclidean distance: ox i 2 , where o is a spectrum from the image and

x i a spectrum from the LUT. In the experiments, k is fixed to one, so only one neighbor is used for the estimation.

Partial Least Squares regression:

The PLS method is closely related to PCA (orthogonal projection onto lower dimensional space) and GRSIR (y is accounted for). PLS searches for the projection of the explanatory variable x onto a lower dimensional space that maximizes the covariance between x and y: max β cov 2 y, β t x . PLS is parametrized by the size L of the subspace spanned by β. The regression in the subspace is necessarily linear and may limit the efficiency of the method if a non linear relationship exists between

x and y.

Support Vector Machines regression: SVM approximates the functional F :

y = F (x) using a solution of the form F (x) = n i=1 α i K(x, x i ) + b,

Validation criteria

To assess the relevance of GRSIR methodology, two aspects of the estimation were investigated: The accuracy of the estimates and the quality of the relationship between the reduced spectra and the parameters. To this end, two validation criteria were used 1. The Normalized Root Mean Square Errors (NRMSE):

NRMSE = n T i=1 ( ŷi T -y T i ) 2 n T i=1 (y i T -y T ) 2 with y T = 1 n T n T i=1 y T i (5)
The NRMSE quantifies the difference between the estimations ŷT i and the real values

y T i .
This measure is normalized enabling direct comparisons between several parameter estimations. The closer NRMSE is to zero, the more accurate are the predicted values.

2. The SIR Criterion (SIRC)

SIRC = β t ℓ (δ)Γβ ℓ (δ) β t ℓ (δ)Σβ ℓ (δ) (6)
The SIRC is the ratio between the "between-slices" variance β t ℓ (δ)Γβ ℓ (δ) of the projections of x i , i ∈ {1, . . . , n} on β ℓ (δ) and the total variance β t ℓ (δ)Σβ ℓ (δ) of these same projections.

It quantifies the quality of the relationship between projected spectra and parameters.

The closer SIRC is to 1, the better is the relationship.

D R A F T November 6, 2008, 2:58pm D R A F T

Choice of the parameters

For GRSIR, the quality of the estimation ŷi T depends on a regularization parameter δ (see Section 3.1). When δ increases, the functional relationship between projected spectra and parameters gets worse and consequently estimation errors increase. On the contrary, if δ is too small, then the estimation errors are considerable in the presence of noise because the problem is ill-posed. So, the choice of δ dictates to a compromise between improving the functional relationship and increasing estimation accuracies by regularization. In this article, δ is chosen for each parameter individually, the NRMSE criterion calculated between the parameter values from a validation data and their estimations is minimized.

The validation data is the look-up table itself, perturbed by multiGaussian noise representative of that affecting the OMEGA hyperspectral images (see Section 2.3). It is noted that, if there is no noise in the data, i.e. if the observed data exactly corresponds to spectra that could be simulated by radiative transfer model, then no regularization is required, and minimizing the NRMSE criterion for GRSIR yields a value δ close to zero.

For k-NN, since k is fixed to one, no parameter tuning is required. For PLS, the dimension L of the subspace is chosen to minimize the NRMSE on the validation set. For SVM, three parameters are fixed. ǫ is fixed to 0.01, while λ and γ are selected to minimize the NRMSE on the validation set. As it is typically done with SVM, the range of each component of Y has been stretched between 0 and 1 for the regression.

Choice of the EDR dimension

In order to choose the EDR dimension, the SIRC is calculated for each GRSIR axis β ℓ (δ) with ℓ ∈ {1, . . . , d}. As ℓ increases, the SIRC decreases rapidly and is almost zero after a few iterations. For instance, when applying GRSIR to the LUT, the SIRC for the

D R A F T November 6, 2008, 2:58pm D R A F T
proportion of CO 2 ice on the first GRSIR axis is 0.985. It indicates that the relationship of interest is captured by the first axis. Calculated the second axis, the SIRC drops to 0.1799 and then is close to zero for other axes. In this case, one can consider incorporating these axes in the regression step does not bring extra information and is unnecessary, and finally fixes the EDR dimension to one. Analyzing the SIRC over all parameters and two different physical models reveals that in most situations only one dimension is necessary to retrieve the model parameters. Hence, in this article L is fitted to one.

Analyzing the GRSIR axis

The spectral variability of the x i constituting the LUT arises from a complex interplay of the different input parameters y i of the model. In particular, two parameters can have a similar influence on the spectra for a certain range of values. Consequently, it is crucial to understand how the GRSIR method can untangle the dependencies in order to find an independent one-to-one functional relationship between spectra and parameters. Accordingly, attempt to correlate each axis β 1 (δ) (which is a vector of spectral weights) with the variability induced by the variation of the corresponding physical parameter. The other parameters are kept constant at mean values. The weights are individually applied to different levels of reflectance, depending on the associated wavelength, to calculate the reduced spectrum β t 1 (δ)x i . Hence, we find it more illustrative to represent the component wise multiplication (wavelength by wavelength) of each axis with a representative spectrum. An illustration is given in Figure 3 for the grain size of CO 2 ice. Our analysis shows that the axis for H 2 O abundance, dust abundance, and grain size of CO 2 ice are quite similar. A strong weight is assigned to 1.43 micron wavelength (the bottom of a very reliable and narrow CO 2 ice absorption band) and, to a lesser extent, to the interval

D R A F T November 6, 2008, 2:58pm D R A F T
between 1.47 and 1.54 microns (H 2 O ice feature), and wavelengths of 1.87 and 2.29 microns (weak CO 2 ice bands but very distinct). The three axes also show differences that allow differentiation between the functional relationships. A relatively strong weight at 2.38 microns on the right wing of the 2.34 micron band is a specificity of the proportion of water ice. The axis for the proportion of dust stresses very much the variation of the spectrum continuum around 1 micron and 1.77 micron that indeed strongly changes with dust concentration. No other parameter affects the continuum as much. The axis for the grain size of CO 2 ice maximizes the influence of the small plateau at 2.62 microns that varies very much with the latter parameter. The axis that is linked to H 2 O grain size is unique in the fact that it emphasizes, on the one hand, the continuum level around 1 micron, but not around 1.75 microns and, on the other hand, the entire spectral range between 2.35 and 2.65 microns. The main conclusions that can be drawn from this study are that, at least for the model presented in this paper, the GRSIR method has the ability to find a unique set of wavelengths where the variability of the spectrum is the most pertinent (but not necessary the highest) for the evaluation of a given parameter. One must note that these key wavelengths not only fall on the bottom of specific absorption bands for both H 2 O and CO 2 ices but also on specific parts of the continuum as well as on specific band wings.

Results

In this section, the experimental results are compared in terms of the NRMSE and SIRC (Table 2), and also with scatter plots of real and estimated parameters (Figure 4).

Except for k-NN, a spectra selection in the look-up table was performed before applying the different algorithms, as mentioned in Section 4.

D R A F T November 6, 2008, 2:58pm D R A F T
From Table 2, it is clear that the worst results are obtained with k-NN. GRSIR outperforms k-NN and PLS for most parameters. Problems with the estimation of the grain size of water ice may be due to the fact that not enough values are considered for simulations in the LUT (only 5). SVM provides the best estimation in terms of the NRMSE, but with a dramatic increase in processing time: Approximately 15 hours against about 1 minute versus k-NN or GRSIR.

Figure 4 shows the scatter plot of the estimated proportions of CO 2 ice with each method versus the real proportions. It appears that the range of the estimated proportions of CO 2 ice is much larger with k-NN than with GRSIR, PLS and SVM. PLS's scatter plot is slightly curved: PLS could not handle the non linear relationship between X and Y , while GRSIR was able to model it.

It is interesting to point out that GRSIR, as a first step, can determine the ranges the parameters of an observed image are varying. Then, a more appropriate LUT can be built in the estimated ranges with a higher density of parameter values. Applying GRSIR with this new extracted LUT leads to more accurate estimations comparable to SVM, see Table 3. Finally, the SIRC, always close to 1, is very satisfying. The PLS benefits from the new LUT while the SVM does not. However, for the SVM , the processing time is decreased since the number of samples in the LUT is reduced.

Retrieval of the physical parameters for the south polar cap of Mars

This section summarizes the first model inversions obtained by the different algorithms for hyperspectral images acquired by OMEGA during orbits 61, 103 and 41 (see Section

2). The study of these images leads to 30 different maps. All the maps can be found in [START_REF] Bernard-Michel | Estimation of Mars surface physical properties from hyperspectral images using sliced inverse regression[END_REF]. For brevity, only some are reported in the article.

D R A F T November 6, 2008, 2:58pm D R A F T
The model inversion on the image observed from orbit 103 by GRSIR, PLS, SVM and k-NN shows that GRSIR and SVM give very smooth mappings for all sets of parameters. An example is given in Figure 5, where it appears that the proportion of dust is nearly always estimated at 0.0003 with k-NN, whereas with GRSIR and SVM, the map is more detailed.

Some problems are encountered, with PLS: Negative values are estimated, making the map difficult to interpret. The estimated map with GRSIR is more detailed and leads to slightly different conclusions.

In particular, the area presenting a poor proportion of dust is more extended than with k-NN. Nevertheless if, globally, maps are much smoother and detailed with GRSIR, they never differ entirely from k-NN's.

Another interesting remark concerns the estimation of parameters in images 61 and 103 that represent approximately the same portion of the polar cap. The analysis should consequently give close estimations for each studied parameter regardless of the method.

Figure 7 shows the distribution of CO 2 grain size values obtained respectively by GRSIR, PLS on the left side (A) and SVM, k-NN on the right side (B) for both observations.

Figure 8 focuses on the distribution of the estimated proportion of water and dust and on the distribution of the estimated grain size of water and CO 2 with GRSIR and SVM.

First we consider the evolution of these histograms between observations 61 and 103 for a given method. For CO 2 grain size and, to a lesser extent for H 2 O grain size, a shift in

D R A F T November 6, 2008, 2:58pm D R A F T
the distribution of values can be noted that is increasing when using GRSIR, then SVM, and finally k-NN. The histograms of abundances for H 2 O ice and dust are relatively stable between 61 and 103 with SVM and GRSIR. The abundance for CO 2 ice would present a similar behavior since it is fully constrained (sum of abundances equals 1).

Second we note that the distributions of parameter values obtained by SVM are always shifted between to observation. The shift is always noticeable, sometimes considerable, and in addition systematically positive.

We explain these facts as follows. Observations 61 and 103 were acquired approximately 12 Mars days apart during a period of the year when the solar illumination was declining over the south pole. The conjugated effect of an increasingly grazing illumination over a slightly dusty atmosphere diminishes the spectral contrast of the observations, i.e. the overall level of reflectance of the spectra and on the CO 2 ice bands intensity and shape. This is very much comparable to the introduction of noise in a relative sense between observations 61 and 103. Since our remote sensing problem is partly ill-conditioned, a non regularized inversion method such as the k-NN will be much more sensitive to this noise than a regularized one such as the GRSIR or SVM. Even though regularization is thus applied to SVM, we note degraded performance when compared to GRSIR. For both methods, the estimation of the optimal value for the regularization parameter δ or λ is sensitive to the noise statistics applied to Tdata. In turn, this value determines the estimation of Y . As a consequence, uncertainties on the noise statistics propagate to bias on the estimation. We suspect that the choice of λ is less robust in the case of SVM than the choice of δ in the case of GRSIR.

D R A F T November 6, 2008, 2:58pm D R A F T
Finally, for the sake of validation, the estimated maps are compared with the Wavanglet approach developed in [START_REF] Douté | A Comprehensive Numerical Package for the Modeling of Mars Hyperspectral Images[END_REF]. "Wavanglet" is a supervised automatic detection method that identifies in hyperspectral images spectral features and thus produces distribution maps of chemical compounds. It uses three steps: 1. Selection of a library composed of reference spectra (the signature of the compounds to be detected); 2.

Application of a Daubechies wavelet transform to referenced spectra and determination of the wavelet subspace that best separates all referenced spectra; 3. In this selected subspace, calculation of the spectral angle between each spectrum of an observation and a given reference spectrum. In particular, this angle called the wavanglet angle, allows to quantify in a relative sense the spatial variations of the different compound abundances at the surface. The cosine of the wavanglet angle between each spectrum of the images and a reference spectrum of martian dust is presented Figure 9 . The closer it is to one, the greater the dust proportion is. A similar map is observed for the water proportion.

Globally, estimates of the latter quantities are more noisy with wavanglet than with GR-SIR and k-NN, especially for the dust proportion but they generally are in agreement with some exceptions. For example, in the image observed during orbit 103, k-NN, SVM, PLS and wavanglet methodologies display an area with strong proportion of dust in the lower right part of Figure 9(B), that is absent with GRSIR. In order to check if this area really contains more dust than other areas, the spectrum denoted by A in Figure 9 One can see that, as expected, the spectrum B is really different from the spectrum A.

On the other hand, spectra A and C are really similar showing that the area of interest does not contain as much dust as the wavanglet methodology predicts. On the contrary, estimations given by GRSIR are more coherent with this spectral analysis.

Conclusion

In this paper, a regularized version of Sliced Inverse Regression has been proposed to retrieve the physical parameters that best explain the spectra observed on Mars by the OMEGA imaging-spectrometer.

Results on simulations are promising, showing that estimations are accurate and most of the time better than the ones given by the k-nearest neighbors algorithm or PLS and close to those given by SVM. On a real data, maps are much smoother than with k-NN or PLS and seem to give a coherent mapping by comparison to the inversion of different hyperspectral images of the same portion of surface of Mars. If best results in terms of the NRMSE were obtained with SVM, the latter presents two drawbacks. First, the choice of the regularization parameter is less robust and, second, the processing time is 900 times more than GRSIR, thus making GRSIR approach more suitable for practical situations. The inversion of each new observed spectrum is really fast as well as the selection of regularization parameter. The main limitation of the proposed approach is that currently no uncertainties of the estimations are given when inverting a real image.

Experimental uncertainties could be computed based on simulations, but it supposes that the noise in the spectra has been well evaluated. If not, uncertainties will probably be underestimated. Some improvements could also be proposed to choose the regularization 1999. IGARSS '99 Proceedings. IEEE 1999International, 2, 1093-1095[START_REF] Douté | A Comprehensive Numerical Package for the Modeling of Mars Hyperspectral Images[END_REF] Functional relationship between reduced spectra on the first GRSIR axis and the grain size of CO 2 ice. This graphics illustrates the GRSIR methodology showing that the relationship is the best when the within slice variance is minimized or equivalently when the between slice variance is maximized. X-axis: reduced spectra from the learning database on the first GRSIR axis. Y-axis: Grain size of CO 2 ice.

D R A F T November 6, 2008, 2:58pm D R A F T X -27 sium,
D R A F T November 6, 2008, 2:58pm D R A F T -3 -2.5 -2 -1.5 -1 -0.5 0

  Training approaches: They have been recently considered in the study of terrestrial vegetation. A functional relationship Y = F (X) (1) between spectra and parameters is assumed. This relationship corresponds to the inverse of the physical model G in the forward problem. The idea is to use a LUT to estimate the underlying mathematical relationship F . This relationship then allow us to estimate the parameters of new spectra. The advantage of such a training approach is that, once the relationship has been established, it can be used for very large sets and for all new images

D

  OMEGA is one of the seven scientific instruments aboard the European spacecraft Mars Express mission, sent to orbit Mars in 2003. It was developed by IAS and LESIA (Observatoire de Paris) with the support of CNES, and with the participation of IFSI (Italy) and IKI (Russia). This visible and infrared instrument can scan most of Mars from orbit in order to observe gas and dust in the atmosphere and look for signs of specific materials such as silicates, carbonates and ice at the surface. It records the visible and infrared light reflected from the planet in the 0.5-5.2 micron wavelength range and with a ground resolution varying from 350 m to 10 km. Three OMEGA hyperspectral images acquired during orbits 41, 61 and 103 . They cover the high southern latitudes of Mars. The spatial resolution is approximately 2km per pixel and 184 wavelengths have been considered in the range 0.95-

  LUT itself. Conversely, to select the most appropriate spectra from the LUT for inversion, the strategy is to retain spectra from the LUT whose projections onto the PCA axes are close to projections of the spectra from the observed image (see Figure2.(A)). In the space spanned by the first m PCA axes, the distance between each projected LUT spectrum and its nearest neighbor from the projected spectra is computed. The histogram of these distances is considered as a mixture of n c Gaussian densities N (µ i , Σ i ) characterized by the proportions of the mixture π i , i ∈ {1, . . . , n c }, the expectation µ i and the covarianceD R A F T November 6, 2008, 2:58pm D R A F T matrix Σ i of each class i, i ∈ {1, . . . , n c }.The number of classes is chosen by the user, and the parameters can be estimated by the Expectation-Maximization algorithm (EM) after random or by the K-means initialization (for further details about mixture models and EM, see Hastie et al. [see 2003, Sections 6.8 and 8.5]). The algorithm is a simple iterative optimization process for computing the maximum likelihood estimate of the parameters. One can then calculate and maximize the posterior probability that a spectrum belongs to a particular class by observing the aforementioned distances. This leads to a classification of the spectra into n c classes: The class of LUT spectra that are far from the observed spectra, and those that are close. An example is given in Figure 2.(C) with the image observed from orbit 41. The histogram allows us to distinguish 3 classes. The third class corresponds well to spectra that belongs to both LUT and observed data. Note that in Figure 2.(B) some of the observed spectra are outside the projected LUT. Since the chosen physical model is not relevant to them, these spectra are removed from the observed data. Hence, in the PCA space, the histogram of the distance of each observed spectrum with its nearest neighbor in the selected sub-LUT facilitates the distinction of two classes: The invertible spectra and the non invertible ones (Figure 2.(D)). The non invertible spectra from orbit 41 correspond to pixels at the boundary of the CO 2 bright area and thus to pixels with a linear geographical mixture of terrains. Thus, the physical model used to simulate the data is not valid. Other examples can be found in the work of Bernard-Michel et al. [2007]. Experimental results in the next section show that estimations are more accurate when applying such a selection.

  where x i are samples from the training set, K a kernel function and (α i ) n i=1 , b are the parameters of F which are estimated during the training process [Scholkopf and Smola, 2002]. The kernel K is used to produce a non-linear function. One widely used kernel is the Gaussian Kernel : K(x i , x j ) = exp (-γ x ix j 2 ). The SVM training entails minimization of 1 n ℓ i=1 l F (x i ), y i + λ F 2 with respect to (α i ) n i=1 , b , and with l F (x), y = 0 if D R A F T November 6, 2008, 2:58pm D R A F T parameters need to be fitted: ǫ which controls the resolution of the estimation, λ which controls the smoothness of the solution and the kernel parameters (γ for the Gaussian kernel).

Figure 6

 6 Figure6shows the proportion of dust estimated by the different algorithms on the

  is selected, corresponding to the greatest proportion of dust, and compared to two other spectra from the cap. The first spectrum (B) has been chosen in a pure CO 2 area with very few dust. The second one (C) has been chosen in a area containing dust. According to k-NN, SVM, PLS and wavanglet results, this area should however contain less dust D R A F T November 6, 2008, 2:58pm D R A F T than the area containing spectrum A. The spectra A, B and C are presented in Figure 10.

Figure 2 .

 2 Figure 2. Selection of the useful spectra in the look-up table and of the invertible spectra

Figure 4 .Figure 5 .Figure 6 .

 456 Figure 4.Scatter plots of the proportion of CO 2 ice from the test data versus estimated

Figure 7 .Figure 8 .

 78 Figure 7. Comparison between the densities of the estimated grain sizes of CO 2 ice with PLS

  Figure 9.Cosines of the Wavanglet angle between the observed spectra and a reference

Table 1 .

 1 , WAVANGLET: An efficient supervised classifier for hyperspectral images, Geoscience and Remote Sensing, IEEE Transactions, Sampling strategy for the simulation of the look-up table. The range of variation is given for each varying parameter as well as the number of distinct values simulated. The increment between two values is constant.

	45 (5), 1374-1385.
	Scholkopf, B., and A. J. Smola (2002), Learning with Kernels: Support Vector Machines,
	Regularization, Optimization, and Beyond, MIT Press, Cambridge, MA.
	Shkuratov, Y., L. Starukhina, H. Hoffmann, and G. Arnold (1999), A model of spectral
	albedo of particulate surfaces: implications for optical properties of the moon, Icarus,
	137, 235-246.
	Tarantola, A. (2005), Inverse problem theory and model parameter estimation, Society for
	Industrial and Applied Mathematics.
	Weiss, M., F. Baret, R. Myneni, A. Pragnère, and Y. Knyazikhin (2000), Investigation of
	a model inversion technique to estimate canopy biophysical variables from spectral and
	directional reflectance data, Agronomie 20, pp. 3-22.

Table 2 .

 2 Validation criteria calculated on Tdata with GRSIR, k-NN, PLS and SVM. Note that the emphasized proportion has been fixed such that the sum of the proportions is one. For k-NN inversion the LUT contains 31500 spectra. For other methods, the LUT after selection contains 15407 spectra.

	Parameters	k-NN	PLS	SVM	GRSIR
		NRMSE NRMSE NRMSE NRMSE SIRC
	Proportion of water	0.86	0.52	0.17	0.40	0.90
	Proportion of CO 2	0.88	0.56	0.18	0.30	0.98
	Proportion of dust	0.44	0.36	0.11	0.17	0.99
	Grain size of water	0.43	0.44	0.17	0.54	0.84
	Grain size of CO 2	0.53	0.47	0.14	0.22	0.95
	CPU time	60s	181s	54,992s	58s

Table 3 .

 3 Validation criteria calculated on Tdata with GRSIR, PLS and SVM. The 3584 spectra that constitute the look-up table have been selected after a first inversion by GRSIR.

	Parameters	PLS	SVM	GRSIR
		NRMSE NRMSE NRMSE SIRC
	Proportion of water	0.35	0.18	0.27	0.92
	Proportion of CO 2	0.32	0.17	0.22	0.99
	Proportion of dust	0.23	0.13	0.13	0.99
	Grain size of water	0.45	0.23	0.39	0.92
	Grain size of CO 2	0.28	0.15	0.19	0.98
	CPU time	832s	2,750s	19s
	D R A F T		November 6, 2008, 2:58pm	D R A F T

table ,

 ,

		6
	first class
	Look-up table, second class Look-up table, third class	5.5			Look-up table Image orbit 41, first class
	Image orbit 41	5			Image orbit 41, second class
		4.5
		4
		3.5
		3
		-3 2.5	-2.5	-2	-1.5	-1	-0.5	0

  table and of the invertible spectra in the observed data. A: Selection of a look-up table. Projections of the observed spectra from orbit 41 and the look-up table on the 2 first PCA axes (PCA applied to the look-up table). 3 classes are identified. The third class is the retained look-up table. B: Selection of invertible spectra in the image from orbit 41. The second class is retained for inversion. C: Histogram of the distances between each spectrum of the look-up table and its nearest neighbor in the image from orbit 41. D: Histogram of the distances between each spectrum of the image from orbit 41and its nearest neighbor in the selected sub-look-up table. Plain line: Series of spectra extracted from the look-up table for CO 2 ice grain size varying between 30 and 165 millimeters, the other parameters being kept constant at mean values. Dotted-dash line: Component wise multiplication between GRSIR axis for the grain size of CO 2 ice and a representative spectrum (see Section 5.5).

					0.6
					0.5
					0.4
				Reflectance	0.1 0.2 0.3
					0
				-0.1	1	1.5	2	2.5	3	3.5	4
								Wavelength (microns)
	Figure 3.
		1								1
	Estimated proportion of CO2	0.995 0.996 0.997 0.998 0.999	k-NN						Estimated proportion of CO2	0.995 0.996 0.997 0.998 0.999	GRSIR
		0.994 0.994	0.995	0.996 Real proportion of CO2 0.997 0.998	0.999	1		0.994 0.994	0.995	0.996 Real proportion of CO2 0.997 0.998	0.999	1
				k-nn							GRSIR
		1								1
			PLS
	Estimated proportion of CO2	0.995 0.996 0.997 0.998 0.999							Estimated proportion of CO2	0.995 0.996 0.997 0.998 0.999
		0.994 0.994	0.995	0.996 Real proportion of CO2 0.997 0.998	0.999	1		0.994 0.994	0.995	0.996 Real proportion of CO2 0.997 0.998	0.999	1
	D R A F T				November 6, 2008, 2:58pm	D R A F T

SVM

PLS SVM

Acknowledgments. The authors would like to thank the reviewers for their many helpful comments. They also appreciate the contribution of Mr. Senan Doyle to this paper. This work is supported by a contract with CNES through its Groupe Système Solaire Program and by INRIA. It would not be possible also without the financial support of the "Agence Nationale de la Recherche" (French Research Agency) through its MDCO program ("Masse de Données et COnnaissances"). The Vahiné project was selected in 2007 under the reference ANR-07-MDCO-013.

Planetary Institute Conference Abstracts, Lunar and Planetary Institute Conference Abstracts, vol. 38, p. #1836. Durbha, S., R. King, andN. Younan (2007), Support vector machines regression for retrieval of leaf area index from multiangle imaging spectroradiometer, Remote Sensing of Environment, 107, 348-361. Hapke, B. (1993), Theory of reflectance and emittance spectroscopy, Topics in Remote Sensing, Cambridge, UK: Cambridge University Press, -c1993.