Irina Shatalina

Frederic Schmidt

Bortolino Saggin

Nicolas Gac

Matthieu Kowalski

Marco Giuranna

Analytical model and spectral correction of vibration effects on Fourier Transform Spectrometer

Keywords: PFS, Fourier Transform Spectrometer, Micro-vibration, Distortions, Spectroscopy, Analytical model, Semi-Blind Inverse problem

Sensitivity to mechanical vibrations of Fourier Transform Spectrometers (FTS) is a well-known phenomenon. It is especially critical for FTS devoted to atmospheric studies (like the Planetary Fourier Spectrometer (PFS) onboard Mars Express 2003), as absorption bands for the gases of low concentration are comparable with the generated instrument spectral noise. The adopted techniques for the vibration sensitivity reduction suffer of limitations in practical implementation, leaving residual modulations of the interferogram and the so-called ghosts in the spectra. Moreover as it is often impossible to measure the vibrations during the FTS measurement, the position and magnitude of these ghosts cannot be evaluated. Up to now the adopted ghost reduction techniques are mostly based on the averaging of spectra, because the disturbance phase is randomly distributed. This paper presents an innovative data treatment technique which allows single spectrum correction from distortions of unknown nature. Such a technique would increase the spatial resolution of the mapping process and becomes crucial when the desired information is linked to a particular mapping area associated to an individual spectrum.

The full study consists in the explicit analysis of the ghost formation and the post-processing algorithm based on the semiblind deconvolution method -an iterative numerical algorithm of the series of consecutive deconvolutions. The technique was tested on the data from the PFS and the algorithm proved to be consistent according to the selected efficiency criteria (coming from the available general information about the signal spectral shape).

INTRODUCTION

Sensitivity to mechanical vibrations of Fourier Transform Spectrometers is a well-known phenomenon. Vibration generated spectral distortions are especially critical for FTS devoted to atmospheric studies, as the gas composition determination is based on the matching of spectral features deriving from absorption bands that for the low concentration element are comparable with the instrument spectral noise. The adopted techniques aiming at the reduction of the vibration sensitivity (such as the constant optical path step sampling based on reference laser signal trigger), suffer of limitations in the practical implementation: time delays in the acquisition chain and cyclical fluctuations of optical misalignments lead to residual vibration induced modulations of the interferogram and eventually to the so-called ghosts in the spectra. Moreover as it is often impossible to measure the vibrations during the FTS measurement, the position and magnitude of these ghosts cannot be evaluated. Up to now the adopted ghost reduction techniques [START_REF] Saggin | Mechanical disturbances in Fourier spectrometers[END_REF][START_REF] Comolli | Evaluation of the sensitivity to mechanical vibrations of an IR Fourier spectrometer[END_REF][START_REF] Toyoshima | Transfer function of microvibrational disturbances on a satellite[END_REF] are mostly based on the averaging of spectra, being the disturbance phase randomly distributed. This paper presents an innovative data treatment technique able to deal with undesired spectral distortions of unknown nature allowing the single spectrum correction. Such a technique would increase the spatial resolution of the mapping process by avoiding the need of spectral averaging and becomes crucial when the desired information is linked to a particular mapping area associated with an individual spectrum.

As first step of the study the ghost nature and mechanisms of formation is analyzed in detail; the modulations of the signal due to the vibration generated misalignments (coming from deviations of the real instrument configuration from the ideal optical system 4 ) and that due to the non-constant OPD (Optical Path Difference) sampling step caused by the phase delay between reference and main signal acquisition chains [START_REF] Saggin | Mechanical disturbances in Fourier spectrometers[END_REF][START_REF] Zachor | Delay compensation: its effect in reducing sampling errors in Fourier spectroscopy[END_REF][START_REF] Cohen | Performance Degradation of a Michelson Interferometer Due to Random Sampling Errors[END_REF][START_REF] Bell | Spectral Errors Resulting from Random Sampling-Position Errors in Fourier Transform Spectroscopy[END_REF][START_REF] Palchetti | Spectral noise due to sampling errors in Fourier-transform spectroscopy[END_REF] are modeled. The result is the explicit mathematical description of the interferogram modulations generated by the mechanical vibrations. The obtained model allows implementing the second step of the research i.e. the post-processing algorithm.

The proposed approach of vibration effect correction is based on the semi-blind quasi-deconvolution method -an iterative numerical algorithm of the series of consecutive quasi-deconvolutions. The general problem of the data post-processing was subdivided into three separate sub-problems: definition of the vibration kernel, recovering of the original spectrum from the distorted one and the results validation.

The technique has been tested on the data from the PFS (Planetary Fourier Spectrometer) onboard the Mars Express ESA (European Space Agency) mission. The PFS is a FTIRS (Fourier Transform InfraRed Spectrometer) composed by two channels: Short Wavelength Channel (SWC) for the 1.2-5 µm range and Long Wavelength Channel (LWC) for 5-45 µm one. The interferometer scheme of PFS is a modified Michelson with swinging Cubic Corner Mirrors instead of translating flat ones [START_REF] Palchetti | Spectral noise due to sampling errors in Fourier-transform spectroscopy[END_REF][START_REF] Formisano | PFS: a Fourier spectrometer for the study of Martian atmosphere[END_REF][START_REF] Formisano | Infrared spectrometer PFS for the Mars94 orbiter[END_REF][START_REF] Formisano | The planetary Fourier spectrometer (PFS) for the orbiter of the spacecraft Mars96[END_REF][START_REF] Formisano | PFS for Mars Express: a new approach to study Martian atmosphere[END_REF] . The problem of microvibrations is mostly referred to the SWC data [START_REF] Giuranna | Calibration of the Planetary Fourier Spectrometer short wavelength channel[END_REF][START_REF] Giuranna | Calibration of the Planetary Fourier Spectrometer long wavelength channel[END_REF] .

The key-point of the procedure was the kernel determination. The lack of information about the vibrations and their nature is the main difficulty: frequency, intensity and phase with respect to the main signal interferogram are only statistically known from PFS onboard self-check measurements that are performed at most once per orbit. The iterative procedure of kernel definition was performed on smoothed spectral data, each iteration loop including constrained quasideconvolutions of measured distorted spectrum with signal and kernel estimates. Proposed general constraints were the spectrum smoothness and kernel sparsity. The final deconvolution step was performed on the unsmoothed data, with the aim to retrieve the original spectrum from the distorted one.

Efficiency of this technique strongly depends on the initial guesses for the semi-blind deconvolution input. Also the correct definition of the deconvolution constraints is important. Utilization of additional information about the expected spectral features (e.g. some gas absorption bands, regions of detector low responsivity, any known vibration components, etc.) makes proposed algorithm more robust.

Finally the algorithm proved to be consistent according to the selected efficiency criteria (coming from the available general information about the signal spectral shape).

ANALYTICAL DIRECT MODEL

Vibration effect: Direct Model, general formulation

Mechanical disturbances modulate the interferogram measurements in two ways 1 :

• by creating sampling step errors due to the different phase delays in the acquisition systems;

• by generating cyclic misalignments in the optical system.

In the following the mechanical disturbance effect is modeled for the case of a monochromatic radiation source and the disturbance being represented by one pure harmonic component.

The modulated part of the monochromatic source interferogram in the ideal interferometer can be expressed as ::

= cos 2π (1)
, where x k is the OPD at the k ith zero-crossing of the reference signal, σ s is the wavenumber of the observed line and m is the modulation factor referred to the misalignments of the optical system from its ideal position, I s the source halfintensity.

The pure harmonic disturbance can be described as follows:

= sin + (2),
where y is the vector of the vibration component generated at the source, being = 2π its angular frequency and ϕ d its phase. The component phase generally is of a random nature with respect to single interferogram (while it is unambiguously defined for each ω d ), with uniform probability distribution in the ±π range for the whole set of measurements.

We represent the mirror speed variations and optical misalignments in the PFS in the instrument coordinate system, taking one axis along the optical axis of cube corner (x), and the others perpendicular to it (u and w):

∆ = sin + (3)
∆ = sin + * (4)
The component phases ϕ u* d and ϕ x d can be different, as the transfer function of vibration propagation from the source to the instrument will be in general different for each direction.

The modulation factor m is a function of Cubic Corner Mirror (CCM) position error u (in any direction perpendicular to its optical axis) and in the first order approximation it can be written in the following form [START_REF] Saggin | Mechanical disturbances in Fourier spectrometers[END_REF] :

, ≅ # , $ + % , $ • sin' + ( (5)
,

where a is proportional to the vibration amplitude u d and depends on the wavenumber σ s of the modulated component and the initial interferometer alignment quality u o (zero for a perfectly aligned instrument).

For the vibration component along theCCM axis we have first order approximation for the sampling steps 1 :

= ) * + , + -. / 0 + -# / 0 cos + (6),
where T D is the time delay in the sampling chain, λ r is the reference laser wavelength, t k is the time of the k th sampling and v o is the vibration amplitude expressed as speed (v o = ω d x d ).

The interferogram now can be rewritten substituting Eq. ( 5) and Eq. ( 6) in Eq. ( 1):

= 1 # , $ + % , $ sin + 2 × cos 42π 5 6* + , + / 0 -. 7 + 2π -# / 0 cos' + (8 (7)
Applied substitutions:

• ≅ )9 : / 2-. ; • = 2π -. / 0 ; • /-. = 2π /-. = 2π , = /-. k.
Developing cosine into Taylor series up to 1 st order the Eq. ( 7) is rewritten as follows:

= < # , $ + % , $ sin 42π 5

Later on the m o and a coefficients dependence on wavenumber σ s will be neglected. Such an approximation is possible due to the fact that in the ghost component sampling error term is dominant (demonstrated [START_REF] Comolli | Mechanical Disturbances in Fourier Spectrometers[END_REF] in the ghost studies on PFS spectra).

Having "2π -# / 0 %" as second order infinitesimal factor, the product development goes as follows:

= # cos 42π 5

Applying substitutions A # , A B %/ 2 # , A , C-# / 0 and rewriting Eq. ( 9) in the wavenumber domain, the measured signal spectrum becomes:

A exp G δ > A < A B exp1G 2 A , exp 4G I , 8= δJ > K A < A B exp1G > 2 A , exp 4G > I , 8= δJ > > K (10)
After some mathematical reorganization (and expressing the unknown amplitudes and phases of ghost components as B, C and ϕ σB , ϕ σC ) the previous Eq. ( 10) can be rewritten in the simplified form:

A exp'G ( δ > 1 A exp'G ( δ > 2 * L Mexp G N OJ > P K Qexp G R OJ > > KS TUVWX 1 TUVWX 2 * Y Z [ (11)
,

where I signl (σ) is the undistorted spectrum and K σd (σ) is the single harmonic vibration kernel, "δ" and "*" are respectively the Dirac's function and the convolution operator.

For the generic non-monochromatic source of input radiation I continuum (σ) and arbitrary vibration kernel shape K(σ), the Eq. ( 11) can be extended as following:

\#V]TV . J \#V]TV . K * Y (12)
Y ∑ <M _ exp 4G N ` 8 O 4 > 5 _ 78 _ Q _ exp 4G R ` 8 O 4 > > _ 8= (13)

Vibration effect: Direct Model, PFS data

Typical PFS raw spectra is given on the Fig. 1. It is possible to subdivide the whole spectrum into two parts to be treated separately:

• signal domain in the wavenumber range of 0 to 5000 cm -1 ;

• laser line domain in the wavenumber range above 5000 cm -1 .

The actual energy is present in the signal domain, above the 1700 cm -1 , due to the used detector responsivity curve [START_REF] Giuranna | Calibration of the Planetary Fourier Spectrometer short wavelength channel[END_REF] . Below 1700 cm -1 there are only ghosts of the continuum present in the spectrum and no meaningful signal. In the region above 5000 cm -1 the continuum Signal-to-Noise Ratio becomes low due to the strong decrease of the detector responsivity [START_REF] Giuranna | Calibration of the Planetary Fourier Spectrometer short wavelength channel[END_REF] , and thus the laser line domain is characterized mostly by the laser line shape and its ghosts.

In the current research the signal domain was treated only. Actually, after the orbit 662 the SWC reference laser was switched off, leaving the spectrum region above 5000cm -1 close to the zero value.

Calling back the Eq. (12-13), we rewrite the above expression for the measured raw PFS spectrum I PFS (σ), having the input spectrum I Mars (σ) and considering the wavenumber in the second term to be constant, σ ≅ σ k = 2500 cm -1 . Such an approximation is possible due to the fact that the most part of significant signal is actually concentrated in the 2000..3000 cm -1 wavenumber region (Fig. 1). We also include the error term ξ, accounting for used approximation in convolution kernel modeling and for other distortions of spectrum, besides the modulation by mechanical vibrations. 2)

Y abc = O + M expJG N K + Q expJG R K (15.3)
Generally the B(σ) and C(σ) functions are sparse due to the vibration kernel nature: it is present by a set of separate components, each appearing as the bell-shaped function in the wavenumber domain, although being a pure harmonic in the time domain. This peculiarity is caused by the uneven sampling: because of non-constant mirror speed, the equal OPD step does not correspond to the equal time step. The estimation of those functions will be performed by the inversion procedure described in the next section.

PFS SPECTRA CORRECTION TECHNIQUE THOUGH THE INVERSE PROBLEM SOLUTION

Problem Inversion

Starting from the direct model, represented by the Eq. ( 15.2), we proceed to the inversion [START_REF] Idier | Bayesian Approach to Inverse Problems[END_REF] of the problem (estimation of the Martian spectrum) through a semi-blind deconvolution method. For that purpose we introduce a Loss Function LF, whose minimum would provide the estimates of Martian spectrum h dW: and vibration kernel Y i jkl . This function not only quantifies the fitting of the estimated data to the original model, but also includes additional terms characterizing the available information about the physical data, such as the sparsity of vibration kernel and smoothness of the Martian spectrum. h dW: , Y i jkl = arg min q rs+ ,e tuv wx dW: , Y jkl (16.1)

wx = B , ‖ jkl -Y jkl * dW: ‖ , , + 9 z ‖Y jkl ‖ B + * rs+ , ‖{ * dW: ‖ , , (16.2),
where λ K and λ Mars are the regularization coefficients for the kernel sparsity and spectrum smoothness correspondingly and D is the discrete derivation operator.

Iterative procedure for the inverse problem solution

The solution of the inverse model, represented by the Eq. ( 16), is performed iteratively, starting from the initial estimates of spectrum h dW: Since the spectra h dW: } is estimated without knowing the kernel Y i jkl } , the deconvolution is blind. But it has to be noticed, that as the very first step of the proposed algorithm is the kernel estimation Y i jkl B , then the crucial initial parameter for the efficient algorithm is the initial guess of Martian spectrum h dW:

|

, which has to possess the properties of the real expected solution, otherwise algorithm might converge to some non-physical result. That's why we can qualify this deconvolution as semi-blind. The initialization of kernel Y i jkl | instead does not affect the quality of results, but only the algorithm convergence rate.

Initialization choices for the semi-blind-deconvolution method

Vibration Kernel Initialization. As it was mentioned above the quality of kernel initialization provides faster convergence but is not a crucial parameter for the whole algorithm. As an initial guess of kernel for the treated PFS data (Fig. 1: Orbit #0032, spectrum #106) we used an approximate kernel estimation from the laser line ghosts by their deconvolution by the laser Dirac in accordance with Eq. (15.2). For the orbits after 662 such an initialization becomes impossible due to the SWC reference laser switching off, and the proposed initialization would be the zero-pattern.

Initial Martian spectrum guess. The Martian spectrum for initial guess estimated by two Planck functions, representing the Martian thermal emission and the solar energy reflected by the Martian surface, with the CO 2 absorption band gap in the region of 2200..2400 cm -1 . The Martian temperature for the first Planck function was estimated from the 2500..3000 cm -1 region of the acquired spectrum, while the second Planck function was scaled to the 3800..4200 cm -1 region. Finally to obtain the initial guess of spectrum h dW: | we calibrated data by the known detector responsivity and deep space measurements. [START_REF] Giuranna | Calibration of the Planetary Fourier Spectrometer short wavelength channel[END_REF] What regards the phase of the initial guess, we took it similar to one of the acquired spectrum in the region of high expected signal, and an extrapolation for the regions of expected ghosts.

To compensate the iterative procedure sensitivity to high frequency noise, both PFS actual data I PFS and the initial guess , where spectral resolution of acquired data ∆σ is equal to 1.04 cm -1 .

The filtered synthetic initial guess of spectrum is shown on Fig. 3

Criteria for result evaluation

The proposed criteria for the result quality estimation are the following ones:

• distance between acquired PFS spectrum I PFS and estimated PFS spectrum h jkl ~ h dW: ~ * Y i jkl ~, as if the model is valid the estimate h jkl ~ has to be as close as possible to I PFS ;

• removal of ghosts in the signal domain below 5000 cm -1 .

RESULTS

Estimated spectra and vibration kernel

For Mars spectrum estimation coming from the Orbit #0032, spectrum #106, the estimation of signal (after the additional step run on unfiltered data) is shown on the Fig. 4.

Results validation

As it is seen from Fig. 4, the lack of fit between acquired PFS spectrum I PFS and the estimated one h jkl

•€}•' h dW: •€}•' * Y i jkl •€}•' is very small (∼10 -6
), proving the solution to agree with actual measured data.

Also the ghost removal in the low wavenumber signal domain (below 1500cm -1 ) is efficient: the reduction of the energy in the above region is of factor of ∼2.

CONCLUSIONS

The analytical formulation of modulation effect of mechanical disturbances and the proposed problem inversion makes it possible to perform the correction of the above effect on the PFS data, as well as data coming from any other FTS instrument. Nevertheless, the current approach has some drawbacks:

• the algorithm does not permit to evaluate the high frequency kernel components effect and thus does not perform the correction of narrow spectral features; an improved implementation should correct this phenomena;

• due to the linear assumption of the proposed algorithm, the model treats only part of the information, available from the PFS spectra (i.e. the signal region and not the laser line ghosts one); a non-linear model should be proposed to estimate the vibration kernel and the spectrum through the whole PFS spectral region;

• the implemented iterative procedure is controlled by two regularization parameters and is not optimized in terms of time consumption; robustness of the algorithm shall be improved to be applied on a large dataset.

The next step would be the extension of the direct model also on the laser line region. This will provide the additional criteria for the validation of results: comparison of the evaluated laser line ghost with the actually present ones. Another model modification will be pointed towards the improvement of the algorithm robustness and efficiency, in order to make possible the automatic data treatment procedure.

Figure 1 .

 1 Figure 1. Typical PFS measurement in SWC (data from the Symmetrized spectrum #106, ORBIT0032).

|

  and kernel Y i jkl | , and at n th iteration determining the estimates h dW: } and Y i jkl } . The output of algorithm after N iterations would be the final estimates of the kernel Y i jkl ~ and spectrum h dW: ~. The iterative process schematically is shown on Fig. 2.

Figure 2 .

 2 Figure 2. Iterative procedure for the inverse problem solution.

  a low-pass filter of cut-off frequency B ,|∆Z

Figure 3 .

 3 Figure 3. Filtered PFS measured spectrum and the initial guess h dW: | .

Figure 4 .

 4 Figure 4. Final results for the spectrum #106 ORB0032 .


Achraf Ben Romdhane
email: achraf.ramdhan@gmail.com

Salma Jamoussi
email: jamoussi@gmail.com

Abdelmajid Ben Hamadou
email: abdelmajid.benhamadou@gmail.com

Kamel Smaïli
email: smaili@loria.fr

Phrase-Based Language Model in Statistical Machine Translation

Keywords: Machine Translation, Phrases, Phrase based language model, Decoding optimization

As one of the most important modules in statistical machine translation (SMT), language model measures whether one translation hypothesis is more grammatically correct than other hypotheses. Currently the state-of-the-art SMT systems use standard word n-gram models, whereas the translation model is phrase-based. In this paper, the idea is to use a phrase-based language model. For that, target portion of the translation table are retrieved and used to rewrite the training corpus and to calculate a phrase n-gram language model. In this work, we perform experiments with two language models word-based (WBLM) and phrase-based (PBLM). The different SMT are trained with three optimization algorithms MERT, MIRA and PRO. Thus, the PBLM systems are compared to the baseline system in terms of BLUE and TER. The experimental results show that the use of a phrase-based language model in SMT can improve results and is especially able to reduce the error rate.

Introduction

Machine translation systems have evolved since several decades from the use of a word to the use of a sequence of words (phrases) as basic units for translation. Currently, all the Statistical Machine Translation (SMT) systems are based on phrases. Succinctly, in the decoding step, the source sentence is segmented into phrases, each phrase is then translated into the target language and finally phrases are reordered [START_REF] Koehn | Statistical phrase-based translation[END_REF]. At each step of the decoding phase, translation hypotheses are created and expanded until all words of the source sentence are covered. The expansion step produces a huge number of possible translations which are constrained by a cost estimation depending on many parameters such as the language model and the translation model probabilities. To achieve good translation quality, SMT researchers make a lot of effort in improving the translation model which moved from the original single-word-based to phrase-based model [START_REF] Koehn | Statistical phrase-based translation[END_REF], in order to better capture the context dependencies of words in the translation process. Despite the improvements made in language modelling [START_REF] Sarikaya | Joint Morphological-Lexical Language Modeling for Machine Translation[END_REF], [START_REF] Khalilov | Improving target language modeling techniques for statistical machine translation[END_REF], the state-of-the-art SMT systems still use standard word n-gram models.

The idea, in this paper is to enhance the quality of SMT systems by improving their Language Models (LM). For that, we propose to use a phrase-based LM. This kind of models has already shown good performances in speech recognition tasks [START_REF] Zitouni | Statistical language modeling based on variablelength sequences[END_REF], [START_REF] Deligne | Language modeling by variable length sequences: Theoretical formulation and evaluation of multigrams[END_REF], [START_REF] Kuo | Phrase-based language models for speech recognition[END_REF], [START_REF] Lamel | Speech Recognition Of European Languages[END_REF] and hopefully it can improve the quality of SMT systems as well.

In SMT, few research work has been done for designing phrase-based language models. Many researchers attempt to improve the translation system with Continuous-Space Language Models (CSLM) [START_REF] Schwenk | Continuous space language models[END_REF], [START_REF] Le | Structured output layer neural network language mode[END_REF]. These works have shown that CSLM can improve the SMT when compared to back-off n-gram language models (BNLM). Schwenk in [START_REF] Schwenk | Continuous space language models[END_REF] uses a CSLM in reranking step of the n-best translation hypotheses. In this approach, the authors uses two differents language models. BNLM is used in decoding step to produce a n-best translation. Then, a CSLM is used to rerank those translations. In the same way, [START_REF] Wang | Converting Continuous-Space Language Models into N-Gram Language Models for Statistical Machine Translation[END_REF] proposes a method for converting CSLM into BNLM that will be used directly in SMT decoding.

Recently, [START_REF] Xu | Phrase based language model for statistical machine translation[END_REF] proposes a phrase-based language model for statistical machine translation. This model considers the sentence segmentation into phrases as a new variable to estimate sentence probability. For the sentence segmentation, all the word sequences are considered as phrases. This leads to huge phrases vocabulary and increases the number of possible segmentation solutions. In the same way, [START_REF] Jiajun | Beyond Word-based Language Model in Statistical Machine Translation[END_REF] propose a phrase-based LM using a limited set of sequences. This work, has shown that the Kneser-Ney phrase-based LM cannot outperform the Kneser-Ney word-based LM. This can be explained by the strict phrases vocabulary size constraint and the data sparsity problem. In [START_REF] Nasri | Statistical Machine Translation Improvements based on Phrase Selection[END_REF], the authors showed that the phrase-based language model could improve the translation quality of SMT. In this work, the Interlingual Triggers based on Conditional Mutual Information (CMI) are used to extract a translation table. The target sequences of this table are used for segmenting the training target corpus phrases in order to learn a Phrase-Based Language Model (PBLM).

In this paper, we propose a new method for using a phrase-based language model in machine translation (SMT-PBLM) and we argue that this model can improve the SMT quality. This method is not limited on a small set of phrases. Indeed, we use all the phrases of translation table for rewriting the training corpora where the words are joined to constitute phrases. The new corpora is used to train the phrase-based language model. Then, we tried three optimization algorithms (MERT, MIRA and PRO) to tune our translation system. Finally, we developed an optimization method in order to better take into account the change of the structure of lexical units in our SMT-PBLM.

In Section 2, we present the phrase-based language model. Then, in section 3, we give an overview of the sentence extraction method and we present how to define a phrase-based training corpus in section 4. A description of the used corpora and the results achieved are presented and discussed in section 5. We end with a conclusion which points out the strength of our method and gives some tracks about future work.

Phrase-Based Language Model

Language models play an important role in SMT, it measures whether one translation hypothesis is grammatically better than other hypotheses. A statistical language model assigns a probability P (e m 1 ) to string of words e m 1 = w 1 , w 2 ..., w m . In the case of n-gram model the probability is calculated as follows:

P (e m 1 ) = m i=1 P (w i |w i-1 1 ) (1)
after making the Markov assumption which retains the most recent n -1 words to predict the next word.

m i=1 P (w i |w i-1 1 ) ≈ m i=1 P (w i |w i-1 i-n+1 ) (2)
Currently, most of the state-of-the-art MT systems use word-based n-gram language model, considers a sentence as a suite of words, whereas phrase-based translation model considers a sentence as a suite of phrases. In this paper, we propose a phrase-based language model in order to use the same units for both language and translation models.

In phrase-based SMT, the target translation hypothesis is a suite of translation options. Each translation option represents a word or phrase. A phrase-based language model considers the n -1 previous phrases to predict the next phrase. Given a translation hypothesis h = h 1 h 2 ...h n , the phrase-based language model calculates the following probability:

P (h) = P (h 1 h 2 ...h n ) = p(h 1 )p(h 2 |h 1 )...p(h n |h 1 ...h n-1 ) (3)
where h i is a translation option (word or phrase). The phrase based LM can capture longer relationship easily e.g. a 5-gram phrase-based language model could take into account phrases up to 40 words (in the case of a 5-gram where each gram is composed of a phrase of 8 words). A word-based language model is actually a special case of a phrase-based language model, where each phrase is reduced to its smallest unit: a word.

Extracting Phrases

The goal is to select automatically a set of phrases that will be extracted and integrated into the training corpus and consequently into the language model. In [START_REF] Kuo | Phrase-based language models for speech recognition[END_REF], the authors determined the sequences which reduce the perplexity of a language model. This, is an iterative process that merges at each iteration, word pairs into a single unit (phrase) such as "white house" which will be transformed into "white house". At each iteration, the best phrases according to a selection criterion are retained and used into the training corpus. This algorithm has been applied to a monolingual corpus and at the end of this process the authors obtain a set of phrases and a training corpus re-written in terms of words and phrases.

Other work focuses on the extraction of sequences based on a bilingual corpus. There are two main methods. One is based on word-to-word alignment [START_REF] Koehn | Statistical phrase-based translation[END_REF], thus the phrases are selected with a set of heuristics. Any contiguous source words must be a translation of any contiguous target words under the condition that words from both sides are mapped to each other. The second method extract phrases without word alignment. Lavecchia et al. in [START_REF] Lavecchia | Discovering phrases in machine translation by simulated annealing[END_REF] proposed to determine correlations between words coming from two different languages. This method use the intra-lingual triggers to extract the source phrases then the inter-lingual triggers associates to each source phrase of n words a set of target sequences of variable size m. In fact, for each source phrase of k words, one or more target sequences of length k ± ∆k are retained.

In this work we propose to use Moses toolkit to train phrase-based translation model from parallel corpora [START_REF] Och | A Systematic Comparison of Various Statistical Alignment Models[END_REF]. Then we collect all target phrases from translation table and use them to train the phrase-based language model. Afterward we merge the words of the target phrases and we replace these in the translation table by the new phrases without modifying the initial parameters (P (e|f ),P (f |e),lex(e|f ),lex(f |e)) of the translation table (see 1). This is done due to the fact that the translation process would propose them such as translations which will be used in the target language model. Table 1 illustrates some examples of phrases in the new translation table. It is a crucial issue to rewrite adequately the training corpus in order to train a phrase-based language model. Due to the high number of phrases in the translation table, it is necessary to take into account the issue of the overlapping phrases in the segmentation phase of the same sentence. For example in the sentence "a lot of sociologists actually are quite disappointed" and the following list of sequences (lot of, a lot, a lot of, are quite, actually are) what are the phrases to use for merging words in the above sentence? actually are or are quite ? At this point, there are two possible solutions. The first solution consists in finding the segmentation that minimizes the likelihood or the perplexity of the sentence. This ensures a good quality language model in terms of perplexity but increases the problem of OOV words because the minimization of the perplexity in the experiments we achieved, reduces the number of phrases used. In fact, only those that guarantee an improvement of the perplexity are selected. The second solution do not segment in order to avoid the errors of wrong segmentation. What we propose here is to increase the size of the training corpus by inserting all the possible segmentations. Indeed, we can produce all the possible hypotheses directed by using the right part phrases of the translation table. For example, for the sentence "a lot of sociologists actually are quite disappointed", there are four possible segmentations with the list of sequences given in the above example:

a lot of sociologists actually are quite disappointed a lot of sociologists actually are quite disappointed a lot of sociologists actually are quite disappointed a lot of sociologists actually are quite disappointed a lot of sociologists actually are quite disappointed a lot of sociologists actually are quite disappointed

This method ensured that all the sentences of the translation table will be integrated in the training corpus of the language model. Thus, we get a bilingual training corpus where the target corpus is written in terms of words and phrases containing some redundancy of sentences. This redundancy would be useful for sequences that have low number of occurrences. The obtained corpus is then used to train the phrase-based language model.

Experiments

This section describes the performance of the proposed language model in a machine translation task. We use the IWSLT2010 test data for the French to English translation system [START_REF] Michael | Overview of the IWSLT 2010 Evaluation Campaign[END_REF]. Table 2 provides relevant statistics about the data used. The language models (word and phrase based) have been trained with SRILM toolkit [START_REF] Stolcke | SRILM: An Extensible Language Modeling Toolkit[END_REF]. The word alignment of the parallel corpora is generated by GIZA++ Toolkit [START_REF] Och | A Systematic Comparison of Various Statistical Alignment Models[END_REF] in both directions. Afterwards, the alignments are combined using the grow-diag-final-and heuristic to obtain symetric word alignment model [START_REF] Koehn | Statistical phrase-based translation[END_REF]. For decoding we used Moses toolkit [START_REF] Koehn | Moses: Open Source Toolkit for Statistical Machine Translation[END_REF] and finally an optimization algorithm is applied to estimate the optimal value of each weight on the development data set. Eight SMT features are used namely: Bidirectional phrase translation probability (p(e|f ), p(f |e)), Bidirectional lexical probability (lex(e|f ), lex(f |e)), Phrase penalty, Word penalty, Distortion, language model (word or phrase based). It should be noted that the system parameters were trained on the development corpus dev2010 and the test is performed on the corpus tst2010. In this evaluation, we compare the performance of the different language models in a translation task in terms of BLEU [START_REF] Papineni | A method for automatic evaluation of machine translation[END_REF], TER [START_REF] Snover | A Study of Translation Edit Rate with Targeted Human Annotation[END_REF] and (TER-BLEU)/2. BLEU is an n-gram precision metric, i.e. higher values are better, while TER is an error rate, i.e. lower values are better and (TER-BLEU)/2 aims at simultaneously maximizing BLEU and minimizing TER. Our SMT-PBLM is compared to a baseline SMT. For the baseline system, we evaluate 3 language models (5, 4 and 3-grams standard word-based language models). For the SMT-PBLM, we use 5, 4, 3 and 2 grams. In this work, we observed that the optimization algorithms are not well suited to the use of phrase-based language model. That is why we used, obviously the most widely algorithm MERT3 , but also we tried two other optimizations algorithms PRO4 and MIRA 5 . Finally, we were not satisfied by the results, we developed an optimization method inspired from the greedy search algorithm that will be explained in section (5.3).

MERT optimization

An optimization procedure in SMT attempts to improve translation quality by searching the weights minimizing a given error measure, or equivalently maximizing a given translation metric. MERT, proposed by [START_REF] Och | A Systematic Comparison of Various Statistical Alignment Models[END_REF] obeys to this concept. Table 3 shows the results obtained with SMT-WBLM and SMT-PBLM where the optimization task is realized with MERT algorithm.

In terms of BLUE we find that the word-based language model is more efficient than the phrase-based language model. Table 3 shows that it could be acceptale to use just a bigram of phrase-based language model since the results are similar to 4-grams PBLM. This could be explained by the fact that, rarely succession of 5 or 4-grams of phrase have been found on the test corpus. In terms of TER, SMT-PBLM gives better results than the best SMT-WBLM. The improvement is of 0.42 in comparison to SMT-WBLM (4-grams) and of 0.55 in comparison to SMT-WBLM (5-grams). This means that it is easier for a human being to correct a translation achieved by SMT-PBLM than with SMT-WBLM.

Overall, the combined score (TER-BLEU)/2 is substantially improved.

For the test corpus, in terms of BLEU the difference is of 0.54, while for the development it is of 0.85. This could be explained by the fact that the MERT algorithm is not able to handle adequatly the parameters of the SMT-PBLM. In fact phrases are composed of words which have been concatenated. Consequently, they are considered such as words and not as sequence of words. Table 4 shows the weight of each parameter obtained by MERT. For SMT-PBLM, we notice that the weight of the phrase penalty (PP) is much greater than the weight of the word penalty (WP). This means that the decoder promotes the translations that have a large number of sequences and small number of words. Unlike the baseline system that increases the number of words and minimize the number of sequences in the translation. Consequently, we need an other algorithm of optimization which can correctly handle the phrases of our language model. In the following, we will test other algorithms of optimizations. In Table 5, we presents some translations achieved by both MT systems, we can observe that globally our system achieves well translated sentences, and sometimes better and more understandable than the baseline system. Furthermore, we can see that the last two sentences produced by our SMT-PBLM are closer to the reference sentencs than those given by SMT-WBLM. the second factor is the services we use . SMT-WBLM the second aspect is about the services that we use . SMT-PBLM the second aspect is the services we use . SRC ces dernières années , nous avons commencé à apprendre des choses sur le bonheur des deux " moi " . REF so in recent years , we have begun to learn about the happiness of the two selves . SMT-WBLM in the last few years , we 've started to learn things about happiness of the two " me . " SMT-PBLM in recent years , we started learn things about happiness of the two " me . "

MIRA and PRO optimization

Alternative discriminative parameters training algorithms for SMT have been proposed in the last few years, such as using a margin infused relaxed algorithm (MIRA) [START_REF] Taskar | Learning structured prediction models: a large margin approach[END_REF] and pairwise ranking optimization (PRO) [START_REF] Hopkins | Tuning as Ranking[END_REF]. PRO generates lists of k-best candidate translations for each sentence, and tunes the weight vector for those candidates. This method seeks the weight vector which classifies pairs of candidate translations into correctly ordered and incorrectly ordered, based on the scoring function. Tables 6 and7 show the obtained results by the baseline and the SMT-PBLM where the optimization task is realized respectively with PRO and MIRA. From Table 6 we see that MIRA algorithm allows us to improve the results of SMT-PBLM. Indeed, we achieve a BLEU of 32.80 with MIRA, when we obtained a BLEU of 32.69 with MERT. An improvement of the results with MIRA has been achieved for BLEU and TER. The improved results are presented in bold style in Table 6. This shows that it is possible to improve the SMT-PBLM by using the adequate optimization algorithm. From Table 7, we can see that PRO on the baseline system has greatly improved results compared to MERT and MIRA algorithms. But, for the SMT-PBLM we notice a diminution of 0.2 points BLEU compared to MERT and 0.4 points BLEU compared to MIRA. This decrease is explained by the fact that PRO applies a clustering method on the n-best distinct hypothesis (in our experiment n = 100) to extract the features weight. This constraint disadvantages SMT-PBLM because in the n-best we have several sentences which are different in terms of segments but exactly the same in terms of words. Table 8 shows an example of 4-best hypothesis which are different in terms of sequences but are identical in terms of words. Using the PRO algorithm for this type of n-best can distort the classification task in PRO. Indeed, it can classify the same sentence (in terms of words) in two different classes. In our experiments, we found that only 30% of the n-best are really distinct in terms of words.

To solve this problem, we apply PRO algorithm on the n-best distinct hypothesis in terms of sequences and words. Table 9 shows the obtained results where we provide to PRO n-best distinct sentences in terms of phrases and words. The use of the n-best distinct words and sequences shows an improvement of 0.51 points BLEU in test set while, for the development it is of 0.36 points BLEU.

Greedy search optimization

With these experiments, we claim that the existing algorithms of parameter optimization are not suitable for the model we propose. In order to find the best weight for the SMT-PBLM, we applied the greedy search algorithm. In this algorithm, we use the configuration of the MIRA algorithm as a starting point. the same weight obtained by MIRA for Phrase Penalty parameter therefore MIRA estimate properly this parameter. While we observe a drop for weights of the Phrase Penalty and translation model and an increase for weights of the Distortion and language model. The greedy search algorithm optimization enables us to improve the SMT-PBLM score of 0.13 points BLEU compared to MIRA and 0.24 points BLEU compared to MERT optimization.

Conclusion

In this paper, we presented a new method that uses a phrase-based language model for statistical machine translation task. This method has two major advantages. First, it can capture longer dependencies between words. Second, it uses the same phrases as those used in the translation table and more exactly those of the target part of each entry. In addition, our PBLM based method uses all target phrases of the translation table. The extensive experiments on Frenchto-English translation show that the phrase-based language model can improve the quality of the SMT task. Indeed, the SMT-PBLM translation quality is better than the baseline model in terms of TER which means that our model allows less work in post-edition operation. Thus, we showed that the state of the art optimization algorithms are not suitable for concatenated phrases used in our model. For this reason, we used a new optimization method based on a greedy search strategy to optimize the weights. This method allowed us to improve the results, we started with a BLEU of 32.80 and we improve it till 32.93 which is still slightly under the baseline which achieves a score of 33.16 which actually is just 0,23% better than ours. In future work, we plan to propose an extension of the optimization algorithms to better treat concatenated sequences. We also look at combining WBLM with PBLM in order to better estimate the grammatical quality of the target side and improve accordingly the quality of the translation system.

Table 4 .

 4 Weight of each parameter obtained by MERT System WP PP D LM TM SMT-WBLM -0.37 0.05 0.11 0.13 0.08 0.075 0.09 0.06 SMT-PBLM -0.21 0.31 0.10 0.10 0.10 0.008 0.07 0.07

Table 1 .

 1 Some examples of phrases in the intial and new translation tables where P (e|f ) is direct phrase translation probability, lex(e|f ) is direct Lexical weighting, P (f |e) is an inverse phrase translation probability and lex(f |e) is an inverse lexical weighting.

		source (f)	target (e)
				13 0.044	0.588 0.0206
	Initial	un petit peu plus ||| a little more	||| 0.0447 0.02261 0.176 0.070
	Translation voudrais dire	||| 'd like to say	||| 0.2	0.0089 0.181 0.0029
	Table	voudrais dire	||| want to say	||| 0.105 0.0060 0.363 0.017
		voudrais dire	||| would like to say ||| 0.5	0.0043 0.181 0.0023
		voudrais dire	||| would like to tell ||| 1	0.0045 0.090 0.0012
		un petit peu plus ||| a little bit more ||| 0.13 0.044	0.588 0.0206
	New	un petit peu plus ||| a little more	||| 0.0447 0.02261 0.176 0.070
	Translation voudrais dire	||| 'd like to say	||| 0.2	0.0089 0.181 0.0029
	Table	voudrais dire	||| want to say	||| 0.105 0.0060 0.363 0.017
		voudrais dire	||| would like to say ||| 0.5	0.0043 0.181 0.0023
		voudrais dire	||| would like to tell ||| 1	0.0045 0.090 0.0012

P (e|f ) lex(e|f ) P (f |e) lex(f |e) un petit peu plus ||| a little bit more ||| 0.

Table 2 .

 2 Details about corpora used in this work.

	Corpus	Sentences Tokens Unique Tokens
			(fr-en)	(fr-en)
	Train	176857 3.4M-3.1M	75K-58K
	Dev (2010)	887	20K-20K	3.8K-3K
	Test (2010) 1664	34K-32K	4.7K-3.8K

Table 3 .

 3 Results for the French → English translation task with MERT optimization algorithm.

		Dev	Test
	System	BLEU BLEU TER (TER-BLEU)/2
	SMT-WBLM LM(5-grams)	26.45	33.23 49.68	8.225
	SMT-WBLM LM(4-grams)	26.41	33.18 49.49	8.155
	SMT-WBLM LM(3-grams)	26.40	33.09 49.93	8.42
	SMT-PBLM LM phrase (5-grams)	25.60	32.69 49.13	8.22
	SMT-PBLM LM phrase (4-grams)	25.34	32.63 49.07	8.22
	SMT-PBLM LM phrase (3-grams)	25.32	32.58 49.31	8.365
	SMT-PBLM LM phrase (2-grams)	25.35	32.64 49.22	8.29

Table 5 .

 5 Few examples of translations based on the phrase-base language model SRC et c' est ce qui s' est passé à la fin de ces trois mois . REF and that 's what happened at the end of that three month period . SMT-WBLM and this is what happened to the end of these three months . SMT-PBLM and that 's what happened at the end of these three months .

	SRC	qu' est-ce qui définit une histoire ?
	REF	what defines a story ?
	SMT-WBLM what is it that defines a story ?
	SMT-PBLM what that defines a story ?
	SRC	le deuxième facteur concerne les services que nous utilisons .
	REF

Table 6 .

 6 French → English translation result with MIRA optimization algorithm.

		Dev	Test
	System	BLEU BLEU TER (TER-BLEU)/2
	SMT-WBLM 26.30 33.16 49.86	8.35
	SMT-PBLM 25.19 32.80 49.18	8.19

Table 7 .

 7 French → English translation result with PRO optimization algorithm.

		Dev	Test
	System	BLEU BLEU TER (TER-BLEU)/2
	SMT-WBLM 25.62 33.92 48.06	7.07
	SMT-PBLM 24.81 32.49 48.41	7.96

Table 8 .

 8 Example of 4-best hypothesis.

	today everybody speaks of happiness .
	today everybody speaks of happiness .
	today everybody speaks of happiness .
	today everybody speaks of happiness .

Table 9 .

 9 Obtained results when updating the n-best list.

	Dev	Test

  Then iteratively we vary the weights one by one by keeping the other weights fixed. At each iteration we calculate the BLEU score on the development set. The weights that give the best BLEU score are used for the decoding. After a set of tests, the highest obtained score BLEU on the development corpus is 25.86 and 32.93 points BLEU in the test corpus. Table10shows the best weight obtained by greedy search. Table10indicates that the greedy algorithm retains

Table 10 .

 10 Weights of each parameter obtained by our greedy search method. .22 0.12 0.11 0.10 0.05 0.13 0.001 PBLM (greedy search) -0.29 0.22 0.15 0.13 0.08 0.05 0.11 0.08

	System	WP PP D LM	TM
	PBLM (MIRA)	-0.22 0

MERT: Minimum Error Rate Training

PRO: Pairwise Ranking Optimization

MIRA: Margin-Infused Relaxed Algorithm