Bernard Xerri
email: xerri@isitv.univ-tln.fr

Jean-François Cavassilas

Bruno Borloz

Passive tracking in underwater acoustics 1 Passive Tracking in Underwater Acoustics

Keywords: source localization, passive listening, tracking, cross-ambiguity function, Doppler, whitening, propagation delays estimation, AR models, lattice filter

This paper provides a novel passive underwater acoustic method to track a moving object called source or target, with the following constraints: the sensors location is fixed and imposed, and classical array processing techniques cannot be applied. The method proposed has been successfully used to track a surface vessel (2D problem) or an underwater target (3D problem). All the results presented have been obtained with real signals. The localization of the target requires the estimation of propagation delays, that means the duration between the instant of the signal emission and its reception on each receiver. The cross-correlation function is a suitable tool when the target is motionless, but needs to be extended to the ambiguity function when it is moving. The signal to noise ratio, the uniformity of power spectral density and the integration time are determining factors for the accuracy of the localization. We show that a whitening method and a Doppler compensation are necessary, and we propose a way to eliminate the significant problem of the reflected signals. Furthermore, a new configuration of the receivers is proposed, based on the idea of coupling receivers whose distance is chosen from experiment derived results. Furthermore, the algorithm proposed is susceptible to parallel implementation, thereby facilitating real-time uses. Experimental results with real time domain data are presented and compared to trajectories obtained by an active method.

Résumé

Cet article présente une nouvelle méthode d'écoute passive destinée à trajectographier un objet en mouvement appelé la "source" ou la "cible", avec les contraintes suivantes : le réseau de capteurs est fixe et imposé, et les méthodes classiques de traitement d'antenne ne peuvent pas s'appliquer. La méthode proposée ici a été utilisée avec succès pour trajectographier un bâtiment de surface (problème à deux dimensions) ou une cible sous-marine (problème à trois dimensions). Tous les résultats présentés ont été obtenus sur des signaux réels. Le calcul de la position de la cible nécessite l'estimation des temps de propagation, c'est-à-dire du temps écoulé entre l'instant d'émission du signal et sa réception sur chaque capteur. La fonction d'intercorrélation est un outil opportun lorsque la cible est immobile mais elle doit être étendue à la notion d'ambiguïté quand la source est en mouvement. Le rapport signal à bruit, l'uniformité de la densité spectrale de puissance et la durée d'intégration sont des facteurs déterminants pour la précision de la localisation. Nous montrons qu'une méthode de blanchiment et une compensation de l'effet Doppler sont nécessaires et nous proposons un moyen d'éliminer le problème des trajets multiples. Par ailleurs, une nouvelle configuration des capteurs est proposée, fondée sur l'idée d'associer des hydrophones séparés d'une distance déterminée de manière expérimentale. Des résultats de trajectographies sont présentés et comparés aux trajectoires obtenues par une méthode active.

Notations

1 st -order Doppler coefficient related to Hi at time t DDi(t) 2 nd -order Doppler coefficient related to Hi at time t Dji(t)

1 st -order differential Doppler coefficient related to the jth and the ith receivers at time t. DDji(t) 2 nd -order differential Doppler coefficient related to the jth and the ith receivers at time t. fog(x) = f(g(x)) te emission time of S SNR signal to noise ratio

Introduction

Underwater acoustic tracking using passive sensors has been extensively studied, and source localization continues to be a main area of interest in ocean acoustics. Although, the approach presented in this paper is innovative.

This paper addresses the problem of tracking a surface or underwater moving target from its radiated noises received on fixed and geographically separated hydrophones [START_REF] Xerri | Etude et Synthèse d'un trajectographe passif en acoustique sous marine[END_REF] [START_REF] Cavassilas | Interférométrie acoustique sous-marine relative à des sources rayonnantes en déplacement rapide[END_REF]. The available sensors do not form an array as commonly accepted. Therefore, as we are in a near field configuration, the propagation duration which is the time elapsed between the time a signal is emitted and the time it is received by a sensor is finite and cannot be ignored. Our paper deals explicitly with the propagation delays which are time varying and with the resulting Doppler effect.

Initially, the receivers field was used to perform active tracking of surface vessels equipped with transmitters. Of course, such a method is no more feasible in the case of fast maneuvering underwater sources. Then, it was natural to implement a passive method as a complement to the active one.

Active methods can be separated into several techniques : the targets to track can be equipped with a transmitter [START_REF] Woodward | Radar theory[END_REF] [START_REF] Woodward | Probability and information theory, with applications to radar[END_REF] [START_REF] Knight | Digital signal processing for sonar[END_REF]. Then, the emitted signal is known, and array processing techniques, with linear or nonlinear arrays, are generally used. But such methods require the installation of powerful and expensive transmitters on objects to track, disturbing electronic systems on board and modifying hydrodynamic qualities of underwater targets. These disadvantages make unacceptable such a method for underwater targets. Without any transmitter, under the hypothesis of far field, linear array are used. The emitted signal is generally a known line spectrum signal (chirp, modulated or not) ; several ways are possible : short impulses which do not take into account the Doppler effect, long impulses for which the Doppler effect must be considered, or specific signals insensitive to Doppler effect [START_REF] Mammode | Estimation optimale de la date d'arrivée d'un écho sonar perturbée par l'effet Doppler[END_REF] [START_REF] Westwood | Broadband matched-field source localization[END_REF]. The disadvantages of active methods are evident, especially for reasons of discretion, but in the absence of such restrictions, they are very useful, as for instance for communications between maneuvering autonomous underwater vehicles [START_REF] Bayan | Computationnally efficient Doppler compensation system for underwater acoustic communications[END_REF].

However, our topic is not interested in active methods but in passive methods, and the results obtained with active methods are only used to verify those obtained with our method for surface targets.

Several passive methods have been developed. Array processing techniques are generally used to estimate the source azimuth; for narrow-band signals, the tracking of line spectrum or more geometrical methods are used [START_REF] Milios | Acoustic tracking from closest point of approach time, amplitude, and frequency at spacially distributed sensors[END_REF][24] [START_REF] Chen | VLF source localization with a freely drifting acoustic sensor array[END_REF]. The receivers form antennae which are linear or nonlinear, moving or fixed. For example, the use of sonobuyos equipped with GPS (global position system) receivers can be considered for the detection of multiple impacts on the ocean surface [START_REF] Stotts | Multiplesource localization using GPS technology and received arrival time structure analysis in an air-deployed system[END_REF]. In this case, however, the authors try to detect impulsive sources, not to track a moving vehicle ; in such a scheme, the Doppler effect is not taken into account.

Thus, short baseline systems and long baseline systems have been developed [START_REF] Mickael | Synthetic long baseline navigation of underwater vehicles[END_REF] [START_REF] Westwood | Broadband matched-field source localization[END_REF]. Our paper considers the case of fixed, imposed and nonlinear receivers, and mixes the two systems according to the phase (initialization or tracking).

Taking into account the characteristics of the available sensors, our problem is to develop a passive method to track a target which is emitting an unknown continuous broadband spectrum, with fixed and spatially distributed hydrophones. The hydrophone field is large enough to contain the whole path of the target.

The localization problem is solved by estimating the different delays of reception between the receivers ; commonly, generalized correlation methods are used [START_REF] Ch | The generalized correlation method for estimation of time delay[END_REF][4] [START_REF] Mammode | Estimation optimale de la date d'arrivée d'un écho sonar perturbée par l'effet Doppler[END_REF][10][17] [START_REF] Carter | [END_REF]. There are at least two ways of obtaining the delays of reception between two receivers : time domain working methods are chosen to estimate them, even though a frequency correction is performed to improve the result [START_REF] Borloz | Comparaison de méthodes de blanchiment et application à la trajectographie passive en acoustique sous marine[END_REF] [START_REF] Harris | On the use of windows for harmonic analysis with the discrete Fourier transform[END_REF]. Such methods can also be applied in other domains like communications between vehicles [START_REF] Sharif | Closed loop Doppler tracking and compensation for non-stationary underwater platforms[END_REF].

Because of the signal to noise ratio which is not always favourable, it is necessary, to evaluate the quantities of interest, to use an integration time which cannot be as small as possible [START_REF] Davenport | An introduction to the theory of random signals and noises[END_REF] [START_REF] Middleton | An introduction to statistical communication theory[END_REF]. Because the target is maneuvering and considering the unavoidably significant integration times, it is not possible to ignore the Doppler effect, and therefore a Doppler compensation is necessary. To enable robust communications between moving vehicles, such a compensation is usually achieved by multirate sampling and linear interpolation [START_REF] Bayan | Computationnally efficient Doppler compensation system for underwater acoustic communications[END_REF] [START_REF] Sharif | Closed loop Doppler tracking and compensation for non-stationary underwater platforms[END_REF]. For a fast maneuvering target, a linear interpolation is no more sufficient. A higher order compensation (parabolic or more) must be achieved.

In brief, a hydrophone field consisting of fixed sensors which are not aligned is used to track a fast moving object emitting an unknown broadband signal which might contain narrow-band components. Therefore, a long baseline system is imposed. The signal to noise ratio is unknown, but customarily lies between -20 and 40 dB.

The main problems are the necessity of a compensation of the Doppler effect and the presence of bottom bounces or surface-reflected paths of the emitted signal. A remedy to eliminate the reflected paths is proposed, based on lattice filters and a AR modelling [6][7][8] [START_REF] Porat | Digital Processing of random signals[END_REF]; this method has been successfully tested on real data. Note that a suitable processing of the reflected paths can contribute to additional information.

Considering the maneuverability of the target and the integration time, a first-order or a second-order Doppler compensation is necessary.

As the domain of delays and Dopplers explored is large, it is necessary to consider a short baseline system which needs to be related to the long baseline system to realize a complete system.

The results obtained for surface vehicles with real data have been compared with those obtained with a radar. It is pivotal to note that the whole experimental results presented have been obtained with real data.

Furthermore, the implementation on a parallel architecture machine of the algorithm proposed has been performed, allowing a real-time calculation of the path.

This paper is organized as follows. Section 2 presents the experimental conditions and constraints ; convincing arguments are given for the choice of our method. Section 3 formulates a mathematical model and a method to track targets, using the existing ranges of hydrophones ; general and useful tools are introduced and adapted to the chosen method which is a long baseline method. Section 4 presents the practical method and tracking results ; after a critical analysis, a second method (short baseline) is proposed, requiring additional hydrophones whose position is chosen from experimental results. The both methods (long and short baselines) are mixed in a more general method. Section 5 is dedicated to the design of algorithms adapted to the new configuration of the hydrophones. Definitive results are presented in section 6, with the realization of a real-time tracking machine.

Finally, this study has been performed for the CEM ('Centre d'Essais de la Méditerranée' or 'Mediterranean test range') which owns three underwater ranges of hydrophones named Tremail.

Problem statement

The problem is formulated in the three dimensional case, and we assume that the source path owns straight lines and curve parts.

The Tremail ranges

Each of the three ranges owns fixed hydrophones whose positions are perfectly known. The field of interest is the shallow range called T.F.F. ("Tremail Faible Fond") which contains 8 receivers located approximately 250 m under the sea surface and on average 400 m away from one another (cf. figure 11).

Each hydrophone is connected to a reception center which records and digitizes analog signals. The hydrophones of a range are not immersed exactly at the same depth ; that allows to address the 3D localization problem ; however, the precision in z coordinate (depth) will be lower than in other coordinates. The depth indicated above is a mean value ; a discrepancy of several tens of metres is possible.

The nature of noises and hydrophones characteristics

Radiated noises of ships and underwater targets can be divided [START_REF] Urick | Principles of underwater sound[END_REF] into mechanical noises (engine, propellers, vibrations, …) and hydrodynamic noises (flow on the hull, air bubbles, cavitation,…). The former are quasi-periodic signals whereas the spectral representation of the latter is continuous. The hydrophones of the Tremail cannot detect very low frequency signals (their low cutting-off frequency is 100 Hz) ; their high cutting-off frequency is 100 kHz.

The surrounding noise results from the addition of several noises the origins of which are various : noises due to sea state, biological noises, molecular turbulence… These noises are located in different spectral bands. Number of studies have classified them according to their importance and frequency.

For the following study, it is pivotal to note that, as many experiments proved, sea noises measured on two sensors of the Tremail are uncorrelated.

The interferometry method

Among the aforementioned methods which could be used for the localization of a maneuvering target, we choose the interferometry method.

In fact, an azimutal method would use the Tremail as an array and requires the observation of signals with very low frequencies, which is not compatible with the low cut-off frequency of the sensors. Concerning the method which consists in following the line spectrum shifted by the Doppler effect, it needs a precise knowledge of the emitted signal for every target ; in our case, this a priori information is not available.

Due to the characteristics of the hydrophones and because the emitted signal is unknown, the chosen method is the broadband interferometry which has the advantage of exploiting the broadband component of the emitted signal. What's more, the main interest of this approach is that, as said before, noises on two sensors of the Tremail are uncorrelated. Furthermore, the absolute power fluctuations are not taken into account by such a method. According to experimental data analysis, the sampling rate is 10 kHz for ships.

Statements related to tracking

The propagation model assumptions

The velocity of the sound in water c is supposed to be not affected by the medium considered homogeneous (c is constant). We note {M(te)} the trajectory of the target.

If S(te) is the signal emitted by the target, the signal received on Hi is

( ) ( ) ( ) ( ) ( )
the coefficients i are introduced to respect the conservation of energy. As mentioned above, the noises Bi are uncorrelated. The remainder of the paper will ignore the coefficients i because interferometry methods are not interested in absolute energy level. Calculated cross-correlations will be exact except for a multiplicative coefficient.

The localization problem

We do not know the different values tpi(te) ; so we have to estimate them. From (2), we define the differential time delay between the arrivals on two hydrophones Hi and Hj by

( ) ( ) ( ) ( ) ( ) j e i e ij
-  . ( 3
)
Such an equation represents an hyperboloid the focuses of which are Hi and Hj ; then, three equations are necessary to know M(te) from the different ij(te). But the localization of the target requires the knowledge of ij(te) for the same emission time te. Then one receiver must be taken as a reference (this will be H1); the differential time delays can be written

( ) ( ) ( ) 1j e pj e p1 e - t t t t t =  .
For a fixed error  on propagation delays, the accuracy of the localization is all the better as the distance between the receivers grows.

Time delay estimation classical methods for stationary signals

Two classes of estimators can be used, working in time or frequency domains, to estimate the differential delays 1i at the same time te. They both use the fundamental properties of the correlation function SS() of a stationary signal S(t) : SS() is maximal for  = 0.

The cross-correlation function of two signals received on the sensors Hi and Hj is

( ) ( ) ( ) ( )   ij 1 2 ij 1 2 i 1 j 2 ,, SS t t t t S t S t  = =E . ( 4
)
If the target and the receivers are fixed, our model leads to

( ) ( ) ( )   ij 1 2 1 pi 2 pj , t t S t t S t t   - - E ,
because noises on two sensors are uncorrelated.

If the emitted signal S(t) is stationary with a microscopic correlation, and power

 2 then ( ) ( ) ( ) ( )   ( ) 2 ij ij pi pj ij , τ τ τ δ τ τ t t S t t S t t  + =  - + - = - E which is maximum for  = ij.
The ij could also be found by evaluating the slope of the phase of the cross-power spectrum. More generally, it has been established that the estimation of ij is simply the abscissa value at which the crosscorrelation peaks [START_REF] Ch | The generalized correlation method for estimation of time delay[END_REF]. However, the maximum likelihood optimal estimator of time delay is a 'filtered' cross-correlation function called 'generalized cross-correlation' and written

( )     ij (g) ij S S ij τ = P(ν) (ν) = P(ν) (ν)    FF ,
that is the Fourier Transform of the weighted cross-spectral density ij(). For P()= 1, ( )

(g) ij τ  is the common correlation function. Several weighting functions have been proposed [3][4] : -PHAT (phase transform) : ij 1 () (ν) P =  , -SCOT (Smoothed COher- ence Transform) : ii jj 1 ( ) = ( ) ( ) P      , -HT (Hannan-Thomson) : ij ij ij () 1 ( )= ( ) 1 ( ) C P C     -  , where ij ij ii jj () () ( ) ( ) C  =    
is the coherence function.

The PHAT transformation enhances the spectral areas with a low signal to noise ratio, whilst the HT method enhances the spectral rays reduced by the coherence function.

Hence, the choice of the SCOT method which leads to the coherence function whose properties are adapted to the interferometry methods is natural [START_REF] Borloz | Comparaison de méthodes de blanchiment et application à la trajectographie passive en acoustique sous marine[END_REF]. Such a weighting whitens spectral areas where cross-information is high ; the crosscorrelation peak becomes more narrow, improving the estimation of time delay. What's more, the phase of ij() is not modified so that the abscissa value at which the crosscorrelation peaks is not changed.

extension to the case of non-stationary signals

Considering a zero mean non stationary signal S(t), we can also define a correlation range c such as

  c (,τ) ( ) ( -τ) 0 | τ | τ SS t S t S t  = =   E .
If the stationarity fluctuations are small with respect to the duration c, then

( ) ( ) , 0 , τ , | τ | 0 SS SS t t t       .
The following function

2 2 1 ξ(τ) ( , τ)d T T SS T tt - = 
is also maximal for  = 0. It is essential to note that this function can be defined even for non stationary signals, and that even though () is not representative of the signal characteristics, its maximum is all the same reached for  = 0. The choice of the integration time length T is delicate and must be made according to experimental analysis : it depends closely on the SNR : a low SNR requires to increase T, but in return, increasing T too much makes the Doppler effect prominent and involves expensive computations. Hence, for fast maneuvering targets, T cannot be chosen too large.

In the same way, we can define, using (4)

2 2 ij ij 1 ξ (τ) ( , τ)d T T T tt - =  . ( 5
)
It can be proved [START_REF] Xerri | Etude et Synthèse d'un trajectographe passif en acoustique sous marine[END_REF] that, as above, ( )

ij ij argmax ( ) =    . (6)
In the particular case where S(t) is a stationary signal, SS(t,) does not depend on t and

() = SS() or ij() = SS(-ij
) and the relation ( 6) is still true.

In this study emitted signals are non stationary and consequently ij() is used instead of ij(t,). What's more, such a quantity is suitable for the ergodicity assumption. We want to estimate ij and not statistical characteristics of the emitted signal, thus the calculation of ij() is always suited to our problem : it will allow to calculate delays even if the integrated functions cannot be interpreted as correlation functions.

variance of the estimation of the differential delays

The coherence function verifies the relation |Cij()|  1 ; in the absence of noise, the equality is reached if the underwater medium behaves as a linear filter. Several factors contribute to coherence destruction -a low SNR in a cross-spectral band, -the medium does not behave as a linear filter, -the existence of reflected signals with no perceptible reduced power.

It has been established [START_REF] Hannan | The estimation of coherence and group delay[END_REF] that the variance of the error of estimation of ij is

( ) ( ) 2 2 ij 2 0 ij ij 22 2 ij 0 1( ν) ν ν 1 τ -τ = lim 8π |( ν)| ν ν B B B Cd T Cd → -        E
where B is the upper frequency bound of S(t), under the following hypotheses -the shift of the cross-correlation peak is due to the additive noises, -T is large enough to ensure that, around the peak, the estimated cross-correlation is identical to its second sum series expansion, -T is much larger than the signal correlation support, -Si and Sj are jointly gaussian.

So if ij (ν) C =d constant in the band [0,B],
( )

2 2 ij ij 2 3 2 31 τ -τ = 8π d TB d -   E .
In the absence of noise, Si and Sj are coherent, and

( ) 2 ij ij τ -τ = 0   E .

The Doppler effect

The aforementioned reasons explain why the integration time T cannot be chosen as small as wanted. This constraint will induce a Doppler effect. What's more, as the target is moving, tpi(te) changes with time. Such changes will lead to modifications of the received signals which can be significant enough to make the cross-correlation peak undetectable. The higher the speed and acceleration of the target are, the greater the deformation of the signal becomes. Because of the low value of c, the envelope of the received signal is distorted. It is necessary to consider a first-order or second-order expansion of tpi(te) to find and balance the distortion affecting the signal.

Even though we suppose that S(t) is stationary, there is usually no chance that the received signals Si(t) are stationary too. But, as mentioned before, we try to measure ij which remains approximately constant during the integration time used to evaluate it. The fluctuations must be small beside the correlation time : they are linked to the value ij τ ( )

t t   .
Let's note i e ( ) t the unitary vector directed from Hi to M(t), and HiM(t) = c tpi(t) the corresponding distance.

Because hydrophones are fixed,

( ) ( ) i t HM vt t  =  is the speed of the target. Then ( ) ( ) ( ) ( ) ( ) ij j τ e -e i t vt tt tc  =  .
Let define the first and second-order Doppler coefficients related to Hi at time t, as follows

▪ pi i i () ( ).e ( ) () tt v t t Dt tc  =  ▪ ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
i i i pi i i i i pi 0 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

H M t S t S t S t t t c S f t S t t

= + = + = = +    (7) that means t + tpi(t) = ti + tpi(0). ( 8
)
t t i t = 0 t pi (0) t t+t pi (t) t i = 0 t i t k t k = 0 t k t pk (0) t+t pk (t)  ik (0)  ik (t) (H k ) (H i )

Fig. 1 -time scales for emission and receptions

Thus, we consider three different time scales -the absolute scale t : the one of the source, -the relative scales ti related to the receivers Hi.

Clearly, the relation between the emitted signal and the received signal must be established at time tpi(t).

We will note In fact, [START_REF] Friedlander | Lattice filters for adaptative processing[END_REF] and ( 8) implies ( ) ( ) ( )

1 i i i i S t S f t - = .
For ti close to tri , that means ti = tri + t where t  0, let's note

( ) ( ) ( ) ( ) ( ) (
) For ti = tri + t with t  0, ( ) ( )

-1 i i ri i ri i ' θ S t
-1 i i e ie 1 t f t t Dt =+ + .
The signal received on Hi corresponds to the emitted signal S(te), with a delay tri(te) and expanded or compressed according to the value of Di(te).

With our hypotheses, the function ji can be calculated as

( ) ( ) ( ) je ji ie 1 θ 1 Dt tt Dt + = + . ( 9
)
Usually, the speed of the source can be considered very small beside c, so Di(te) and Dj(te) are very small beside 1; we can approximate ji(t) by the following expression ji(t)  (1+Dj(te)) ( In both cases, the signals S ' i and S ' j correspond one to the other except for the transformation ij(t).

The function ij(t) must be defined on a duration T' corresponding to the integration time necessary for the evaluation of the function fij defined above. For a fixed value of T', we will be able to validate, according to the speed of the target, the first-order expansion of i(t) and j(t).

Extension of the ambiguity function

If Sj is the signal of reference, we can modify Si so that it becomes comparable to Sj from the cross-correlative point of view. For non-stationary signals, ij() defined by [START_REF] Borloz | Comparaison de méthodes de blanchiment et application à la trajectographie passive en acoustique sous marine[END_REF]



. This is the cross-ambiguity function [START_REF] Hannan | The estimation of coherence and group delay[END_REF] which depends on three parameters. As seen above, this expression can be simplified if the acceleration of the target is negligible (then DD=0). In fact, we could create a function with more parameters, considering a higher order series expansion of tpi(te). Experiments show that the maximum of ij ˆ(τ, , ) D DD  is reached for ij, Dij and DDij which are respectively the differential time delay, the first and second-order differential Doppler coefficients.

T = b-a is the integration time. The estimation of ij is optimal for the good parameters Dij and DDij, otherwise it is a sub-optimal estimation. It has been proved [START_REF] Xerri | Etude et Synthèse d'un trajectographe passif en acoustique sous marine[END_REF] that the error of the estimation of ij is

    ij ij ij ij ij 1 τ -τ - 12 ba DD D + = + EE .
To have an unbiased estimation, it is necessary to choose

( ) ( ) ( ) 2 2 2 ij ij ij ij 2 ij 1 τ -τ - 12 1+ DD D T =     EE .

results

Now, we present some results of calculated envelopes of cross-correlation functions obtained with and without a correction of the signal distortions. The figures above show the envelopes of the cross-correlation between receivers H11 and H14 ; the first one with no correction (D=0 and DD=0) and the second one with the optimal Doppler compensation (D=0.6% ; DD=0 because, in this case, the acceleration is negligible).

Fig. 3 -optimal cut of the ambiguity function

Without correction, no peak stands out. However, a peak emerges after a first-order Doppler compensation. The crossambiguity function between the signals received on hydrophones H11 and H12 is shown below. Fifty first-order Doppler coefficients were used with a 0.02% step, from 0.072% to 0.172%. In this case, the maximum is reached for ij = 33 ms and Dij = 0.122 %.

Fig. 4 -ambiguity function between the signals received on H11 and H12

The problem of multiple reflections

model and choice of a method

The presence of signal reflections on the sea bed can be modeled as follows ; the signal received on a sensor Hi is

i pi ik pi ik i 1 ( ) ( ) ( τ ) ( ) R N k S t S t t r S t t B t = = -+ -- +  ( 11
)
where NR is the number of reflected signals, ik is the delay of the k-th reflection and rik is the magnitude of each reflected signal. The reflections modulate the spectral power density, destroy the coherence in certain frequency areas and create secondary peaks in the cross-correlation function, introducing errors on differential propagation delay estimation. Generalized cross-correlation methods, as SCOT, cannot eliminate secondary peaks due to reflections [START_REF] Hanna | Analyse spectrale par filtrage autorégressif[END_REF].

In theory, the finite MA model [START_REF] Woodward | Radar theory[END_REF] can be approximated by an infinite AR model. Practically, a finite AR model must be used, and the order p must be chosen large enough to take into account the secondary peaks and make them disappear. This AR model is also used to whiten the received signals. It is indeed possible to interpret the emitted signal as the response of a linear filter to a white noise. By reversing the AR filter, we estimate the input white noise or innovation n and then we realize the whitening of the signals [7][8]. This whitening is performed on raw data, and the correlation function is then calculated on transformed signals.

practical development

The choice of the AR order p is delicate, and several criteria have been proposed as SVD analysis of the covariance matrix [START_REF] Makhoul | Linear Prediction : a tutorial review[END_REF]. To achieve the estimation of the model parameters, the Yule-Walker equations could be used. The estimation of this parameters can also be performed from raw data, commonly based on a least mean square-error of prediction of the signal criterion. Two classes of methods are possible, using the forward prediction or the forward and the backward predictions.

Lattice filters are commonly used to perform this calculation [START_REF] Porat | Digital Processing of random signals[END_REF]. We can define a basic section linked to the evolu-tion of the pth-order (forward and backward) prediction errors ep(t) and rp(t) from the (p-1)th-order errors. It leads to a system of recursive equations on both time and order where  is a forgetting factor verifying "  1". We find x(t) (1)

 k k-1 k k-1 * k k-1 k k-1 1 1 ( ) ( ) ( ) ( ) ( ) ( )

e t e t K r t r t r t K e t

      k-1 k-1 * kk 22 k-1 k-1 ( ) ( 1) ( ) ( 1
e 1 (t) r 1 (t) z -1
(2) If x(t) is the signal received on one sensor, en(t) is the forward residual error which is white if the process is really a AR process. Hence applying this filter to the signals Si(t) amounts to whitening them. The final basic schema is the following

e p-1 (t)
S i AR S j AR e i e j

 ij ambiguity

Fig. 7 -basic scheme of the whitening procedure

It is pivotal to apply the same transformation to the whole signals so that, even though the spectra are modified, this method does not modify the phase of the cross-spectrum, and the abscissa of the peak of the cross-correlation function is not shifted.

Furthermore, such a method can be slightly modified to process non-stationary data, and can be implemented to process data in real time; finally, normalized methods can improve precision and convergence speed of calculation.

A convincing result is presented below ; the figures 8 and 9 show the cross-correlation obtained with the SCOT whitening method : secondary peaks are clearly present on both sides of the main peak. The figure 10 shows the crosscorrelation obtained with a AR model with an order p=100 : secondary peaks close to the main one have been seriously softened.

In fact, this method does not eliminate the secondary peaks, but push them away the main peak while reducing their magnitude. Thus, the order of the AR model must be chosen all the larger than the peaks are high or far away the main peak delay (ms)

Fig. 8 -envelope of the cross-correlation function with secondary peaks due to reflections on the sea bed (H11 -H14) Fig. 9 -envelope of the cross-correlation function with secondary peaks due to reflections on the sea bed (H11 -H12)

Tracking Introduction

We want to calculate one point of the trajectory every t second. From experiments, we will take t = 1 s for a surface vessel and less for a fast underwater target. The choice of t is linked to the possible dynamic evolutions of the target. It must be small enough to assure that the coordinates of the ambiguity function peak at time t+t are close to the values found at time t ; that means that the prospecting area of the parameters (, D and DD) is restricted to a priori defined values.

Practical method description : long baseline system

We suppose that at each time t we are able to estimate ij, Dij and DDij as described before, for every available couple of receivers. As seen before, to ensure that these parameters correspond to the same emission time, we must choose a sensor of reference H1 ; the others will be used to create pairs. A Doppler compensation of S1 is required to make it comparable to other received signals : that means a compression or dilatation of this signal, obtained by an interpolation t' = (1+D) t + DD t 2 , because we took a second-order Doppler compensation as a limit. All the transformations are performed at the same time on the signal of reference S1. The cross-ambiguity function is computed at the same time between some receiving pairs ;

  

by taking into account (3), ( 9) and [START_REF] Ch | The generalized correlation method for estimation of time delay[END_REF]. Then, the approximation to the state of the target for te+t is

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )
+  = +  +   +  = +    +  =        
As said above, it delimits the prospecting area of the parameters , D and DD at time t+t. This process can be repeated at time t+2t, and so on.

A trajectory obtained with real signals emitted by a surface vessel on the T.F.F. is shown on the following figure, where DD was assumed to be null, so  (te) could not be reached

Fig. 11 -result of the target tracking

Theoretically at least 3 pairs are necessary to estimate the state of the target. With four pairs, it is possible to estimate the value of c instead of fixing its value a priori : of course the estimation is a constant value. Practically, proceeding that way, the estimation of the state of the target becomes better.

An extended Kalman filter can been used to estimate a priori position and speed at time t+t (and then differential time delay and Doppler coefficients), used to initialize algorithms even if previous calculations at times t, t-t, t-2t,… did not permit to estimate these parameters.

The estimation of the ambiguity function has been restricted to 3 parameters ; that means the Doppler compensation has been restricted to the second-order. We could have used a third-order correction to improve results, especially for fast target. The cost for a better precision would be a higher sum of computation.

Estimation errors

The error on the position is difficult to reckon, because we do not know exactly where the supposed punctual source is located. It is probably situated at the back of the target (owing to the propellers, cavitation,…) as it will appear in the next study. It is difficult to compare the passive trajectory with a radar trajectory obtained with a reflector put on the middle of the target, because the tracking point is not the same in the both cases. The error between the trajectories is almost constant in the straight parts in comparison with the half size of the target. The tracking 'point' is located in the back of the target. In some cases, for active trajectory, we have noted a bad zero point because the radar was not satisfactorily calibrated.

Therefore we evaluate the errors on the position M(te) and on the speed

( ) e vt 
from the width of the peak of the cross-ambiguity function: ij and Dij. The available results are the position, the speed and the associated errors. Following, we can have a cartography of the maximum position error in the new Tremail range. Experiments have shown a maximum value of 0.5 ms for ij.

With this value, we can calculate, a cartography of the maximum error occurring in the future Tremail range.

Practical difficulties

Practically, two sorts of difficulties are encountered :

physical difficulties

To ensure a precise decision, the signals from two receivers must keep a minimal mutual coherence to make emerge a peak from the ambiguity function. As experiments have shown, this condition is not necessarily verified when the angle ( ) ij , H M MH = tends to . Then, the Doppler influence is greater and a second-order Doppler compensation may be insufficient. Then, the multiple reflections of different natures increase the dissymmetry between received signals. Finally, the dissymmetry of the radiation diagrams of the source is maximum in this configuration.

Obviously, the signal to noise ratio is a determining factor for the quality of the tracking.

calculation difficulties

They mainly appear during the initialization phase, and are due to the large possible range of delays and Doppler coefficients, which grow with the angle  and the distance between the receivers. The computation of the ambiguity function is so heavy that we cannot have access to a priori knowledge about the speed and the position of the target.

Conclusion

With the Tremail range in passive listening, we have shown that it is possible to obtain the trajectory of surface or underwater targets. In the multiple cases which are considered, we have given the trajectory of the targets.

Because the possible ranges of parameters are very wide, the initialization procedure requires a lot of computations. The position of the hydrophones can be improved to allow passive tracking in a shorter time. It is necessary to create pairs of near receivers to improve results and to facilitate the initialization procedure. The distance chosen between the receivers of a pair will be about 100 m because noises on two receivers must remain uncorrelated and the respect of this condition imposes that the distance cannot be indefinitely decreased. In addition, an existing pair (H11-H12 distant of 100m) gave satisfying ambiguity functions.

This necessary short baseline system is described in the following section, and then mixed with the long baseline one to create a complete and autonomous system for tracking.

Of course, at least 3 pairs are necessary to perform a 3D tracking. But, as a precautionary measure, we will create more pairs.

Tracking mixing short and long baseline systems

Introduction

As mentioned above, the previous method (long baseline system) has to be modified ; a reference pair of receivers, the first one (H11,H12), is chosen ; a pair of receivers is noted (Hi1,Hi2) with i=1 to Np the number of pairs. This section aims at developing the short baseline system and mixing it with the long baseline system.

The different ambiguity functions lead to differential time delays i linked to the pair (Hi1,Hi2). As Np pairs are available, we have Np equations (i=1 to Np)

Hi1M -Hi2M = c i . ( 13
)

Principle of the evaluation of the target position

The position of the target M(te), is obtained by convergence of a gradient method. We define the error associated to a pair i = Hi1M -Hi2M -c i . We want to minimize the following criterion

J(M) = 1 2 i ε N i p =  .
A recursive algorithm converges towards the real position of the target (corresponding to the delays i) is performed. Experiments and simulations have shown that the convergence is ensured in the Tremail range.

Evaluation of the error on the position of the target

The new Tremail range is based on coupled sensors ; it tends to decrease the z component (depth) of gradients wi where wi=gi1-gi2 and gij is the gradient of ' ij MM HM = . The location precision on depth will be insufficient. So, it is necessary to study horizontal precision (x and y) for a fixed z.

Equations [START_REF] Cavassilas | Interférométrie acoustique sous-marine relative à des sources rayonnantes en déplacement rapide[END_REF] represent hyperboloids. For a fixed z, the set of points that verify them are curves whose parameters are the i. For a delay k'  [k-k, k+k], we have a strip on the z-plane. The intersection between two strips provides the area D where each point M' verifies

' i1 i2 i ' j1 j2 j ' ' .τ ' ' .τ H M H M c H M H M c -= -=    with k'  [k-k, k+k] for k = {i,j}.
For simulations, imaginary hydrophones are created ; for example, if we consider two pairs H23-H231 and H24-H241 by adding the imaginary hydrophones H231 and H241, the domain D looks like on the figure below.

Four pairs are available for the short baseline system, designing an other domain D. For a finite number K of points {Mi} taken on the outline of D, we define a criterion to quantify the maximal mean error on the position of the target for a fixed value of  The evaluation of the coordinates of Mi is performed with the following system, where the position error is Mi = Mi -M and wi is the gradient vector at point M. The error ranges between two meters in the center of Tremail and more than twenty meters on the sides.

For the needs of our problem, using the existing pair H11-H12, three new pairs are created : H22-H27, H15-H16 and H13-H21. Four pairs are then available for the short baseline system, designing an other domain D. We can see below the new T.F.F., proposed after the present study ; new receivers have been added to create near couples (H11-H12, H22-H27, H15-H16, H13-H21).

The new Tremail owns near receivers; with these pairs, it is not possible to form coupling system with one sensor of reference as previously discussed. With this short baseline, it is necessary to adapt a new method associated to the four pairs.

Fig. 14 -new Tremail configuration

Initialization of a tracking

The Tremail range has been changed ; so, we have to define a new simple initialization mechanism with the short baseline system. This procedure can also be used when the target is lost. In this configuration, it is impossible to take a unique reference sensor to determine all the ambiguity functions. In this problem, the position, the speed and the acceleration of the target, and also the emission time te and then reception times tri(te) are unknown. The wave emitted at time te arrives on the receiver of reference of each pair at ( )

i1 e ri e
() e H M t t t t c =+ .

Fig. 15 -position of the target and pairs of sensors

Because the position is indeterminate, the reception times for each pair is unknown. Assuming we calculate the ambiguity functions for the same reception time, the maximum error in the reception times is linked to the greatest distance between the pairs of receivers : for T.F.F., it is about one second. where Hi1 is the first receiver of the i-th pair. The calculation of ambiguity functions for the same reception time for all the pairs does not tally with a real posi-tion of the target. But we can find a position that minimizes the mean square error, which cannot be a real position. Having found this estimate position, we can modify the reception time for each pair except for one that we do not change and that is considered like the reference pair.

Consider a pair of reference, called number one. Calculation of propagation delays for reception times are initialized as follows for each pair ( )

(0) i ri τ t with (0) (0) ri r1 tt = p i 1,...,N = .
The minimization of the mean square error allows us to estimate the position of the target M0 ; this permits to readjust the reception time (1) ri t . The same reception time is kept for the reference pair. We have

11 0 i1 0 (1) (0) ri r1 H M H M tt c - =- .
A new evaluation of delays i at times (1) ri t , by ambiguity functions, allows us to calculate M1.

A convergence by iterations towards the position of the source is performed. All experiments and simulations related to studied targets showed the convergence is reached. The criterion used to stop iterations is

NN (n) (n+1) (n) (n+1) ri ri ri ri i=1 i=2 t t t t - = -    .
In a second stage, we can approximate the differential delay i( denotes the differential Doppler. This approximation is always justified with the chosen configurations.

We have to solve the following system of equations

Hi1M -Hi2M = c ( ) i ri τ t , p i 1,...,N = . ( 14
)
Let suppose that each ( ) (0) i ri τ t is calculated by the ambiguity functions. A series expansion of ( 14) around (0) ri t leads to ( ) ( )

i (0) (0) (0) (0) i1 i2 i r1 ri 1 r1 τ - τ ( - ) r H M H M c t c t t t t  =+  By definition ( ) (0) (0) r1 ri 11 i1 c t t H M H M - -=
, so that for all i, we obtain the following system

( ) ( ) ( ) ( ) ii (0) (0) (0) r1 i1 i2 11 r1 i r1 ττ 1- τ t H M H M H M t c t tt  - + = 
which is solved by iterations by a mean square error minimization.

Conclusion

A process has been proposed for position estimation in the case of tracking initialization ; the speed of the target can also be estimated in the same way by a second-order series expansion of i( (j+1) ri t

) in the neighborhood of (j) ri t . From differential time delays and Doppler coefficients computed for all pairs at reception time (0) r1 t , it is possible to evaluate the parameters of the target and the associated time.

This procedure is attractive because the distance between the two receivers of a pair is small and consequently the delays and Doppler ranges are reduced. This advantage is also a handicap for the precision on the estimated parameters.

The result obtained by this short baseline system procedure is used to initialize the long baseline system procedure described in the previous section. With this second method, the results are obtained with an increased accuracy. As long as the target is efficiently tracked, this procedure is used. In the case where the target is lost while tracked with the long baseline procedure, the short baseline procedure is launched.

The tracking machine

Presentation

A passive tracking machine, based on the principle presented before, containing a two channels numerical acquisition board and several specific fast calculation boards, has been built in order to perform real time evaluation of differential time delay and first-order Doppler coefficient for one pair of receivers. An a posteriori trajectory calculation can also be performed because received signals are recorded on a magnetic tape.

Experimental results

The trajectory shown below (fig. 16) was obtained (one point every second), using four pairs of receivers, with signals emitted by a fast patrol boat doing 15-20 knots during 3 minutes. About 15 minutes were needed to reconstruct. The part of the trajectory used contains a straight line and a curved part in order to better appreciate the quality of processes. The sampling frequency is 10 kHz (the useful frequency band is [100 Hz ;3 kHz]); integration time was taken equal to 1 s; 21 different Dopplers (first-order) were computed for each point of trajectory. The trajectory obtained simultaneously by a radar is superposed.

Absurd points are present on the passive trajectory (figure 17) ; in fact, this test underscored a problem due to the acquisition board of the tracking machine (a disturbing correlation between the two channels present for  = 0).

The error made on the estimated position (by the passive method) is difficult to evaluate, because we do not know exactly where the supposed punctual source is located. However, it seems more likely that it comes from the back of the target (owing to the propellers, cavitation, …).

Furthermore, it is difficult to compare the passive trajectory with the radar trajectory obtained with a reflector put on the middle of the target, because the tracking point is not the same in the two cases. The error between both trajectories is almost constant in the straight line parts compared to the half size of the target. In some case, for active trajectory, we have noted a bad zero point because the radar is not satisfactorily calibrated. Disregarding the absurd points (most of which could easily be eliminated), a difference of about 15 metres can be seen, probably due to a too strong smoothing of the radar trajectory. After a suited correction of the acquisition board, the result obtained is the following (this is not the same part of the trajectory than above) .

The figure 18 shows the trajectory (projected on the horizontal plane x-y) obtained a posteriori for an underwater target during 15 seconds. A point is computed on every 0.5 s. The sampling frequency is 48 kHz (the useful frequency band is [100 Hz;20 kHz]); integration time is taken equal to 0.5 second; 49 different first-order Dopplers were used for each point of the trajectory during the initialization phase, 15 only during the tracking phase. No second-order Doppler compensation was made here. No radar could be used during this test. Much less absurd points are present (figure 19).

On the figure 20, each estimated point is drawn with a vector whose direction and length represent those of the estimated speed vector. The target goes from right to left. This figure is zoomed ; this is why the right part of trajectory is lacking.

In all cases, the trajectories obtained were satisfactory. Nevertheless, for such targets, a second-order Doppler com-pensation becomes necessary to avoid the loss of the target, especially during turns.

Conclusion

This study shows the feasibility of underwater passive tracking with the Tremail system. This feasibility is built from real signals that surface or underwater targets emit.

The evaluation of the trajectory requires the calculation of the general ambiguity function for all pairs of receivers and eventually a data pre-processing to suppress echoes. This one is dependent on the bottom of the area. The tracking is made up of two parts : one part is the tracking step. It consists in the estimation of some parameters at time t+t with some knowledge of them at time t. It is performed with pairs of receivers a long way apart. One receiver named reference is present in every pair. This is the long baseline system.

the second part is the initialization step. No knowledge is necessary on the parameters we have to evaluate. It is performed with pairs of near receivers. This system minimizes calculation but is less accurate in parameter estimation. This is the short baseline system. Our study presents a global system using a short baseline system and a long baseline system of receivers.

The last concrete aspect of this study is the realization of a parallel machine. It assumes a real time calculation of the parameters deduced from one pair of receivers.

Nevertheless, a trajectory can be evaluated a posteriori with recorded signals.

2 Tb

 2 = . The variance of this estimator is[START_REF] Xerri | Etude et Synthèse d'un trajectographe passif en acoustique sous marine[END_REF]

Fig. 2 .

 2 Fig. 2. envelope of the correlation with no compensation

  : k for order (k=0,…p) and t for time. Kk and * k K are the PARCOR (partial correlation) coefficients and are calculated to minimize a weighted least-squares criterion

  leads to the following basic lattice section e k-1 (t)

Fig. 5 -

 5 Fig. 5 -basic section (number k) of the lattice filter

Fig. 6 -

 6 Fig. 6pth-order AR model : lattice filter

Fig. 10 -

 10 Fig. 10 -envelope of the cross-correlation function with reduced reflections effects

Fig. 12 -

 12 Fig. 12 -precision on the position estimation of source for 2 pairs of sensors (H23-H231 and H24-H241)

  represents the graph of the error r obtained for a depth z = -100 m.

Fig. 13 -

 13 Fig. 13 -position error for z = -100 m

Fig. 16 -Fig. 17 -

 1617 Fig. 16 -passively estimated and radar trajectoriesThe error on the coordinate x is shown below :

Fig. 18 -

 18 Fig. 18 -passively estimated and radar trajectories

order Taylor series expansion of tpi(te)

	S t	t + =	S f t	+	t	=	S	t	,
	then, for i  j, ' -1 i i oθ SS =	' j	oθ	-1 j	.
	Thus S ' i can be deduced from S ' j by
	( ) ( S t ' i == ) ( ) ( ) ( ) ' -1 ' j j i j ji oθ oθ oθ S t S t	.
	first-If we note
	( ) ri e t t t e = +	( ) ie H M t c	( ) ( ) e pi e i e t t t f t = + =	,
	a first-order series expansion of tpi(te) leads to
	( i e f t	+	d t	e	)	=	t	e	+	d t	e	+	( ) ie H M t c	+	( ) i e d D t t	e
					=	( ) ri e t t	+	( ) ri e . t t d
	Then
			( ) ri e t t d	( 1 =+		( ) ) i e d D t t e	.

second-order Taylor series expansion of tpi(te)

  If the acceleration of the target is no more negligible, a second-order series expansion of tpi(te) must be used, leading to DDji(te) t2 , where DDji(te) is the second-order differential Doppler coefficient related to Hj and Hi. We easily see that DDji(te) can be approximated by

					( =+ ( '' i j ( ) 1 S t S	( ) e D t ji	)	t	)
	( i e f t	+	d t	e	)	e = + t		d t	e	+	( ) ie H M t c	+	...
								...	+		( ) i e d D t t	e	+	1 2	( ) i e d DD t t	2 e	ri = + t	d t	ri
	or	d t	ri		=	d t	e	+	( ) e D t i	d t	e	+	1 2	( ) 2 e e d DD t i t	.
	In that case,
			i θ() tt  + e	1	+	( ) ie Dt t	-	( ) ( ) ) 2 ie ie DD t t ( 21 Dt +	3	.
	The calculation of ji(t) leads to
	( ) ( ) je ie Dt tt 1 θ ( )= ji 1 Dt + +		( 2 1 + ) 2 1 ( ) ie Dt +	...
				( ) j e DD t  ( ) ( ) ( ) 1 ... 1 je i e ie Dt DD t Dt + -+  	t	2
			( ) ji e DD t		( ) j e DD t -( 1 2	( ) ) i e DD t	.
	In that case S ' i can be deduced from S ' j by a parabolic relation
	( ) S t ' i	=	S	' j	(	(	1	+		( ) e D t ji	)	( ) e t DD t t ji +	2	)	.
	j by the
	linear relation

1-Dj(te)) t or ji(t)  (1+Dj(te)-Dj(te)) t = (1+Dji(te)) t where Dji(te) is the first-order differential Doppler coefficient related to Hj and Hi. Then S ' i can be deduced from S ' (10) For the same reasons described above, Di(te), Dj(te), DDi(te), and DDj(te) are very small beside 1, and we can approximate ji(t) by the following expression ji(t)  (1+Dji(te)) t +

  is used instead of the cross-correlation function. The transformation ij, previously presented, is used with coefficients noted D and DD ji(t)  (1+D) t + DD t 2 .

	1 ba a b S -	' i	( (1	+	) D t	j	+	) ( ) ' j j τd DD t S t 2 j . t -	j
									The estimated cross-correlation function ij ˆ(τ, , ) D DD 	can
									be written


Manel Rhif

Hazem Wannous

Imed Riadh Farah

Action Recognition from 3D Skeleton Sequences using Deep Networks on Lie Group Features

This paper addresses the problem of human action recognition from sequences of 3D skeleton data. For this purpose, we combine a deep learning network with geometric features extracted from data lie on a non-Euclidean space, which have been recently shown to be very effective to capture the geometric structure of the human pose. In particular, our approach claims to incorporate the intrinsic nature of the data characterized by Lie Group into deep neural networks and to learn more adequate geometric features for 3D action recognition problem. First, geometric features are extracted from 3D joints of skeleton sequences using the Lie group representation. Then, the network model is built from stacked units of 1-dimensional CNN across the temporal domain. Finally, CNN-features are then used to train an LSTM layer to model dependencies in the temporal domain, and to perform the action recognition.

The experimental evaluation is performed on three public datasets containing various challenges: UT-Kinect, Florence 3D-Action and MSR-Action 3D. Results reveal that our approach achieves most of the state-of-the-art performance.

I. INTRODUCTION

Human action recognition is a hot topic in the computer vision and pattern recognition fields. It is used for several applications related to machine learning techniques and human behavior as robotics, intelligent monitoring, human-computer interaction, virtual reality and video games, among others. The similarity of action due to the complexity of human movement and the variety of the same actions performed by distinct subjects are the fundamental challenges of actions classification.

In recent years, many approaches dealing with human action and activity recognition from depth sensors have received growing attention. These approaches can be categorized into 3D skeleton approaches, depth-based approaches and hydride approaches. In this paper, we focus on 3D skeleton data because it gives richer information about human kinematics. 3D skeleton data record the trajectories of human body joints and undergo less intra-class invariance compared to RGB and depth [START_REF] Zhang | On geometric features for skeletonbased action recognition using multilayer lstm networks[END_REF]. It is robust to illumination change and invariant to camera views. However, these skeletons are often noisy due to the difficulty in localizing body-parts, self-occlusions and sensor range errors [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF]. Variety of techniques reformulating computer vision problems over non-Euclidean spaces, such as Riemannian manifolds, have received growing attention [START_REF] Slama | Accurate 3d action recognition using learning on the grassmann manifold[END_REF], [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF], [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Devanne | 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold[END_REF], [START_REF] Devanne | Combined shape analysis of human poses and motion units for action segmentation and recognition[END_REF], [START_REF] Anirudh | Elastic functional coding of riemannian trajectories[END_REF]. In this paper, we consider such a manifold representation which considers the geometry of space, and more particularly focus on Lie Group features [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF], [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF]. Lie group is one of the manifold-based approaches that achieved the best result on several public datasets [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF], [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Anirudh | Elastic functional coding of riemannian trajectories[END_REF]. The main constraint of its representation is the misalignment of sequences in time and that makes distance metric inexact. Dynamic Time Warping (DTW) is the most common way to solve this problem. This step cost supplemental time. In addition, features are extracted per skeleton and then stacked, which makes this representation extremely high-dimensional. Despite the effectiveness of Lie group presentation, it is computationally expensive and extracts only spatial information from skeleton joints. Hence, temporal information needs to be exploited to express the dynamics of human motion [START_REF] Gaidon | Temporal localization of actions with actoms[END_REF], [START_REF] Ke | A new representation of skeleton sequences for 3d action recognition[END_REF].

The Convolutional Neural Networks (CNNs) are deep neural network models, which have obtained state-of-the-art results in many tasks due to their ability to act as translation-invariant features extractors. They create a hierarchy of features, progressively more abstract, thanks to the staked convolutional operators. CNNs nowadays have achieved great success in image classification and video action recognition [START_REF] Lecun | Convolutional networks for images, speech, and time series[END_REF], [START_REF] Ciregan | Multi-column deep neural networks for image classification[END_REF]. On the other hand, the Long Short Term Memory networks (LSTMs) which are a special kind of the Recurrent Neural Networks (RNN) [START_REF] Huang | Deep learning on lie groups for skeleton-based action recognition[END_REF], [START_REF] Zhang | On geometric features for skeletonbased action recognition using multilayer lstm networks[END_REF], [START_REF] Liu | Spatio-temporal lstm with trust gates for 3d human action recognition[END_REF], [START_REF] Du | Hierarchical recurrent neural network for skeleton based action recognition[END_REF], [START_REF] Ke | A new representation of skeleton sequences for 3d action recognition[END_REF], are capable of learning long-term dependencies in time series problems like action recognition. However, it is still difficult for LSTMs to memorize the information of entire sequences with many time steps [START_REF] Weston | Memory networks[END_REF] and to extract high-level features [START_REF] Sainath | Convolutional, long short-term memory, fully connected deep neural networks[END_REF]. The combination of CNNs and LSTMs has already been proposed in literature domains (speech, text and image recognition). This work has been motivated firstly by the powerful capability of deep neural networks in learning compact and discriminative representations for images and videos, and second by the successful use of traditional manifold-based analysis in many computer vision applications. In particular, we build a deep neural network architecture performing action learning on Lie Groups. The architecture of our proposed solution captures time dependencies on more progressively abstract features extracted by convolutional operations from the geometric representation offered by Lie Groups.

The rest of this paper is organized as follows. Section 2 presents the related work on skeleton-based action recognition. In Section 3, our approach is described and then Lie group representation and proposed deep network architecture are introduced. Section 4 presents the experimental results and finally Section 5 concludes the paper.

II. RELATED WORKS

3D action recognition has been a widely explored topic in computer vision. We will focus then in reviewing the works we consider relevant to two main categories: handcrafted geometric and deep learning methods based on skeleton data. a) Handcrafted geometric approaches: Before the recent advent of deep learning techniques, most of the works focused on the design of a traditional chain, including handcrafted feature extraction from motion, dynamic modeling and classification. The first type of methods performed action recognition from direct measures of 3D joint parameters of the human body. Indeed, Wang et al. [START_REF] Wang | Mining 3d key-pose-motifs for action recognition[END_REF] developed an algorithm based on key pose mining of each action, which is considered as a set of ordered poses which is required to be close but not necessarily adjacent in the action sequences. The action classification was then performed by matching into the motif of each class and taking the maximum of matching score. Luvizon et al. [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] proposed a new framework to extract spatial and temporal features from subgroups of joints. Then, they aggregated features using k-means classifier, and finally, combined them using a metric learning method.

Besides, many other approaches have been recently proposed to exploit the differential geometry to represent skeleton data, so as to consider the non-linear nature of human motion. In [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF], the authors represented each skeleton as one element on the Lie Group, and the action sequence corresponded thus to a curve on this manifold. To handle rate variability among curves, DTW was employed to temporally align the curves. Finally, Fourier temporal pyramid representation is applied before a linear classification by SVM. The manifold assumption was computationally expensive, that is why [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF] tried a new representation of data. They proposed a method catching the maximum of relationships between skeleton joints. They proposed two kernel tensor representations; Sequence Compatibility Kernel (SCK) and Dynamics Compatibility Kernel (DCK). SCK captured the spatiotemporal compatibility of joints in one sequence against those in the other. DCK explicitly modeled the action dynamics of a sequence. Tensors formed from these kernels were used to train an SVM. Slama et al. [START_REF] Slama | Accurate 3d action recognition using learning on the grassmann manifold[END_REF], [START_REF] Slama | Grassmannian representation of motion depth for 3d human gesture and action recognition[END_REF] exploited no-Euclidean geometric properties to express the time series of skeletons as one point on a Grassmann manifold, where the classification is performed benefiting from the Riemannian geometry of this manifold. Similarly, Devanne et al. [START_REF] Devanne | 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold[END_REF] extended this idea to represent a spatiotemporal motion characterized by full human skeleton trajectory. These motion trajectories are extracted from 3D joints and expressed in R 60 . The action recognition is performed using a K-NN classifier using geodesic distances obtained in open curve shape space.

b) Deep learning approaches: After the recent progress in deep learning techniques, many applications of computer vision field, including action recognition, have shown a change of paradigm. Veeriah et al. [START_REF] Veeriah | Differential recurrent neural networks for action recognition[END_REF] proposed a differential Recurrent Neural Network (dRNN) which represented a gating scheme for the LSTM. Their dRNN emphasized the change in information gain caused by the salient motions between the successive frames. Instead of using the whole skeleton sequence, Du et al. [START_REF] Du | Hierarchical recurrent neural network for skeleton based action recognition[END_REF] divided it into five parts according to the human physical structure. Then, they fed each part separately into bidirectional RNNs/LSTMs. As the number of layers increased, the representations extracted by the subnets were hierarchically fused to build a higher-level representation. The final representation of the skeleton sequences were fed into a singlelayer perceptron, and the temporally accumulated output of the perceptron formed the final decision. Liu et al. [START_REF] Liu | Spatio-temporal lstm with trust gates for 3d human action recognition[END_REF] proposed a spatiotemporal LSTM (ST-LSTM) to explicitly model the dependencies between the joints and apply recurrent analysis over spatial and temporal domains. Besides, they introduced a trust gate mechanism to make LSTM robust to noisy input data. Zhang et al. [START_REF] Zhang | On geometric features for skeletonbased action recognition using multilayer lstm networks[END_REF] investigated a set of simple geometric features using 3-layer LSTMs, and showed that using joint-line distances as input requires less data for training. Lee et al. [START_REF] Lee | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks[END_REF] proposed a new representation of data and they used a Temporal Sliding LSTM (TS-LSTM). In the first step, they transformed the coordinate system of all skeleton using translation, scale and rotation. Then, they extracted salient motion from the new data representation. Finally, they used a TS-LSTM network which calculated the average of multiple parts -short-term, medium term and long term -to get final features.

III. APPROACH

For the problem of human action recognition in the 3D joint space, we propose a deep network architecture, which we refer to as the Long-term Recurrent Convolutional Network on Lie Groups (LRCNLG), to learn the Lie group representations of skeletal data. The network model is then built from stacked units of 1-dimensional CNN across the temporal domain. Then we use the CNN-features to train a LSTM model to recognize action categories. The architecture of the deep neural network of our approach is presented in Figure 1.

A. Lie Group Representation for 3D Skeletal Data

Lie group is a topological group and a smoothed manifold presented as a vector space [START_REF] Hall | Lie groups, Lie algebras, and representations: an elementary introduction[END_REF]. As presented by Vemulapalli et al. [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], the human action can be represented by the movement of the rigid body part. The relative geometry T between two body parts can be described using the 3D rotation R and translation d required to take one body part to the position and orientation of the other. Mathematically, the relative geometry between body parts is presented as a point in Lie group space (SE(3) × ... × SE(3)), and the relative geometry between all pairs of body parts is then presented as a curve in SE(3) × ... × SE(3). However, the classification of the curves in SE(3) × ... × SE(3) into different action categories is a difficult task due to the non-Euclidean nature of the space. That's why the authors in [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF] map between Lie Group and Lie Algebra as presented in the equation [START_REF] Anirudh | Elastic functional coding of riemannian trajectories[END_REF]. where G represents the Lie Group, g its Lie Algebra, u the tangent space vector, e and log respectively represent the usual matrix exponential and logarithm. The Lie Algebra curve from Lie Group is given by an 6M (M -1) vector, where M is the number of body parts. The action is then represented as temporal evolution of d-dimensional vector.

Lexp G (u) = e u , Llog G (g) = log(g) (1)

B. Our LRCNLG model

To learn the features extracted from Lie Groups, we develop a LRCNLG model. The properties of our model follow those of classical CNN and LSTM ones. The LRCNLG model, trainable with stochastic gradient descent (forward and backward), takes as input Lie Group curves (ζ 1 (t)...ζ n (t) where t=0,1,....T). This input is presented as a 3D tensor for variable-length sequences of (m, f )-dimensional vectors where m represent the maximum length for sequences and f is the feature dimension. More precisely, the model is built from stacked units of 1dimensional CNN across the temporal domain followed by LSTMs. The different layers of our model are described below.

First, a layer of Batch Normalization (BN) allows to normalize data and to act like a scale-warping step. It restricts the inputs to follow a normal distribution. Then, a convolution layer is applied across the temporal domain. It is responsible for applying a mathematical computation of discrete convolution on the input feature (X) as 1-dimensional filters (K) presented in the equation ( 3):

(k * X) i = m X i-m k m (2)
The 1-dimensional filter height is fixed to the feature size f and its width to the length m. The kernel size is flipped to obtain the commutative property of convolution operations which leads to less variation of valid values. There are three hyper-parameters which control the output size of convolution layer: the kernel size which controls the number of neurons in the convolution layer that connects to the same region of the input tensor (its size is fixed to 8 in our model), the stride which specifies how many positions apart a filter is moved across the input, and the zero-padding which represents the size of the extending border values outside with 0s. In this implementation, the stride is always equal to 1 and there is no zero-padding performed. After sliding the filter over all the locations, we use a Rectified Linear Activation (ReLu) to the output in order to introduce nonlinearities and to leaves the size of the volume unchanged into the model. It is evaluated to 0 for negative inputs, and positive values remain untouched. ReLUs smooth approximations to the sum of many logistic units and they produce sparse activity vectors. Below is the equation of the ReLu function:

y = max{0, n i=0 w i x i + x 0 } (3)
The last layer in the CNN part of our model is a max-pooling one, which performs a down-sampling operation along the spatial dimensions. The 1-dimensional max-pooling partitions the input tensor data into 1D sub-tensors along the dimension, selects an element with the maximal numeric value in each sub-tensor, and transforms the input tensor to the output tensor by replacing each sub-tensor with its maximum element. In our model, we use four convolution and pooling layers.

Concerning the RNN part of our model, a typical LSTM is adopted to get the contextual dependency in the temporal domain. In our LRCNLG model, We concatenate two LSTM layers, and a dropout layer is then introduced to avoid overfitting problem. Finally, we used dense layer with a softmax function to transform the output codes to probability values of class labels. A dense layer represents a matrix vector multiplication. The values in the matrix are the trainable parameters which get updated during backpropagation. As a result we get a c-dimensional vector as output with c classes.

IV. EXPERIMENTAL RESULTS

This section summarizes all obtained results and provides an analysis of the performances of our proposed approach tested on three public 3D action recognition datasets -UTKinect-Action [START_REF] Xia | View invariant human action recognition using histograms of 3d joints[END_REF], Florence 3D-Action [START_REF] Seidenari | Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses[END_REF] and MSR-Action 3D [START_REF] Li | Action recognition based on a bag of 3d points[END_REF] -and compared with state-of-the-art methods.

A. Implementation Settings

For the feature extraction, we use the code of [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF] to represent action sequence as a Lie Group curve. Each action sequence is then represented as a matrix of 6*M(M-1) lines and f columns, where M is the number of joints per skeleton (20 for MSR-Action 3D and UTKinect, 15 for Florence 3D Action) and f a fixed number of the sequence length (normalized as in [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Vemulapalli | Rolling rotations for recognizing human actions from 3d skeletal data[END_REF] to 76, 74 and 35 respectively for the 3 datasets). We used an initial learning rate of 0.01, stochastic gradient descent with nesterov acceleration with a momentum of 0.9 and a dropout with rate 0.5 after all activation layers to prevent overfitting. We used the Keras deep learning framework with a TensorFlow backend [2] on a laptop with an i5-2320 (3.00GHz) without GPU.

B. Human Action Datasets

UTKinect-Action [START_REF] Xia | View invariant human action recognition using histograms of 3d joints[END_REF]: In this dataset, skeletons were extracted using a single stationary Kinect with Microsoft SDK. Totally, there are 199 sequences whose length varies from 5 to 120 frames. These sequences were recorded from 10 subjects: 9 males, 1 female; one was left-handed. Each subject performed each action twice. There are 10 actions. UTKinect gives information regarding 20 joint locations for the 3D skeleton data. It is a challenging dataset due to the different variations of the views among records and its high intra-class variability. Furthermore, the human object interactions and the absence of some body parts in the view cause different occlusions.

Florence 3D-Action [START_REF] Seidenari | Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses[END_REF]: This dataset was collected using a stationary Kinect sensor at the University of Florence. It includes 9 actions, each one is performed by 10 subjects several times leading to a total of 215 sequences. The sequences were acquired using the OpenNI SDK, skeletons are represented by 15 joints. The main challenges of this dataset are the similarity between actions, the human object interactions and the diverse ways of representing the same action. The later generates a high intra-class variation, for example, the same action is performed using left hand in some sequences and right hand in other sequences.

MSR-Action 3D [START_REF] Li | Action recognition based on a bag of 3d points[END_REF]: This dataset was captured using a depth sensor similar to Kinect at Microsoft research. It contains totally 567 sequences consisting of 20 actions presented by 10 persons facing the camera. Each action is performed 2 or 3 times. It is the most common dataset for 3D action recognition. It contains 20 joints to represent skeleton. Contrary to UTKinect and Florence, there is no interaction between persons and objects for all actions. Actions are taken in the context of gaming which induces many variations of the motions of the legs, arms, torso and their combination. In our experimentations, we use only 557 sequences because there are 10 actions that contain missing information and are erroneous [START_REF] Wang | Mining actionlet ensemble for action recognition with depth cameras[END_REF]. This dataset is challenging due to the strong inter-class similarity and the speed variations for the execution of actions.

C. Action Recognition Analysis

To fairly compare our approach with the state-of-theart methods, we follow the same experimental setup and evaluation protocol presented in these methods. Then, we present our result separately for each dataset.

1) Florence 3D-Action: The two most used protocols for this dataset are Leave-one-subject-out-cross-validation (LOOCV) [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] and cross-validation protocol [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF] for which half of the subjects is used for training and the other Method Protocol Accuracy (%) Lie groups [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF] cross validation 90.88 Kernel Linearization [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF] cross validation 95.23 shape analysis [START_REF] Devanne | 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold[END_REF] LOOCV 87.04 Mining key pose [START_REF] Wang | Mining 3d key-pose-motifs for action recognition[END_REF] LOCCV 92.25 Feature combination [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] LOOCV Fig. 2. Confusion matrix for our approach on Florence 3D Action using (a) LOOCV protocol and (b) cross-validation protocol half is used for testing. To evaluate our proposed method, we choose to use both of protocols to be more extensive. As summarized in Table I, our approach achieves the best accuracy of 95.37% using LOOCV protocol. However, using the cross-validation protocol, it achieves an accuracy of 92.55%, just below the best one which used a kernel Linearization and reported in [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF]. They used a combination of two methods the SCK and DCK kernels making their approach complex. As shown from the confusion matrix in Figure 2, despite the high accuracies for most actions, there are little confusions, like between answer phone and drink from a bottle as they have visually the same trajectory, or between read watch and clap hand, where both need two hands to perform the action.

2) MSR 3D-Action: There are two basic protocols used by the state-of-the-art methods for this dataset: cross-validation protocol and 5-fold protocol which uses subjects 1, 3, 5, 7, 9 for the training while the subjects 2, 4, 6, 8, 10 are used for the testing. In our experiments, we choose to use cross-validation protocol because using the average between 10 splits is more precise than using just one split.

MSR Action 3D is a challenging database that is why generally authors divide it into three subsets and then calculate the average to get the accuracy of this dataset. Action presented in the first subset (AS1) are Horizontal arm wave, Hammer, Forward punch, High throw, Hand clap, Bend, Tennis serve, Pickup and throw. High arm wave, Hand catch, Draw x, Draw tick, Draw circle, Two hand wave, Forward kick, Side boxing are the actions in subset two (AS2). Finally, actions of the subset three (AS3) are High Throw, Forward kick, Side kick, Jogging, Tennis swing, Tennis serve, Golf swing, Pickup and

Method

Protocol Accuracy (%) Lie groups [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF] cross validation 93.52 Kernel Linearization [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF] cross validation 93.96 Grassmann Manifold [START_REF] Slama | Accurate 3d action recognition using learning on the grassmann manifold[END_REF] cross validation 91.21 dRNN [START_REF] Veeriah | Differential recurrent neural networks for action recognition[END_REF] cross validation 92.03 shape analysis [START_REF] Devanne | 3-d human action recognition by shape analysis of motion trajectories on riemannian manifold[END_REF] cross validation 92.1 Mining key pose [START_REF] Wang | Mining 3d key-pose-motifs for action recognition[END_REF] 5-fold 94.40 Feature combination [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] 5-fold 97.1 HBRNN-L [START_REF] Du | Hierarchical recurrent neural network for skeleton based action recognition[END_REF] 5-fold 94.49 TS-LSTM [START_REF] Lee | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks[END_REF] 5-fold 97. II presents the accuracies of our approach obtained on this dataset, compared to the most relevant state-of-the-art methods using different protocols.

As shown, our approach achieve the best accuracy (94.27%) using the cross validation protocol, with 96.64% in AS1, 87.52% in AS2 and 98.71% in AS3. Our accuracy is however, outperformed by [START_REF] Lee | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks[END_REF], [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] where they used only the 5-fold protocol. The confusion matrix presented in Figure 3 gives us more information on each action. Indeed, in AS1, we get the high accuracies for most actions (e.g. Pickup and Throw, Tennis serve), but Hammer is the worst classified action with 83.7% which has been confused with a similar one namely Horizontal arm wave. In AS2, there are many actions that are not perfectly classified e.g., Hand catch with 66.2% and High arm wave with 72.7%. In AS3, all actions are practically well classified with an accuracy over 95%.

3) UTKinect-Action: In this dataset, we kept the same protocols used for Florence 3D-Action dataset. As presented in Table III, our approach achieves the best performance (98.5%) using the LOOCV protocol, compared to all other skeletonbased approaches. As presented in Figure 4-a, all actions have been correctly classified (100%) expect throw (85%). These results prove the robustness of our approach against different challenges of the database. Using cross-validation protocol, we get a performance comparable to [START_REF] Zhang | On geometric features for skeletonbased action recognition using multilayer lstm networks[END_REF] and [START_REF] Lee | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks[END_REF] with an accuracy of 96.68%, but stills 0.4% less than [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF].

If we compare accuracies obtained by our approach to those obtained by the original Lie group representation [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF], we note Method Protocol Accuracy (%) Feature combination [START_REF] Luvizon | Learning features combination for human action recognition from skeleton sequences[END_REF] LOOCV 98 St-LSTM+trust gate [START_REF] Du | Hierarchical recurrent neural network for skeleton based action recognition[END_REF] LOOCV 97 Grassmann Manifold [START_REF] Slama | Accurate 3d action recognition using learning on the grassmann manifold[END_REF] LOOCV 88.5 JLd+RNN [START_REF] Zhang | On geometric features for skeletonbased action recognition using multilayer lstm networks[END_REF] cross validation 95.96 TS-LSTM [START_REF] Lee | Ensemble deep learning for skeleton-based action recognition using temporal sliding lstm networks[END_REF] cross validation 96.97 Lie groups [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF] cross validation 97.08 Kernel Linearization [START_REF] Koniusz | Tensor representations via kernel linearization for action recognition from 3d skeletons[END_REF] cross validation 98. that our approach outperforms them for all actions, except for three ones: Pick up, Carry and Clap hands. These actions involve an interaction with objects and variations in viewpoint which make the classification harder. Note that results reported in [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF] using the LOOCV protocol have showed a lower accuracy of 96.5% compared to our approach. It can be concluded that the LOOCV protocol is the more suitable one for deep learning models as a sufficient quantity of data sequences is needed for training subset, contrary to the case where only half of data is used for training.

D. Effectiveness of Geometric Lie Group Features

To assess the effectiveness of Lie Group features combined with our deep network model, we have repeated all experiments using our model applied directly on the 3D coordinates of skeleton data represented in R 3 Euclidean space. We note here that we have performed a view normalization prior to feeding the 3D coordinates into the model. The results obtained by LRCNLG model are superior to those obtained by the model on R 3 for all the datasets and using the two protocols. Table IV summarizes all the results obtained by our LRCNLG model on Lie Group features compared to those obtained by the same model, we call LRCN, on skeleton coordinates in R 3 , using two different protocols tested on all datasets. This can be explained by the fact that our LRCNLG model learns more adequate geometric features than traditional ones [START_REF] Vemulapalli | Human action recognition by representing 3d skeletons as points in a lie group[END_REF] but also than that with original Euclidean coordinates, for 3D action recognition task.

V. CONCLUSION

This paper addressed the problem of human action recognition in the 3D joint space. We introduced a novel framework, in which we built a neural network model to deeply learn geometric feature captured by Lie Group representations of skeletal data. We then formulated our learning architecture as a hierarchy of spatial CNN features extracted from each Lie Group point, followed by the LSTMs to model dependencies in the Lie Group curve representing the action sequence on a Riemannian manifold. Experimental results on 3 human action datasets consistently demonstrated the effectiveness of the proposed approach.

Fig. 1 .

 1 Fig. 1. The architecture of our LRCNLG model. The network model is built from stacked units of 1-dimensional CNN on Lie Group points across the temporal domain

Fig. 3 .

 3 Fig. 3. Confusion matrix for our approach on MSR Action 3D dataset: (left) AS1, (center) AS2 (right) AS3.

Fig. 4 .

 4 Fig. 4. UTKinect Confusion matrix using (a) LOOCV protocol and (b) crossvalidation protocol

TABLE II OBTAINED

 II RESULTS ON MSR 3D ACTION DATASET throw. Table

TABLE III OBTAINED

 III RESULTS ON UTKINECT-ACTION DATASET USING TWO DIFFERENT PROTOCOLS.

TABLE IV OBTAINED

 IV ACCURACIES BY OUR MODEL ON LIE GROUP REPRESENTATION (LRCNLG) COMPARED TO THE SAME MODEL ON SKELETON COORDINATES IN R 3 (LRCN).