Nicolas Audibert
email: nicolas.audibert@gmail.com

Cécile Fougeron
email: cecile.fougeron@univ-paris3.fr

Distorsions de l'espace vocalique : quelles mesures? Application à la dysarthrie

Keywords: dysarthrie, espace vocalique acoustique, réduction, centralisation, Parkinson, Syndrome cérébelleux, Sclérose Latérale Amyotrophique dysarthria, acoustic vowel space, reduction, centralization, Parkinson, Cerebelar Syndrom, Lateral Amyotrophic Sclerosis

Cet article présente différentes métriques dérivées de mesures F1/F2 pour la description et la quantification de variations observées sur un espace vocalique. 8 métriques issues de la littérature ou adaptées à nos données sont évaluées sur des productions des voyelles /a, e, i, u, o/ extraites d'un texte lu par 78 patients dysarthriques (parkinsoniens, cérébelleux et atteints de SLA) et par 26 locuteurs témoins sains. La capacité des métriques à décrire les altérations de l'espace vocalique associées aux différentes dysarthries par rapport au groupe témoin est comparée. L'interrelation entre les métriques et leur rapport avec l'intelligibilité perçue des patients est également discutée. Les résultats montrent la nécessité de prendre en compte plusieurs métriques complémentaires afin de rendre compte de la multidimensionnalité des altérations possibles dans un espace vocalique.

Introduction

Qu'il soit question de décrire la variation dans la production de voyelles pour une comparaison entre locuteurs, entre langues, entre styles de parole ou en pathologie, le chercheur est confronté au problème de trouver des métriques appropriées non seulement pour la description de la nature des variations observées, mais aussi pour la quantification de différences entre conditions testées. Les variations dans le timbre de voyelles spécifiques sont relativement bien capturées par une observation de la variabilité des formants F1, F2 (et F3).

Lorsqu'il s'agit d'étudier des variations dans la réalisation d'un système vocalique, le problème se complexifie. Il est nécessaire de quantifier des réalisations sur un espace vocalique (projeté en général sur un plan F1/F2) où la variation peut toucher diverses dimensions : (a) réductions de l'espace vocalique dues à un aplatissement sur l'axe F1, un rétrécissement sur l'axe F2, et/ou une centralisation des cibles vocaliques vers un conduit vocal neutre ; (b) dispersions des réalisations au sein d'une même catégorie de voyelles ; et (c) chevauchement dans les réalisations de voyelles de catégories différentes.

Dans cette étude, nous nous intéressons aux altérations dans la production des voyelles chez des patients souffrant de différents types de dysarthries. Ces troubles moteurs de la parole se traduisent par une altération du contrôle des mouvements articulatoires pouvant affecter leur magnitude, vitesse, stabilité ou force. Plusieurs études ont montré l'intérêt d'une analyse acoustique pour l'exploration de ces altérations articulatoires [START_REF] Kent | Acoustic studies of dysarthric speech: Methods, progress, and potential[END_REF]. Pour autant, les études concernant l'articulation vocalique se sont souvent limitées à une mesure de l'étendue de l'espace vocalique (aire du triangle ou quadrilatère) et/ou de centralisation pour capturer l''imprécision articulatoire' telle qu'elle est jugée dans des évaluations perceptives (ex. [START_REF] Turner | Acoustic differences between content and function words in Amyotrophic Lateral Sclerosis[END_REF]. La question qui est posée est le plus souvent de savoir si la métrique, ou quelle métrique, utilisée distingue les patients des témoins ou si elle est corrélée avec une dimension perceptive comme l'intelligibilité (ex. [START_REF] Weismer | Acoustic and intelligibility characteristics of sentence production in neurogenic speech disorders[END_REF].

Il nous semble que ces évaluations acoustiques ne donnent qu'une image partielle des altérations présentes dans la production des voyelles et n'exploitent que très minimalement la richesse des informations articulatoires qui peuvent être inférées à partir de simples mesures des formants F1/F2. En effet, si l'on peut relier une diminution de l'espace acoustique à une réduction de la mobilité des articulateurs, il semble intéressant de savoir si cette mobilité est restreinte dans l'axe antéro-postérieur de la langue et/ou dans l'axe d'ouverture-fermeture du complexe langue/mâchoire. De plus, une dispersion importante entre les réalisations d'une même catégorie vocalique peut être un indice d'instabilité articulatoire. Enfin, une forte centralisation des cibles vocaliques et/ou un chevauchement important entre les ellipses de dispersions des catégories vocaliques peut traduire une perte de contrastes vocaliques. Notre objectif dans ce papier est donc de montrer l'intérêt d'une description prenant en compte les multiples dimensions pouvant être sujettes à variations dans la production des voyelles chez des patients présentant 3 types de dysarthries différentes. Dans l'optique d'une automatisation des traitements, l'extension possible de cette description aux mesures moins robustes de F3 ne sera pas considérée. La pertinence et la complémentarité des métriques étudiées seront évaluées selon leur potentiel à distinguer différentes populations, leur inter-corrélation et leurs liens avec l'intelligibilité perçue.

Méthode

Populations

104 locuteurs ont été sélectionnés à partir d'enregistrement faits à Paris, Aix et Marseille (voir Fougeron et al., 2010). Ils se répartissent en 1 groupe de témoins sains et 3 groupes de dysarthries illustrant des atteintes sur les 3 grands systèmes neurologiques (extrapyramidal, cérébelleux , pyramidal). Pour chacun de ces groupes, la nature différente des troubles peut produire des altérations variables dans l'articulation des voyelles. Le groupe 'GrPark' inclut 30 patients (22 hommes, 8 femmes) souffrant de la maladie de Parkinson et présentant une dysarthrie hypokinétique où rigidité, hypokinésie et hypertonie affectent l'amplitude des mouvements. Le groupe 'GrCereb' inclut 22 patients (14 hommes, 8 femmes) atteints d'un syndrome cérébelleux pur et présentant une dysarthrie ataxique caractérisée par une altération de la coordination temporo/spatiale lors de l'exécution des mouvements. Le groupe 'GrSLA' inclut 26 patients (11 hommes, 15 femmes) atteint de Sclérose Latérale Amyotrophique et présentant une dysarthrie mixte de type paralytique qui se traduit par des mouvements réduits, lents et instables. Enfin le groupe 'GrTem' comprend 26 locuteurs témoins (11 hommes, 15 femmes) couvrant la distribution d'âge des patients. Les productions de ces locuteurs ont été évaluées perceptivement par 10 juges sur différents aspects dont l'intelligibilité (sur une échelle à 4 points avec 3=altération sévère).

Voyelles et métriques acoustiques

1) Aire de l'espace vocalique (pVSA) : Aire du pentagone délimité par les valeurs moyennes de F1 et F2 des 5 catégories de voyelles (en supposant les points ordonnés pour éviter les auto-intersections) :

= ∑ ( 1 2 -1 2 )
2) Ratio de centralisation de formants (cFCR) : Rapport entre les valeurs formantiques supposées s'accroitre avec la centralisation et celles supposées décroître avec la centralisation (FCR adapté pour le français et nos 5 voyelles

) : cFCR=(F2 u +F1 i +F1 u +F2 o )/(F2 i +F1 a +F2 e )
3) Carré moyen de la distance au centroïde du pentagone (CMinter) : Somme des carrés des écarts entre le centroïde de chaque catégorie vocalique et le centre de l'espace vocalique, pondérée par le nombre de voyelles dans les différentes catégories, et normalisée par le nombre de catégories -1 [START_REF] Huet | Contribution à la quantification du degré d'organisation des systèmes vocaliques[END_REF] 4) Ratio d'étendue de F2 (F2RR) : F2RR= F2 i /F2 u (cf. [START_REF] Sapir | Formant Centralization Ratio (FCR): A proposal for a new acoustic measure of dysarthric speech[END_REF] 5) Ratio d'étendue de F1 (F1RR) : F1RR=2 F1 a /(F1 i +F1 u ) 6) Carré moyen de la dispersion intra catégories (CM intra) : Somme des carrés des écarts entre les exemplaires de voyelles et le centroïde de la catégorie vocalique correspondante, normalisée par le nombre de voyelles considérées -le nombre de catégories [START_REF] Huet | Contribution à la quantification du degré d'organisation des systèmes vocaliques[END_REF] 7) Aire totale de recouvrement des ellipses (tOverlap) : Somme de l'aire de recouvrement estimée par échantillonnage des paires d'ellipses correspondant à la dispersion (évaluée par l'écart-type) des différentes voyelles : = ∑ ( ( 1), ( 2)") # 8) Indice Phi d'organisation du système [START_REF] Huet | Contribution à la quantification du degré d'organisation des systèmes vocaliques[END_REF] :

$ℎ = &' &' ⁄
TABLEAU 1 -Description des métriques étudiées Les voyelles étudiées ont été extraites d'enregistrements de parole lue (un texte d'environ 200 mots). 10 à 12 occurrences des voyelles /i, E, a, O, u/ (avec E=/e, ɛ/ et O=/o, ɔ/) ont été sélectionnées de façon à contrôler au mieux le contexte consonantique environnant. Un total de 5746 voyelles ont été segmentées manuellement pour extraire leur durée et une valeur moyenne de leur F1 et F2 prise en trois points (1/3, 1/2 et 2/3) qui a été transformée en bark. Les valeurs de F1 et F2 jugées irréalistes ont été vérifiées et remplacées si nécessaires par une valeur relevée manuellement pour l'ensemble de la voyelle.

Le tableau 1 présente les différentes métriques étudiées. Les métriques 1 à 5 caractérisent la distribution des voyelles dans l'espace défini par F1 et F2. L'aire de l'espace vocalique est calculé sur les 5 voyelles et non sur /i, a, u/ (voir Fougeron et Audibert, 2011 pour une comparaison entre ces deux aires). Deux métriques de centralisation sont testées : la mesure cFCR est une adaptation d'une métrique qui a été utilisée dans des études clinique sur les voyelles /i, a, u/ de l'anglais [START_REF] Sapir | Formant Centralization Ratio (FCR): A proposal for a new acoustic measure of dysarthric speech[END_REF]. Elle repose sur des prédictions spécifiques sur le mouvement de F1 et F2 des différentes voyelles en cas de centralisation ; la mesure CMinter est plus standard [START_REF] Huet | Contribution à la quantification du degré d'organisation des systèmes vocaliques[END_REF]. Les ratios F1RR et F2RR cherchent à capturer des variations dans la mobilité du complexe langue/mâchoire sur un axe d'aperture et de la langue sur un axe antéro-postérieur1 , respectivement. La métrique 6 (CMintra) rend compte de la dispersion des réalisations au sein d'une même catégorie de voyelles et traduit donc la variabilité dans la production des cibles acoustiques. La métrique 7 est une mesure de chevauchement moyen calculé à partir du chevauchement entre les ellipses des voyelles i/e, e/a, a/o, i/u et e/o sur les plan F1 et F2. Cette mesure cherche à refléter une possible perte de contraste entre catégories vocaliques. La métrique Phi introduite par [START_REF] Huet | Contribution à la quantification du degré d'organisation des systèmes vocaliques[END_REF] mesure le degré d'organisation du système inspirée de l'analyse de variance, prenant en compte le rapport entre la dispersion de toutes les catégories de voyelles par rapport au centre de l'espace vocalique et la dispersion au sein d'une catégorie de voyelle (une valeur basse de phi traduirait la désorganisation du système vocalique).

Résultats

Description des populations

Pour chaque métrique considérée comme variable dépendante, l'effet de la population a tout d'abord été évalué par une ANOVA avec la population comme facteur fixé à 4 niveaux, indiquant un effet significatif de la population sur toutes les métriques à l'exception de F1RR. Des comparaisons par paires ont ensuite été effectuées par des tests t avec correction de Bonferroni, afin de quantifier via la taille de l'effet la capacité de chaque métrique à discriminer le groupe témoin des différentes populations dysarthriques. Comme illustré sur le tableau 2 et la figure 1, les GrPark et GrSLA se distinguent du groupe témoins par une réduction de l'aire de l'espace vocalique (pVSA). Celle-ci est associée à une centralisation du système qui est capturée par la métrique CMinter pour le GrPark et par la métrique cFCR pour le GrSLA. Le GrSLA se distingue également du GrTem par une diminution de la plage de F2 traduisant une mobilité réduite de la langue sur le plan antéro-postérieur. Contrairement aux deux autres dysarthries, l'espace vocalique des cerebelleux apparaît comme préservé, sans réduction d'aire, de F1 ou F2, ni de centralisation. Pour autant, cette 'normalité' n'est qu'apparente. Si l'on considère les mesures de dispersion au sein d'une même catégorie vocalique (CMintra) et de chevauchement entre catégorie (tOverlap), une altération de l'articulation des voyelles dans ce groupe par rapport aux témoins apparait. En effet, le GrCereb tout comme le GrPark présentent une forte variabilité entre les exemplaires d'une même voyelle (dispersion mesurée par CMintra) qui pourrait refléter l'instabilité articulatoire propre à ces dysarthries. Ceci contraste avec la dysarthrie dans la SLA qui par son aspect paralytique se traduit par une limitation stable et constante des mouvements. Enfin, comparé au groupe témoin, les 3 groupes dysarthriques présentent un chevauchement accru entre les ellipses de dispersion des différentes catégories vocaliques (tOverlap), qui suggère une perte possible de contrastes vocaliques. D'autre part, seul le GrPark présente une désorganisation du système telle que définie par la métrique Phi, avec une dispersion plus importante des voyelles de chaque catégorie (CMintra plus grand que témoins), et une attirance vers le centre de l'espace vocalique des nuages correspondant aux différentes catégories (CMinter plus faible que témoins

Relations inter-métriques

Les corrélations entre métriques étudiées ne seront pas toutes présentées ni discutées ici par manque de place. Deux types de rapports inter-métriques nous intéresseront.

Premièrement, les métriques qui sont peu corrélées avec les autres sont a priori informatives pour la description des productions vocaliques puisqu'elles ne peuvent pas être prédites par une autre mesure. Dans cette catégorie, on trouve les métriques tOverlap, CMintra. Il est intéressant de comparer ces deux mesures qui pourraient être fortement dépendante l'une de l'autre puisqu'un espace vocalique avec de grandes ellipses de dispersion pour chaque catégorie vocalique (CMintra) pourrait présenter un plus fort chevauchement entre ces ellipses (tOverlap). Or il apparaît dans nos données, illustrées figure 2, que cette relation n'est présente que pour GrSLA (r=.7) pour lequel une augmentation de CMintra s'accompagne d'une augmentation du chevauchement. Pour les autres dysarthries, GrCereb et GrPark, qui se distinguent des témoins par une plus grande dispersion intra-catégorie et un plus fort chevauchement, cette relation est moins nette. Le groupe GrCereb (r=.4) présente un fort chevauchement même pour des valeurs de dispersion intra-catégories moyennes, alors que le groupe GrPark (r=.4) présente un chevauchement moyen avec des ellipses de dispersion très larges. Les relations entretenues entre la mesure tOverlap et les mesures de centralisation sont également intéressantes, en ce sens que chevauchement et centralisation peuvent être interprétés comme un indice de neutralisation des contrastes vocaliques. Pourtant, la mesure tOverlap et les mesures cFCR ou CMinter sont peu corrélées quelque soit la population (r=0 à .4).

FIGURE 3a,b -Distribution des valeurs de pVSA en fonction de F1RR et F2RR par population

Le second type de rapport qui nous intéressera ici est celui entre des métriques présentant de fortes corrélations pour certaines populations mais pas pour d'autres. En effet, s'il n'est pas surprenant d'observer une forte corrélation entre une diminution de l'aire de l'espace acoustique et le degré de centralisation (cFCR ou CMinter) pour tous les groupes (r=.7 à .9), les relations entre la diminution de l'aire et les réductions sur les plans F1 et/ou F2 diffèrent entre populations. Comme illustré sur la figure 3(a), la diminution de l'aire du pentagone est relativement bien prédite par les variations de la plage de F1 (F1RR) pour tous les groupes (r=.8 à .9). En revanche, la contribution de la plage de F2 à l'aire du pentagone (figure 3 b) dépend des populations : faible pour les groupes GrTem (.2) et GrPark (r=.4), cette relation est forte pour les groupes GrSLA (r=.7) et GrCereb (r=.8). Ces deux groupes présentent respectivement la plus petite et la plus large plage de variation de F2. Il apparaît donc qu'une quantification des modifications de l'espace acoustique vocalique basée uniquement sur l'aire ne rende pas compte des réductions conjointes ou indépendantes des dimensions F1 et F2 (et donc de la mobilité linguale dans les deux dimensions).

Concernant les deux mesures de centralisation (cFCR et CMinter) nous avons vu en 3.1 qu'elles permettaient de différencier soit le GrSLA, soit le GrPark du GrTem. Ces deux mesures sont relativement bien corrélées pour le GrSLA (r=.7) mais moins pour les 3 autres groupes (r=.4 à .5). Encore une fois, l'examen du rapport avec les variations sur les axes F1 et F2 est informatif. Si cFCR et CMinter sont bien prédites par les variations sur l'axe F1 pour tous les groupes (r=.8 à .9), seule cFCR est sensible aux variations sur l'axe F2 pour tous les groupes (r=.7 à .9). La diminution de la distance au centroïde mesurée par CMinter est corrélée à la diminution de la plage de variation sur l'axe F2 pour le GrSLA (r=.6) mais pas pour les autres groupes (GrTem : .1, GrPark : .2, GrCereb : .4).

Rapport avec l'intelligibilité perçue

Les relations entre le score perceptif d'intelligibilité et les différentes métriques sont évaluées pour les différentes populations.

Conclusion

Les résultats obtenus confirment les données de la littérature sur l'altération de l'espace vocalique chez les patients parkinsoniens et atteints de SLA, mais pas dans la population des patients cérébelleux pour lesquels les caractéristiques vocaliques sont réputées globalement préservées (ex. [START_REF] Kent | Acoustic characteristics of dysarthria associated with cerebellar disease[END_REF].

Plusieurs des métriques évaluées permettent de discriminer les patients parkinsoniens des témoins sur nos données, en particulier la mesure adaptée aux données du français de l'aire de l'espace vocalique pVSA, l'aire totale de chevauchement des ellipses de dispersion tOverlap et l'indice Phi de niveau d'organisation du système vocalique. Les résultats équivoques décrits dans la littérature pour les parkinsoniens, pour lesquels une réduction vocalique est observée sans permettre une distinction significative du groupe témoin (ex. [START_REF] Weismer | Acoustic and intelligibility characteristics of sentence production in neurogenic speech disorders[END_REF] pourraient s'expliquer par l'emploi de mesures trop globales pour capturer les altérations de l'espace vocalique pour cette population.

Comparativement aux métriques élémentaires fréquemment utilisées, les mesures de dispersion intra-catégorie comme CMintra et de chevauchement des ellipses de dispersion permettent une description plus fine des altérations associées aux différentes formes de dysarthrie étudiées. Ces métriques semblent notamment plus à même de refléter la perte de contraste entre catégories vocaliques. Toutefois la projection sur une unique dimension de l'ensemble d'un système vocalique dans toute sa complexité reste réductrice : les résultats obtenus soulignent ainsi l'intérêt de la combinaison de métriques complémentaires pour permettre une description plus riche des distorsions de l'espace vocalique.

  variations dans la durée des voyelles (durée moyenne ou écart-type) par rapport aux témoins. À contrario, pour les GrSLA et GrCereb, les variations spectrales observées s'accompagnent d'un allongement significatif de la durée des voyelles (particulièrement pour les SLA) et d'une variabilité accrue des durées.

FIGURE 4 -

 4 FIGURE 4 -Distribution des scores d'intelligibilité en fonction de pVSA par population L'intelligibilité est significativement corrélée à l'aire de l'espace vocalique mesurée par pVSA pour GrPark (r=.6) et GrSLA (r=.7). La distribution des scores d'intelligibilité en fonction de pVSA est présentée sur la figure 4. L'intelligibilité perçue n'est significativement corrélée au recouvrement mesuré par tOverlap que pour GrPark (r=.7), tandis que cette corrélation est inférieure à .2 pour les autres groupes. La centralisation mesurée par cFCR est principalement liée à la sévérité perçue pourGrSLA (r=.7, contre .3 pour GrPark et .4 pour GrCereb). L'autre mesure de centralisation que constitue CMinter est plus faiblement corrélée à ce score perceptif de sévérité : r=.5 pour GrSLA, .4 pour GrPark et .1 pour GrCereb. Enfin, l'indice Phi d'organisation du système vocalique n'est faiblement corrélé avec l'intelligibilité que pour GrPark (r=.5).

  ).

		pVSA	cFCR	CMinter F1RR	F2RR	Phi	tOverlap	CMintra
	GrCereb	ns	ns	ns	ns	ns	ns	>** η²=.17	>* η²=.14
	GrPark	<** η²=.16	ns	<* η²=.07	ns	ns	<** η²=.2	>* η²=.19	>* η²=.08
	GrSLA	<* η²=.11	>** η²=.20	ns	ns	<** η²=.16	ns	>* η²=.14	ns
	TABLEAU 2 -Distinction entre groupes dysarthriques et groupe témoin sur les 8 métriques par
	des tests t avec correction de Bonferroni (>, < = tendance par rapport aux témoins ; *=p<.05, ** p<.01 ; et η² = taille d'effet estimée)
	FIGURE 1 -Valeurs moyennes des voyelles	FIGURE 2 -Distribution des valeurs de tOverlap
	/i, e, a, o, u/ sur le plan F1/F2 par		en fonction de CMintra par population
		population
	Aucune distinction entre les dysarthriques et les témoins ne s'observe sur le plan F1. Pourtant
	on peut observer sur la figure 1 une différence qui n'est pas capturée par nos métriques
	acoustiques. En effet, le GrPark se distingue des témoins et des autres groupes dysarthriques
	par un déplacement de l'espace vocalique vers le haut, avec des voyelles /i, e, a, o, u/ plus

fermées (F1 réduit). La réduction de F1 pour /a/ apparaît plus importante que pour les autres voyelles sans toutefois que la mesure de F1RR soit significativement réduite (p=.08). Une autre différence non capturée par nos métriques est le déplacement de l'espace vocalique des cérébelleux sur le plan F2, avec des valeurs de F2 réduites pour les 5 voyelles /i, e, a, o, u/ réalisées plus postérieures et/ou arrondies.

Enfin, il est à noter que les variations spectrales observées dans le GrPark ne sont pas

Les variations de F2 peuvent aussi être liées à des variations d'articulation labiale qui seront négligées ici.

Remerciements

Les auteurs remercient A. Colazo-Simon pour son aide pour la segmentation et L. Lhoussaine pour la mise à disposition des résultats de l'évaluation perceptive experte. Cette étude est financée par le projet ANR DespPhoAPaDy (ANR-08-BLAN-0125).


Pierre-Francois Marteau

Sylvie Gibet

Clément Reverdy

Adaptive Down-Sampling and Dimension Reduction in Elastic Kernel Machines for Efficient Recognition of Isolated Gestures

In the scope of gestural action recognition, the size of the feature vector representing movements is in general quite large especially when full body movements are considered. Furthermore, this feature vector evolves during the movement performance so that a complete movement is fully represented by a matrix M of size DxT , whose element M i, j represents the value of feature i at timestamps j. Many studies have addressed dimensionality reduction considering only the size of the feature vector lying in R D to reduce both the variability of gestural sequences expressed in the reduced space, and the computational complexity of their processing. In return, very few of these methods have explicitly addressed the dimensionality reduction along the time axis. Yet this is a major issue when considering the use of elastic distances which are characterized by a quadratic complexity along the time axis. We present in this paper an evaluation of straightforward approaches aiming at reducing the dimensionality of the matrix M for each movement, leading to consider both the dimensionality reduction of the feature vector as well as its reduction along the time axis. The dimensionality reduction of the feature vector is achieved by selecting remarkable joints in the skeleton performing the movement, basically the extremities of the articulatory chains composing the skeleton. The temporal dimensionality reduction is achieved using either a regular or adaptive down-sampling that seeks to minimize the reconstruction error of the movements. Elastic and Euclidean kernels are then compared through support vector machine learning. Two data sets

Introduction

Gesture recognition is a challenging task in the computer vision community with numerous applications using motion data such as interactive entertainment, humanmachine interaction, automotive, or digital home. Recently, there is an increasing availability of large and heterogeneous motion captured data characterized by a various range of qualities, depending on the type and quality of the motion sensors. We thus separate (i) databases of high resolution and quality built from expensive capturing devices and requiring a particular expertise, (ii) and low resolution and noisier databases produced with cheap sensors that do not require any specific expertise. Such databases open new challenges for assessing the robustness and generalization capabilities of gesture recognition algorithms on diversified motion data sets. Besides the quality of recognition, the complexity of the algorithms and their computational cost is indeed a major issue, especially in the context of real-time interaction.

We address in this paper the recognition of isolated gestures from motion captured data. As motion data are generally represented by high-dimensional time series, many approaches have been developed to reduce their dimension, so that the recognition process is more efficient while being still accurate. Among them, low-dimensional embeddings of motion data have been proposed that enable to characterize and parameterize action sequences. Some of them are based on statistical descriptors [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF], rely on relevant meaningful trajectories [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF], or characterize the style [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF]. In this paper we focus on dimensionality reduction along two complementary axes: the spatial axis representing the configuration of the skeleton at each frame, and the temporal axis representing the evolution over time of the skeletal joints trajectories. With such an approach, two main challenges are combined simultaneously:

• We use relevant trajectories (end-extremities) whose content may characterize complex actions; • Considering that the temporal variability is of primary importance when recognizing skilled actions and expressive motion, we apply an adaptive temporal down-sampling to reduce the complexity of elastic matching.

These low-dimensional dual-based representations will be coupled with appropriate recognition algorithms that we expect to be more tractable and efficient.

Our recognition principle is based on a recent method that improves the performance of classical support vector machines when used with regularized elastic kernels dedicated to time series matching. Our objective is to show how the spatial and temporal dimensionality reductions, associated with such regularized elastic kernels significantly improve the efficiency, in terms of response time, of the algorithm while preserving the recognition rates.

The second section briefly presents the related works and state-of-the-art for isolated gesture recognition. In the third section we describe the nature of the motion data as well as the main pre-processing of the data. The fourth section gives the major keypoints of the method, positioning it in the context of multivariate sequential data classification. We present in the fifth part the evaluation of our algorithm carried out on two data sets with very distinct qualities and compare its performance with those obtained by some of the state-of-the-art methods. A final discussion is provided as well as some perspectives.

Related work

In human gesture recognition, there are two main challenges to be addressed: dimension reduction closely linked to feature descriptors, and recognition models which cover different aspects of dynamic modeling, statistical and machine learning approaches. In this section, we give a brief and non-exhaustive overview of the literature associated with each challenge.

Dimension reduction

The problem of dimensionality reduction (also called manifold learning) can be addressed with the objective to find a low-dimensional space that best represents the variance of the data without loosing too much information.Action descriptors have thus been defined for characterizing whole motion sequences, or punctual frames that need additional step of temporal modeling to achieve the recognition goal.

Numerous method are available to carry out such dimensionality reduction, the most popular being linear approaches such as Principal Component Analysis (PCA, [Jolliffe, 1986], [START_REF] Masoud | A method for human action recognition[END_REF]), Linear Discriminant Analysis (LDA, [McLachlan, 2004]), or linear projections preserving locally neighborhoods (Locality Preserving Projection) [START_REF] He | Locality preserving projections[END_REF]]. Among nonlinear approaches, Locally Linear Embeddings (LLE, [START_REF] Roweis | Nonlinear dimensionality reduction by locally linear embedding[END_REF]), Metric Multidimensional Scaling (MDS, [START_REF] Kruskal | Multidimensional Scaling[END_REF]) and variants like Laplacian Eigenmap [START_REF] Belkin | Laplacian eigenmaps for dimensionality reduction and data representation[END_REF], Isomap [START_REF] Tenenbaum | A global geometric framework for nonlinear dimensionality reduction[END_REF] have been implemented to embed postures in low dimensional spaces in which a more efficient time warp (DTW, see section 5) algorithm [?] can be used to classify movements. An extension of this method, called ST-Isomap, considers temporal relationships in local neighborhoods that can be propagated globally via a shortest-path mechanism [START_REF] Jenkins | A spatio-temporal extension to isomap nonlinear dimension reduction[END_REF]. Models based on Gaussian processes with latent variables are also largely used, for instance a hierarchical version has been recently exploited for gesture recognition [START_REF] Han | Discriminative human action recognition in the learned hierarchical manifold space[END_REF].

Other methods define discriminative features that best classify motion classes. This is the case in the work of [START_REF] Yu | Human action recognition with extremities as semantic posture representation[END_REF]] that reduces the motion data to only five end-extremities of the skeleton (two feet, two hands and the head), thus giving some meaningful insight of the motion related to the action task. [START_REF] Fothergill | Instructing people for training gestural interactive systems[END_REF] and [START_REF] Zhao | Real-time hand gesture detection and recognition by random forest[END_REF] have in particular applied random forests to recognize actions, using a Kinect sensor, while [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] recently proposed to automatically select the most informative skeletal joints to explain the current action. In the same line, [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF] use the covariance matrix for skeleton joint locations over time as a discriminative descriptor to characterize a movement sequence. Multiple covariance matrices are deployed over sub-sequences in a hierarchical fashion in order to encode the relationship between joint movement and time. In [START_REF] Li | Action recognition based on a bag of 3d points[END_REF] a simple bag of 3D points is used to represent and recognize gestural action. Similarly, in [START_REF] Wang | Mining actionlet ensemble for action recognition with depth cameras[END_REF], actionlets are defined from Fourrier coefficients to characterize the most discriminative joints. Finally, it can be mentioned, among many existing applications that address the use of elastic distances into a recognition process, the recent work described in [START_REF] Sempena | Human action recognition using dynamic time warping[END_REF], as well as the hardware acceleration proposed in [START_REF] Hussain | User independent hand gesture recognition by accelerated dtw[END_REF]. However, to our knowledge, no work exploiting this type of distance has directly studied the question of data reduction along the time axis.

Gesture recognition Recognition methods essentially aim at modeling the dynamics of gestures. Some approaches, based on linear dynamic models [START_REF] Veeraraghavan | Role of shape and kinematics in human movement analysis[END_REF], have used autoregressive (AR) and autoregressive moving-average (ARMA) models to characterize the kinematics of movements, while other approaches, based on nonlinear dynamic models [START_REF] Bissacco | Classification and recognition of dynamical models: The role of phase, independent components, kernels and optimal transport[END_REF], have developed movement analysis and recognition scheme based on dynamical models controlled by Gaussian processes. [START_REF] Mitra | Gesture recognition: A survey[END_REF] propose a synthesis of the major gesture recognition approaches relying on Hidden Markov Models (HMM). Histograms of oriented 4D normals have also been proposed in [START_REF] Oreifej | Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences[END_REF] for the recognition of gestural actions from sequences of depth images. [START_REF] Wang | Hidden conditional random fields for gesture recognition[END_REF] have exploited conditional random fields to model joint dependencies and thus increase the discrimination of HMM-like models. Recurrent neural network models have also been used [START_REF] Martens | Learning recurrent neural networks with hessian-free optimization[END_REF]; among them, conditional restricted Boltzman's machines [START_REF] Larochelle | Learning algorithms for the classification restricted boltzmann machine[END_REF] have been studied recently in the context of motion captured data modeling and classification.

In this paper, we propose a new representation of human actions that results from a dual-based reduction method that occurs both spatially and temporally. We couple this representation to a SVM classification method associated with regularized elastic kernels.

We are working on isolated human motions acquired through motion captured databases. In recent years, there is an increasing availability of these databases, some of them being captured by high resolution devices (infrared marker-tracking system), such as those provided by CMU [cmu, 2003], HDM05 [START_REF] Müller | Documentation mocap database hdm05[END_REF], and other ones captured by low-cost devices, such as MSRAction3D captured with the Microsoft Kinect system. With the first type of sensors, the acquisition process is expensive, as it necessitates for capturing skilled motion with a good accuracy several cameras with many markers located on an actor, and a post-processing pipeline which is costly in time and expertise. Instead, with the second type of sensors, the data acquisition is affordable and necessitates less time and expertise, but with a loss of accuracy which remains acceptable for some tasks.

After the recording, the captured data is filtered and reconstructed so that to eliminate most of the noise, data loss and labelling errors (markers inversion), and the output of this acquisition pipeline is generally a set of 3D-trajectories of the skeleton joints determined from the positions of markers. This kind of data is inherently noisy, mainly due to the quality of the sensors and the acquisition process, but also to approximations made during the reconstruction process. The modeling of the skeleton is indeed subject to some variation for different reasons: in particular the markers being positioned on cloths or on the skin of the actor's body, they can move during the capturing process, also the determination of the segment lengths, of the joints' centers and their associated rotation axis are not trivial and lead to modeling errors. To overcome these difficulties, the skeleton model is obtained through an optimization process such as the ones described in [de Aguiar et al., 2006], [START_REF] O'brien | Automatic joint parameter estimation from magnetic motion capture data[END_REF], or [START_REF] Shotton | Real-time human pose recognition in parts from single depth images[END_REF]. The techniques based on a skeleton model hence convert 3D sensor data into Cartesian or angular coordinates that define the state of the joints over time with various accuracies.

Figure 1 presents two skeletons reconstructed from two very distinct capture systems. On the right, the skeleton is reconstructed from data acquired via the Microsoft Kinect, on the left from the Vicon-MX device used by the Max Planck Institute to produce the HDM05 datasets.

Thus, any skeletal-based model can be represented by a hierarchical tree-like structure composed of rigid segments (bones) connected by joints. A motion can be defined by a sequence of postures over time, formalized as a multivariate state vector describing a trajectory over time, i.e. a time series:

{Y t ∈ R k } T 1 = [Y 1 , ...,Y T ],
where the k spatial dimension (k = 3•N, with N the number of joints) typically varies between 20 and 100 according to the capture devices and the considered task. As this state vector is obviously not composed of independent scalar dimensions, the spatiotemporal encoded redundancies open solutions for dimension reduction as well as noise reduction approaches. This is particularly relevant for motion recognition, as the objective is to aim at improving computation time and error rate.

Dimension reduction of motion capture data

Using elastic distances or kernels for recognition problems has proved to be very accurate and efficient. However, one of the main difficulty of such methods with time series data is to deal with their computational cost, in general quadratic with the length of the time series and linear with the spatial dimension characterized by the number of degrees-of-freedom. This high computational complexity is potentially limiting their use, especially when large amounts of data have to be processed, or when real-time constraint is required. We therefore expect that a dual dimensionality reduction, both on the time and spatial axis is particularly relevant, especially when associated to these techniques. Hence, for motion recognition using elastic distances, we propose to show that there exists a spatio-temporal redundancy in motion data that can be exploited.

Considering the quite rich literature on motion recognition, it appears that while some studies have shown success with dimensionality reduction on the spatial axis, very few have directly addressed a dimensionality reduction along the time axis , and much less work by combining the two approaches. [START_REF] Keogh | Scaling up dynamic time warping for datamining applications[END_REF]] has however proposed a temporal sub-sampling associated with dynamic time warping in the context of time series mining, followed later by [Marteau, 2009]. We address herein after these two lines of research.

Dimension reduction along the spatial axis

Dimension reduction (or manifold learning), is the process consisting of mapping high dimensional data to representations lying in a lower-dimensional manifold. This can be interpreted as mapping the data into a space characterized by a smaller dimension, from which most of the variability of the data can be reproduced.

We consider in this paper a more direct approach based on the knowledge of the mechanism underlying the production of motion data and the way human beings perceive and discriminate body movements. We make the assumption that the perception of human motion is better achieved in the so-called task-space represented by a selection of significant 3D joint trajectories. This hypothesis is supported by [START_REF] Giese | Metrics of the perception of body movement[END_REF] who show that visual perception of body motion closely reflects physical similarities between joint trajectories. This is also consistent with the motor theory of motion perception presented in [START_REF] Gibet | Toward a motor theory of sign language perception[END_REF]. Besides, we may reasonably accept that these joint trajectories embed sufficient discriminative information as inverse kinematics (widely used in computer animation and robotics) has shown to be very efficient and robust to reconstruct the whole skeleton movement from the knowledge of the end effector trajectories (hands, feet, head), possibly with the additional knowledge of mid-articulated joints trajectories (such as elbows and knees) and constraints. Hence a straightforward approach consists in constructing a motion descriptor that discards all joints information but the 3D positions of the mid and end effectors extremities. We thus select the 3D positions for the two wrists, the two ankles (the fingers and toes markers are less reliable in general), the two elbows, the two knees and the head. This leads to a time-varying descriptor lying in a 18D space, while a full body descriptor is embedded in a 60D, space for the Kinect sensor, significantly more for vicon settings in general.

Dimension reduction along the time axis

The straightforward approach we are developing to explicitly reduce dimensionality along the time axis consists in sub-sampling the movement data such that each motion trajectories takes the form of a fixed-size sequence of L skeletal postures. Then it becomes easy to perform a classification or recognition task by using elastic kernel machines on such fixed-size sequences. With such an approach, performance rates depend on the chosen degree of sub-sampling. This approach seems coarse since long sequences are characterized with the same number of skeletal poses than short sequences. For very short sequences, whose lengths are shorter than L, if any, we over-sample the sequence to meet the fixed-size requirement. But we consider this case as very marginal since we seek a sub-sampling rate much lower than the average sequence of movement length. In the following, we will experiment and compare uniform and adaptive down-sampling.

Uniform down-sampling

In order to explicitly reduce dimensionality along the time axis, our first straightforward approach here consists in down-sampling the motion data so that each motion trajectory takes the form of a fixed-size sequence of L skeletal postures, evenly distributed along the time axis. We refer to this approach as uniform down-sampling (UDS).

Adaptive down-sampling

The second approach is the so-called adaptive down-sampling (ADS) approach. Similarly to UDS, each motion trajectory takes the form of a fixed-size sequence of L skeletal postures, but these postures are not evenly distributed anymore along the time axis. They are selected such as to minimize a trajectory reconstruction criteria. Basically we follow the previous work by [START_REF] Marteau | Adaptive sampling of motion trajectories for discrete task-based analysis and synthesis of gesture[END_REF]. A data modeling approach is used to handle the adaptive sampling of the {Y t } multidimensional and discrete time series. More precisely, we are seeking an approximation Y θ * of Y such as: where E is the RMS error between Y and Y θ selected among the set of piecewise linear approximations defined from Y . Since the optimal solution of the optimization problem defined by Eq. 1 is O(L.n 2 ), where n is the length of the input time series, we adopt a near to optimal solution as developed in [START_REF] Marteau | Speeding up simplification of polygonal curves using nested approximations[END_REF] whose complexity is O(n). As an example, in figure 2 the human wrist 3D trajectory is down-sampled using 25 samples positioned along the trajectory by minimizing the piecewise linear approximation.

θ * = ArgMin θ (E({Y t }, {Y θ ,t})) (1)
5 Elastic kernels and their regularization Dynamic Time Warping (DTW), [START_REF] Velichko | Automatic recognition of 200 words[END_REF], [START_REF] Sakoe | A dynamic programming approach to continuous speech recognition[END_REF], by far the most used elastic measure, is defined as

d dtw (X p ,Y q ) = d 2 E (x(p), y(q)) (2)
+ Min    d dtw (X p-1 ,Y q ) sup d dtw (X p-1 ,Y q-1 ) sub d dtw (X p ,Y q-1 ) ins
where d E (x(p), y(q) is the Euclidean distance (possibly the square of the Euclidean distance) defined on R k between the two postures in sequences X and Y taken at times p and q respectively. When performed by a support vector machine (SVM) model, the optimization problem inherent to this type of learning algorithm is no longer quadratic. Moreover, the convergence towards the optimorum is no longer guaranteed, which, depending on the complexity of the task may be considered as detrimental.

Besides the fact that the DTW measure does not respect the triangle inequality, it is furthermore not possible to directly define a positive definite kernel from it. Hence, the optimization problem, inherent to the learning of a kernel machine, is no longer quadratic which could, at least on some tasks, be a source of limitation.

Regularized DTW: recent works [START_REF] Cuturi | A Kernel for Time Series Based on Global Alignments[END_REF], [START_REF] Marteau | On Recursive Edit Distance Kernels with Application to Time Series Classification[END_REF] allowed to propose new guidelines to regularize kernels constructed from elastic measures such as DTW. A simple instance of such regularized kernel, derived from [START_REF] Marteau | On Recursive Edit Distance Kernels with Application to Time Series Classification[END_REF] for time series of equal length, takes the following form, which relies on two recursive terms :

K rdtw (X p ,Y q ) = K xy rdtw (X p ,Y q ) + K xx rdtw (X p ,Y q ) K xy dtw (X p ,Y q ) = 1 3 e -νd 2 E (x(p),y(q)) ∑    h(p -1, q)K xy rdtw (X p-1 ,Y q ) h(p -1, q -1)K xy rdtw (X p-1 ,Y q-1 ) h(p, q -1)K xy rdtw (X p ,Y q-1 ) K xx rdtw (X p ,Y q ) = 1 3 ∑      h(p -1, q)K xx rdtw (X p-1 ,Y q )e -νd 2 E (x(p),y(p)) ∆ p,q h(p, q)K xx rdtw (X p-1 ,Y q-1 )e -νd 2 E (x(p),y(q)) h(p, q -1)K xx rdtw (X p ,Y q-1 )e -νd 2 E (x(q),y(q)) (3)
where ∆ p,q is the Kronecker's symbol, ν ∈ R + is a stiffness parameter which weights the local contributions, i.e. the distances between locally aligned positions, and

d E (., .) is a distance defined on R k . The initialization is simply K xy rdtw (X 0 ,Y 0 ) = K xx rdtw (X 0 ,Y 0 ) = 1.
The main idea behind this line of regularization is to replace the operators min and max (which prevent the symmetrization of the kernel) by a summation operator (∑). This leads to consider, not only the best possible alignment, but also all the best (or nearly the best) paths by summing up their overall cost. The parameter ν is used to control what we call nearly-the-best alignment, thus penalizing more or less alignments too far from the optimal ones. This parameter can be easily optimized through a cross-validation.

Normalization

As K rdtw evaluates the sum on all possible alignment paths of the products of local alignment costs e -d 2 E (x(p),y(p))/(2.σ 2 ) ≤ 1, its values can be very small depending on the size of the time series and the selected value for σ . Hence, K DTW values tend to 0 when σ tends towards 0, except when the two compared time series are identical (the corresponding Gram matrix suffers from a diagonal dominance problem). As proposed in [START_REF] Marteau | On Recursive Edit Distance Kernels with Application to Time Series Classification[END_REF], a manner to avoid numerical troubles consists in using the following normalized kernel:

Krdtw (., .) = exp α log(K rdtw (., .)) -log(min(K rdtw )) log(max(K rdtw )) -min(K rdtw ))
where max(K rdtw ) and min(K rdtw ) respectively are the max and min values taken by the kernel on the learning data set and α > 0 a positive constant (α = 1 by default). If we forget the proportionality constant, this leads to take the kernel K rdtw at a power τ = α/(log(max(K rdtw ))log(min(K rdtw ))), which shows that the normalized ker-nel Krdtw ∝ K τ rdtw is still positive definite ( [START_REF] Berg | Harmonic Analysis on Semigroups: Theory of Positive Definite and Related Functions[END_REF], Proposition 2.7).

We consider in this paper the non definite exponential kernel (Gaussian or Radial Basis Function (RBF) types) K dtw = e -d dtw (.,.)/(2.σ 2 ) constructed directly from the elastic measures d dtw , the normalized regularized elastic kernel K τ rdtw , and the nonelastic kernel obtained from the Euclidean distance1 , i.e., K E (., .) = e -d 2 E (.,.)/σ .

Experimentation

To estimate the robustness of the proposed approaches, we evaluate them on two motion capture databases of opposite quality, the first one, called HDM05, developed at the Max Planck Institute, the other called MSR-Action3D, at Microsoft research laboratories.

HDM05 data set [START_REF] Müller | Documentation mocap database hdm05[END_REF] consists of data captured at 120Hz by a Vicon MX system composed of a set of reflective optical markers followed by six high-definition cameras and configured to record data at 120hz. The movement sequences are segmented and transformed into sequences of skeletal poses consisting of N = 31 joints, each associated to a 3D position (x, y, z). In practice the position of the root of the skeleton (located near its center of mass) and its orientation serving as referential coordinates, only the relative positions of the remaining 30 joints are used, which leads to represent each position by a vector Y T ∈ R k , with k = 90. We consider two recognition/classification tasks: HDM05-1 and HDM05-2 that are respectively those proposed in [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] (also exploited in the work of [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF]) and [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF]. For both tasks, three subjects are involved during learning and two separate subjects are involved during testing. For task HDM05-1, 11 gestural actions are processed: {deposit floor, elbow to knee, grab high, hop both legs, jog, kick forward, lie down floor, rotate both arms backward, sneak, squat, and throw basketball}. This constitutes 249 motion sequences. For task HDM05-2 , the subjects are the same, but five additional gestural actions are considered in addition to the previous 11: {jump, jumping jacks, throw, sit down, and stand up}. For this task, the data set includes 393 movement sequences in total. For both tests, the lengths of the gestural sequences are between 56 and 901 postures (corresponding to a movement duration between 0.5-7.5 sec.) .

MSR-Action3D data set:

This database [START_REF] Li | Action recognition based on a bag of 3d points[END_REF] has recently been developed to provide a Kinect data benchmark. It consists of 3D depth image sequences (depth map) captured by the Microsoft Kinect sensor. It contains 20 typical interaction gestures with a game console that are labeled as follows high arm wave, horizontal arm wave, hammer, hand catch, forward punch, high throw, draw x, draw tick, draw circle, hand clap, two hand wave, side-boxing, bend, forward kick, side kick, jogging, tennis swing, tennis serve, golf swing, pickup & throw . Each action was carried out by 10 subjects facing the camera, 2 or 3 times. This data set includes 567 motion sequences whose lengths vary from 14 to 76 skeletal poses. The 3D images of size 640 × 480 were captured at a frequency of From each 3D image a skeletal posture has been extracted with N = 20 joints, each one being characterized by three coordinates. As for the previous data set, we characterize postures relatively to the referential coordinates located at the root of the skeleton, which leads to represent each posture by a vector Y t ∈ R k , with k = 3 × 19 = 57. The task is to provide a cross-validation on the subjects, i.e. 5 subjects participating in learning and 5 subjects participating in testing, considering all possible configurations which represent 252 learning/testing pairs in total.

Hence, we perform three classification tasks: HDM05-1, HDM05-2 and MSRAc-tion3D, with or without a spatial dimensionality reduction while simultaneously considering a down-sampling on the time axis:

• The spatial dimensionality reduction is obtained by constructing a frame (skeletal pose) descriptor composed only with the end-effector trajectories in 3D (EED) comparatively to a full-body descriptor (FBD) that integrates all the joints trajectories that compose the skeleton. The FBD rests in a 90D space for HDM05 and in a 60D space for MSRAction3D. The EED rests in a 24D space (3D positions for 2 elbows, 2 hands, two knees and two feet) for the three tasks, leading to a data compression of 73% for HDM05 and 55% for MSRAc-tion3D.

• The dimensionality reduction on the time axis is obtained through either a uniform down-sampling (UDS) or an adaptive down-sampling (ADS) based on a piecewise approximation of the FBD or EED trajectories. The number of skeletal poses varies from 5 to 30 for each trajectories, leading to an average data compression of 97% for HDM05 and 74% for MSRAction3D.

KE

Fig. 3 Classification accuracies for HDM05-1 task as defined in [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] uniform downsampling, full body (top), adaptive down-sampling, full body (middle), adaptive down-sampling, end effector extremities (bottom), when the number of skeletal poses varies: K E (red, circle, dash), K dtw (black, square, plain), K τ rdtw (blue, star, dotted). Classification accuracies for HDM05-2 task as defined in [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] uniform downsampling, full body (top), adaptive down-sampling, full body (middle), adaptive down-sampling, end effector extremities (bottom), when the number of skeletal poses varies: K E (red, circle, dash), K dtw (black, square, plain), K τ rdtw (blue, star, dotted). (blue, star, dotted). Additionally, the cross validation on subjects (252 tests) allows to show the variance of the results.

Results and analysis

For the three considered tasks, we present the results obtained using a SVM classifier built from the LIBSVM library [START_REF] Chang | LIBSVM: A library for support vector machines[END_REF], the elastic non definite kernel K dtw = e -d dtw (.,.)/(2.σ 2 ) , the elastic definite kernel K τ rdtw , and as a baseline, the Euclidean RBF kernel, K E = e -d 2 E (.,.)/(2.σ 2 ) .

Figures 3,4 and 5 present the classification accuracies for respectively the HDM05-1, HDM05-2 and MSRAction3D tasks for the test data when the number of skeletal postures selected after down-sampling varies between 5 and 30. For these three figures, the top sub-figure presents classification accuracies when the FBD (Full Body) descriptor associated to a uniform down sampling (UDS) are used, the middle sub-figure classification accuracies when the FBD (Full Body) descriptor associated to an adaptive down sampling (ADS) are used, and the bottom sub-figure gives classification accuracies when the EED (End Extremities) descriptor associated to an adaptive down sampling (ADS) is used.

On all three figures, we observe that the down-sampling does not degrade the classification accuracies when the number of poses is over 10, and for some tasks it may even significantly improves the accuracy. This is likely due to the noise filtering effect of the down-sampling. High levels of down-sampling (e.g. when 10 to 15 postures are retained to describe movements, which represents an average compression ratio of 97 % for HDM05 and 70 % on MSRAction3D) lead to very satisfactory results (90 to 98 % for the two HDM05 tasks: Figures 3 and4, and 82 to 86 % for the MSRAction3D task: 5). Best results are obtained for a number of skeletal poses between 15 and 20, when the EED descriptor is used in conjunction with an adaptive down sampling. The SVM classifier constructed on the basis of the regularized kernel K τ rdtw produces the best recognition rates (>= 96 % for the two HDM05 tasks). We note that the MSRAction3D task is much more difficult since it consists in a cross validation based on the performing subjects. Much lower performance are obtained for the SVM built on the basis of the Euclidean distance; in addition, if very good classification rate (96 %) is obtained on the training data, due to the noisy nature of Kinect data and the inter subject variability, the recognition rate on the test data falls down from 82 to 86 % .

Table 1 gives for the MSRAction3D data set and for the SVM based on K E , K dtw and K τ rdtw kernels, means and standard deviations, obtained on the training data (L) and testing data (T), of recognition rates (classification accuracies) when performing the cross-validation over the 10 subjects (5 train -5 test splits leading to 252 tests), for the full body descriptor (FBD) and the end extremities descriptor (EED) associated either to a uniform down-sampling (UDS) or an adaptive Down Sampling (ADS). For this test, movements are represented as sequences of 15 skeletal postures. The drop of accuracies between learning and testing phases is due, on this dataset, to the large inter subjects variability of movement performances. Nevertheless, our experiment shows that the best average classification accuracies (obtained in general with minimal variance) are obtained for the most compact movement rep- resentation, i.e. when the EED descriptor is used associated to an adaptive downsampling. This is true both for the training and testing datasets.

FBD-UDS K E (L) K E (T) K dtw (L) K dtw (T) K τ rdtw (L) K τ rdtw (T)
For comparison, table 2 gives results obtained by different methods of the stateof-the-art and compare them with the performance of an SVM that exploits the regularized DTW kernel (K τ rdtw ) associated to the end extremity descriptor (EED) and an adaptive down-sampling (ADS) of 15 skeletal poses. To that end, we have reimplemented the Cov3DJ approach [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF] to get, for the MSRAc-tion3D data set, the average result given by a 5-5 cross-validation on the subjects (252 tests). This comparative analysis shows that the SVM constructed from regularized DTW kernel provides results slightly above the current state-of-the-art for the considered data sets and tasks.

HDM05-1

Accuracy (%) SMIJ [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] 84.40 Cov3DJ, L = 3 [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF] 95.41 SV M -K τ rdtw , EED, ADS with 15 poses 96.33

HDM05-2 Accuracy (%) SMIJ [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF], 1-NN 91.53 SMIJ [START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF]

MSR-Action3D

Accuracy (%) Cov3DJ, L=3 [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF] 72.33 ± 3.69 2 HON4D, [START_REF] Oreifej | Hon4d: Histogram of oriented 4d normals for activity recognition from depth sequences[END_REF],

82.15 ± 4.18 SV MK τ rdtw 15 poses, 85.72 ± 3.07

Table 2 Comparative study on the MSRAction3D dataset, according to a cross-validation on the subjects (5-5 splits, 252 tests).

2 according to our own implementation of Cov3DJ Finally, in Figure 6, we give the average CPU elapsed time for the processing of a single gestural MSRAction3D action when varying the number of retained skeletal poses. The test has been performed on an Intel Core i7-4800MQ CPU, 2.70GHz. Although the computational cost for the elastic kernel is quadratic, the latency for the classification of a single gestural action using a SVM-K τ rdtw is less than 25 milliseconds when 15 poses are considered, which effectively meets easily real-time requirements.

Conclusion and perspectives

In the context of isolated action recognition, where few studies explicitly consider dimension reduction along both the spatial and time axes simultaneously, we have presented a recognition model based on the dimensionality reduction of the skeletal pose descriptor and the down-sampling of motion sequences coupled to elastic kernel machines. Two ways of down-sampling have been considered: a uniform downsampling that evenly selects samples along the time axis and an adaptive downsampling based on a piecewise linear approximation model. The dimensionality reduction of the skeletal pose descriptor is straightforwardly obtained by considering only end effector trajectories, which is consistent with some sensorimotor perceptual evidence about the way human beings perceive and interpret motion. On the data sets and tasks that we have addressed, we have shown that, even when quite important down-sampling is considered, the recognition accuracy only slightly degrades. In any case, best accuracies are obtained when an adaptive down-sampling is used on the end effector 3D trajectories. The temporal redundancy is therefore high and apparently not critical for the discrimination of the selected movements and tasks. In return, the down-sampling benefits in terms of computational complexity is quadratic with the reduction of the number of skeletal postures kept along the time axis.

Furthermore, the elasticity of the kernel provides a significant performance gain (comparatively to kernel based on the Euclidean distance) which is very important when the data are characterized by high variability. Our results show that a SVM based on a regularized DTW kernel is very competitive comparatively to the stateof-the-art methods applied on the two tested data sets, even when the dimension reduction on the time axis is important. The down-sampling and dimensionality reduction of the descriptor ensures that this approach meets the real-time constraint of gesture recognition.

This study opens perspectives to the use of elastic kernels constructed from more sophisticated time elastic distances [Marteau, 2009] that cope explicitly with time stamped data, associated to adaptive sampling techniques such the one developed in this paper or more sophisticated techniques capable of extracting the most significant and discriminant skeletal poses in movement sequences, based on semantic segmentation. We also aim at testing these powerful tools to more complex tasks, where skilled gestures are studied, or/and expressive variations are considered.

Fig. 1

 1 Fig. 1 Examples of skeletons reconstructed from motion data captured from the Kinect (left) and from the Vicon-MX device used by the Max Planck Institute (right).

Fig. 2

 2 Fig. 2 Trajectory of the human wrist in the 3D Cartesian space adaptively dow-sampled with the localization of the 25 selected samples (red circles): motion capture data (blue) and reconstructed data by linear interpolation (red).

  Fig.4Classification accuracies for HDM05-2 task as defined in[START_REF] Ofli | Sequence of the most informative joints (smij): A new representation for human skeletal action recognition[END_REF] uniform downsampling, full body (top), adaptive down-sampling, full body (middle), adaptive down-sampling, end effector extremities (bottom), when the number of skeletal poses varies: K E (red, circle, dash), K dtw (black, square, plain), K τ rdtw (blue, star, dotted).

  Classification accuracies for the MSRAction3D data set, uniform down-sampling, full body (top), down-sampling, full body (middle), adaptive down-sampling, end effector extremities (bottom), when the number of skeletal poses varies: K E (red, circle, dash), K dtw (black, square, plain), K τ rdtw

		85
	Recognition accuracies (%)	70 75 80									Kdtw K τ rdtw KE
		65
		10	12	14	16	18	20	22	24	26	28	30
						# Skeletal poses
		88
		86
	Recognition accuracies (%)	76 78 80 82 84									Kdtw K τ rdtw KE
		74
		72
		10	12	14	16	18	20	22	24	26	28	30
						# Skeletal poses
		88
		86
	Recognition accuracies (%)	76 78 80 82 84									Kdtw K τ rdtw KE
		74
		72
		10	12	14	16	18	20	22	24	26	28	30
						# Skeletal poses
	Fig. 5

Table 1

 1 Means and standard deviations of classification accuracies on the MSRAction3D data set obtained according to a cross-validation on the subjects (5-5 splits, 252 tests) (L): on the training data, (T): on test data for a number of skeletal postures equal to 15.

	Mean	87,71 69,73	96,04	81,41	96,65	82,50
	Stand. dev. 2,34	5,73	1,36	5,04	1,13	3,22
	FBD-ADS K E (L) K E (T) K dtw (L) K dtw (T) K τ rdtw (L) K τ rdtw (T)
	Mean	85,06 76,48	92,13	84,72	91,89	85,09
	Stand. dev. 3,01	3,18	2,74	3,31	2,66	3,58
	EED-ADS K E (L) K E (T) K dtw (L) K dtw (T) K τ rdtw (L) K τ rdtw (T)
	Mean	92.46 76.27	97.14	85.16	97.19	85.72
	Stand. dev. 3.45	0.96	3.57	5,04	0.93	3.07

  Elapsed time as a function of the number of skeletal poses (10 to 30 poses): i) RBF Euclidean Kernel, Red/round/dashed line, ii) RBF DTW kernel, Black/square/plain line, iii) normalized regularized DTW kernel(K τ rdtw ), blue/star/dotted line.

		x 10 4
		2.2						Kdtw
		2						K τ rdtw
		1.8						KE
		1.6
	Elapsed time (ms)	1 1.2 1.4
		0.8
		0.6
		0.4
		0.2
		10	12	14	16	18	20	22	24	26	28	30
						# Skeletal poses
	Fig. 6

The Euclidean distance is usable only because a fixed number of skeletal positions is considered to characterize each movement, and this, irrespectively of their initial length