Insaf Ajili
email: insaf.ajili@ufrst.univ-evry.fr

Malik Mallem
email: malik.mallem@ufrst.univ-evry.fr

Jean-Yves Y Didier
email: jean-yves.didier@ufrst.univ-evry.fr

Human Motions and Emotions Recognition Inspired by LMA Qualities

Keywords: Motion recognition, Emotion recognition, Laban movement analysis, Features importance, Machine learning, Human perception

The purpose of this paper is to describe human motions and emotions that appear on real video images with compact and informative representations. We aimed to recognize expressive motions and analyze the relationship between human body features and emotions. We propose a new descriptor vector for expressive human motions inspired from the Laban Movement Analysis method (LMA), a descriptive language with an underlying semantics that allows to qualify human motion in its different aspects. The proposed descriptor is fed into a machine learning framework including, Random Decision Forest, Multi-Layer Perceptron and two multiclass Support Vector Machines methods. We evaluated our descriptor first for motion recognition and second for emotion recognition from the analysis of expressive body movements. Preliminary experiments with three public datasets, MSRC-12, MSR Action 3D and UTkinect showed that our model performs better than many existing motion recognition methods. We also built a dataset composed of 10 control motions (move, turn left, turn right, stop, sit down, wave, dance, introduce yourself, increase velocity, decrease velocity). We tested our descriptor vector and

Introduction

Humans communicate with each other and with their environment through different modalities: facial expressions, speech, touch, gestures, etc. The body moves as we talk, think, play, work, etc. It plays an important role in human social interaction. Moreover, technology today integrates the human body for different applications in order to naturally interact with it. For example, the Kinect for Xbox 360 project enables users to control and naturally interact with a video game console using gestures, without the need to physically touch a game controller. The human body is an important medium for emotional analysis and recognition [START_REF] De Gelder | Why bodies? Twelve reasons for including bodily expressions in affective neuroscience[END_REF]. In some cases, body language is a way of expressing feelings even better than spoken language. For example, when we want to express our emotions to deaf peo-ple, when we are in a long distance conversation or in a noisy environment, we are unable to convey our emotions through spoken language. Then, we interpret the emotional state from the sight of the body. Body language involves gestures, body posture and facial expressions. The bulk of research into emotion perception has usually focused on facial expressions as the primary modality through which emotion is conveyed [START_REF] Russell | Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies[END_REF][START_REF] Paul | Facial Expressions. Handbook of Cognition and Emotion[END_REF]. However, during social interactions, emotional face perception can be influenced when integrating body movement. Some researchers [START_REF] Aviezer | Angry, disgusted, or afraid? Studies on the malleability of emotion perception[END_REF][START_REF] Aviezer | The automaticity of emotional face-context integration[END_REF] have demonstrated that body contexts can dramatically influence the recognition of emotions from facial expressions. Therefore, the role of body movement has become of great importance in the field of emotion, and many motion recognition systems have been used in several application fields, such as human-computer interaction, education, affective computing, etc. However, when it comes to whole body expressions, affective movements can be highly variable in different circumstances (energy, intensity, emotion, expression, etc.), even when the same person repeats the same movement multiple times. Such variations are not easy to measure and it is very difficult to find universal algorithms to completely describe the human motion. To cope with such challenges, body movement representation should be considered as the most important step for human motion recognition. It is than essential to find reliable and discriminative features to represent expressive human motions. In this paper, which extends our preliminary work [START_REF] Ajili | Robust human action recognition system using Laban Movement Analysis[END_REF][START_REF] Ajili | Gesture recognition for humanoid robot teleoperation[END_REF], we propose a suitable representation of motions that captures both the quantitative and qualitative aspects of the movement. Two experimental parts are performed: in the first one we evaluate our descriptor vector for motion recognition using three public datasets and our own dataset of control motions. In the second part, we test our descriptor in recognizing human emotions based on expressive gestures. A second study of emotion perception and descriptor evaluation is carried out by humans in order to evaluate the performance of our recognition system.

The main contributions of this work are:

-A motion representation inspired by LMA qualities for motion and emotion recognition. We also evaluate our system against human perception. -An accessible expressive motion database composed of 5 gestures performed with 4 emotions selected from the arousal-valence model [START_REF] Russel | A circumplex model of affect[END_REF] (happy, sad, angry and calm).

This paper is organized as follows: section 2 discusses previous works in this research area. Section 3 describes our proposed system. In section 4, two experimental parts are presented where the performance of the proposed strategy is analyzed and discussed for the motion and emotion recognition processes, respectively. In the second experiment, we test our descriptor ability to characterize expressive motion with two approaches (learning method and statistical method). The paper ends with conclusions and perspectives for future work.

Related works

Human motion recognition is an active field in computer vision with a wide variety of potential applications such as video surveillance, video content analysis, human-computer interaction, robotics, etc. However, the high variability of human appearance and motion is problematic. It is therefore critical to extract robust representations of these variations. Several descriptors have been proposed in the literature based on the type of gesture. We will review two approaches. The first one considers a gesture as a simple movement without integrating the emotional aspect. The second one integrates the expressive dimension to analyze gestures and recognize emotions.

Body motion description

Gong et al. [START_REF] Gong | Structured Time Series Analysis for Human Action Segmentation and Recognition[END_REF] represented human motion sequences by multivariate time series in the joint-trajectories space.

In each frame, the joint-position of several key human body points was formulated as a point in the multidimensional space. Junejo et al. [START_REF] Junejo | Silhouettebased human action recognition using SAX-Shapes[END_REF] proposed 2D representations by extracting silhouettes from actors. Each extracted silhouette was converted to a 1D time series. Jiang et al. [START_REF] Jiang | Online robust action recognition based on a hierarchical model[END_REF] introduced a hierarchical model for action recognition, which consists in assigning each action to a group based on the motion states of each body part. For each group, joints motion and their relative positions were adopted as input to the KNN model for actions classification. Wang et al. [START_REF] Wang | Dense trajectories and motion boundary descriptors for action recognition[END_REF] introduced a descriptor based on motion boundary histograms. They extracted dense trajectories obtained by tracking densely sampled points using optical flow fields. Xia et al. [START_REF] Xia | Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera[END_REF] extracted STIPs from depth videos (DSTIPs) and proposed the Depth Cuboid Similarity Feature (DCSF) to boost the performance of their system. Oreifej et al. [START_REF] Oreifej | HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences[END_REF] proposed a novel descriptor, a histogram of oriented 4D normals, in which they encoded the distribution of the surface normal orientation in the 4D space of depth, time and spatial coordinates.

Expressive motion description

Previous researchers mainly concentrated on the quantitative aspect of motions and have neglected the qualitative side of a movement which is important to recognize high-level gestures and to highlight the expressive qualities of the movements. For example, the difference between punching and reaching for an object is very slight in terms of body structure, both rely on the extension of the arm. But as the dynamics of the movement differs resulting in different intentions behind motions. Also, when performing the same motion with different moods, we need to characterize the quality of the movement to differentiate them. So another field appeared in the literature dedicated to expressive motion analysis aimed at different purposes.

Motion retrieval and synthesis

There is a wide array of research devoted to motion retrieval and synthesis, such as Chi et al. [START_REF] Chi | The EMOTE Model for Effort and Shape[END_REF] which developed a 3D character animation system named the EMOTE model. They employed Effort-Shape LMA components to produce expressive limb motions using empirical mappings between LMA components and motion parameters. This system enables the modification of a predefined motion in terms of its style in order to produce more expressive and natural simulated motions. Likewise, Kapadia et al. [START_REF] Kapadia | Efficient Motion Retrieval in Large Motion Databases[END_REF] encoded motions using keys which represent motion properties based on LMA factors for indexing motions. These keys were then combined to search for complex motions in large motion databases. Müller et al. [START_REF] Müller | Efficient Contentbased Retrieval of Motion Capture Data[END_REF] introduced a set of geometric features describing geometric relations between certain body points of a pose to identify logically similar motions. For the same purpose, Durupinar et al. [START_REF] Durupinar | PERFORM: Perceptual Approach for Adding OCEAN Personality to Human Motion Using Laban Movement Analysis[END_REF] proposed a motion representation inspired by LMA qualities. They performed a mapping between the human motion parameters and different personality traits in an effort to synthesize motions with personality.

Emotion stylization and style transfer The integration of motion style editing or modification research has also been addressed in the computer vision literature. This approach consists in transferring style from one motion to another to obtain a newly synthesized motion. Hsu et al. [START_REF] Hsu | Style Translation for Human Motion[END_REF] used iterative time warping to map from the input to the output sequence. They transformed a motion sequence into a new motion style while retaining its original content. Xia et al. [START_REF] Xia | Realtime Style Transfer for Unlabeled Heterogeneous Human Motion[END_REF] designed a data-driven animation system to add various styles to the existing animation. They constructed a series of local mixtures of autoregressive models to represent the complex relationships between styles and automatically transform an unlabeled heterogeneous mo-tion data into different styles. Ymer and Mitra [START_REF] Yumer | Spectral Style Transfer for Human Motion Between Independent Actions[END_REF] proposed an approach for style transfer based on spectral analysis, which not only handles heterogeneous motion sequences but also transfers style between independent actions. Aristidou et al. [START_REF] Aristidou | Emotion control of unstructured dance movements[END_REF] employed the LMA model to synthesize human motions from existing motion capture data. They extracted quantitative and qualitative characteristics of the movement based on LMA components. They applied the RBF regression model for mapping motion features to their emotion coordinates on Russell's Circumplex Model (RCM) of affect. The same authors [START_REF] Aristidou | Style-based motion analysis for dance composition[END_REF] adopted the LMA model to synthesize motion by calculating both posture and style correlations. They first extracted the movement's features inspired from LMA components and found their stylistic correlations.

Expressive motion analysis and recognition Expressive motion representation requires high-level features describing both kinematics and semantic characteristics, and LMA answers these requirements. It is a formal and universal language developed by Rudolf Laban [START_REF] Rudolf | The Mastery of movement[END_REF]. It allows to qualify human motion in its different aspects. This approach was used for many purposes, such as motion representation [START_REF] Glowinski | Toward a Minimal Representation of Affective Gestures[END_REF], segmentation [START_REF] Bouchar | Semantic Segmentation of Motion Capture Using Laban Movement Analysis[END_REF], style transfer [START_REF] Aristidou | Emotion control of unstructured dance movements[END_REF], etc. Many authors were inspired by the LMA method to design expressive motion descriptors, such as Glowinski et al. [START_REF] Glowinski | Toward a Minimal Representation of Affective Gestures[END_REF] who proposed a minimal representation of expressive movements. They focused only on the upper body. Visual tracking of trajectories of head and hands was carried out from a frontal and a lateral view. In their approach, the PCA indicated four components (roughly indicating motion activity, temporal and spatial excursion of movement, spatial extent and postural symmetry, motion discontinuity and jerkiness) which could characterize the affective behavior expressed by the human in 2D arousalvalence emotion space. Samadi et al. [START_REF] Samadani | Laban Effort and Shape Analysis of Affective Hand and Arm Movements[END_REF] quantified the Effort component and directional movement factor to analyze hand and arm movements. They designed a database composed of 6 motions in order to convey 6 basic emotions (anger, happiness, sadness, fear, disgust, surprise). After, their dataset was annotated by a certified movement analyst (CMA) to study the statistical correlation between the CMA-annotated and the quantified Laban components. Truong et al. [START_REF] Truong | Laban descriptors for gesture recognition and emotional analysis[END_REF] proposed a descriptor based on the LMA model to recognize the gesture and analyze the emotional content of orchestra conductors. For emotion analysis, they asked professional orchestra conductors to annotate each motion by choosing emotions among the categories proposed in their dataset. However, only a a few authors have addressed the issue of automatic classification of emotions expressed in body motions using the machine learning method [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF][START_REF] Senecal | Continuous body emotion recognition system during theater performances[END_REF]. Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF] presented an LMAbased framework to extract motion features, aiming to classify acted dance movements with regard to the expressed emotion. Professional dancers were invited to perform an emotional state, together with music of their choice. In total, they obtained 72 dances, with approximately 130 minutes of motion. Their framework has also been used by Senecal et al. [START_REF] Senecal | Continuous body emotion recognition system during theater performances[END_REF] who created constituent social agents that recognize the emotion of users by mapping motion onto the Russell Circumplex model diagram using neural networks.

In [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF] professional dancers were invited to express their emotions with dance movements. Their gestures are explicity chosen for special emotions, for example happy emotion will be expressed with open arms. So, their algorithm is helped by the type of gesture. In our case, we want to optimize the algorithm so that it is more effective in the case of the same gesture and able to recognize the emotion without being influenced by the type of gesture. So, we constructed a generic descriptor vector for characterizing expressive motions, able to distinguish different emotions expressed through the same motion. We created our database with emotions expressed by non-professional people for our final application referred to Human-Robot interaction. Also we proposed a statistical method based on human in order to evaluate the reliability and the adequacy of our proposed system. Only very few works have addressed the topic of human emotion recognition while performing the same motion, such as Cimen et al. [START_REF] Cimen | Classification of human motion based on affective state descriptors[END_REF]. The authors focused on a single movement task which was "walking". They used the SVM method for classification. In our work, we used a larger dataset composed of 5 gestures acting with 4 emotions, which requires a more robust descriptor able to characterize the movement and its qualities. We constructed a system invariant to initial positions and orientations of people. We evaluated our approach with 4 learning methods. Moreover we studied the importance of our motion descriptor to estimate the pertinence of each motion feature in characterizing each emotion and we compared the results obtained from the automatic learning method against human perception.

The proposed approach

In this section, the proposed human motion recognition approach is detailed. It includes three important steps: data preprocessing, motion representation and classification.

View invariant system

We use the Kinect sensor for data acquisition to extract 3D skeleton joints in real time. The first step after data acquisition is the normalization of all skeletons which consists in aligning all the skeletons in the initial frame. So if two users perform the same gesture from different positions, our system should give the same result. This step makes our system independent of the initial position and orientation of the user. Given an action sequence S = {J t,j , t ∈ 1, . . . , T , j ∈ 1, . . . , N } with T frames and N joints, J t,j corresponds to the coordinates of the joint j captured at frame t.

We define a local coordinate system for the skeleton placed at the hip center joint (J c ), represented by three axes (X , Y , Z ) and equipped with three unit vectors respectively, the left hip joint vector n lhi , the spine vector n s and their cross product n c = n lhi ∧ n s . For each sequence, we first apply a translation to move the skeleton to the center of Kinect, and after a rotation to align both coordinate systems (Figure 1). So the transformed joint yields:

[J t,j ] (X ,Y ,Z ) = R -1 (X,Y,Z)←(X ,Y ,Z ) ([J t,j ] (X,Y,Z) -J 1,c ) (1)
R (X,Y,Z)←(X ,Y ,Z ) = [ n lhi n lhi n s n s n c n c ] (2)
Where:

J 1,c = ([J 1,lhi ] (X,Y,Z) + [J 1,rhi ] (X,Y,Z) )/2 (3)
n lhi = [J 1,lhi ] (X,Y,Z) -J 1,c (4)
n s = [J 1,s ] (X,Y,Z) -J 1,c (5)
[J 1,lhi ] and [J 1,rhi ] refer to the positions of left and right hip joints, respectively, captured at initial time (t=1).

Since the rotation vectors are orthogonal unit vectors, then

R -1 = R T . [J j ] (X ,Y ,Z ) = R T (X,Y,Z)←(X ,Y ,Z ) [J j ] (X,Y,Z) (6)
Once we apply transformations to all sequences, our representation becomes invariant to the position and orientation of the subject in the scene, we convert the skeleton joint data to a descriptor vector based on the LMA method for human motion representation.

Motion representation

Human body motion representation is considered as one of the most important steps in the motion recognition process since each movement must be described with specific features. For example in walking, the motion of the hip joint is considered an important feature for motion description, which is not the case in sitting, where the bending feature is the most expressive features. Our descriptor vector is derived from the LMA model, used as a tool by dancers, athletes, physical and occupational therapists. This approach can describe and interpret all varieties of human movements. LMA provides a rich description of movement, focusing on the four major components of movement: body, space, shape and effort.

The Body describes structural and physical characteristics of the body in movement. It indicates how the whole body is connected and organized and which body parts are influenced by others. In the upper body part, we compute the following angles in left and right parts respectively (Figure 2): between hands and shoulders(θ l 1 , θ r 1 ), between elbows and hips (θ l 2 , θ r 2 ), between elbows and shoulders in the symmetrical part (θ l 3 , θ r 3 ). And we describe outstretched arms by computing the distance between two hands (d Hs ). To know more about the hands pathway, we add three other features, the distances between the head and left (d h,lh ) and right hand (d h,rh ), and the angle between two hands with respect to the shoulder center joint(θ Hs ). In the lower part, we describe leg extension by computing the angle between foot and hip joints, respectively in left and right parts (θ l 4 , θ r 4 ). Also, we describe leg spread by computing the opening angle of the knees (θ Ls ). To encode body quality, three measures of variability are used for each feature: the mean, the standard deviation and the range.

The Space component describes the movement in the space. It defines directions and paths of a movement. We compute the length of the trajectory followed by upper extremities including the head and hands. It is interesting to know the length (L) of joint trajectories which adds more information about motion type. The Shape component describes the way the body changes shape during movement. It is characterized by the body's interaction with itself as well as the space around it. The shape component is composed of three subcomponents: shape flow, directional movement and shaping.

L = T -1 t=1 J t+1 -J t (7)
-Shape Flow refers to how a body can interact and change shape in a self-to-self relationship. We construct the smallest convex envelope of the skeleton based on the Quickhull algorithm [START_REF] Barber | The Quickhull Algorithm for Convex Hulls[END_REF] and compute its volume to characterize the deformation of the skeleton's shape over time. The convex hull offer a more precise fit than a simple bounding box. Figure 3 illustrates the 3D skeleton convex hull and its shape deformation in the "start music" gesture from the MSRC-12 dataset [START_REF] Fothergill | Instructing People for Training Gestural Interactive Systems[END_REF]. In the first frame, the person is in her initial position. After she raises her hands, the volume of the body envelope convex increases. So, by computing the volume of the convex hull we have more idea about body extension. Mean, standard deviation and the range of the 3D convex hull volume feature are computed to quantify the shape flow factor.

Fig. 3 Variation of convex hull feature in "start music" gesture from the MSRC-12 dataset [START_REF] Fothergill | Instructing People for Training Gestural Interactive Systems[END_REF].

-Directional movement defines the pathway of the movement through space. We describe the pathway of the upper extremities, hands and head, by computing their local curvatures. We usually focus on the upper body part, especially hands and head, since they are the most flexible human body parts. We consider the gradual angular change φ occurring between two successive frames as shown in Figure 4, defined as follows:

φ Jt = arccos( ----→ Jt-1Jt ----→ Jt-1Jt • ----→ JtJt+1 ----→ JtJt+1 ) ( 8
)
where J t is the position of joints (hands and head) at frame t and J t+1 at the next frame. This equation describes the local curvature of the upper extremities pathway. From this angle, we derive the curvature feature (C) defined as:

C = T -1 t=2 φ Jt ( 9
)
where T is the number of frames. In straight-line trajectories the Curvature index (C) is close to 0 and describes a linear movement. In curved paths it changes to a very high value. -Shaping describes the qualitative changes in the shape according to three planes: the Horizontal plane describes spreading and enclosing movements; spreading is, for example, a movement of both hands starting in front of the body and opening to the sides; enclosing is the opposite motion, as used in a hug. The Frontal plane characterizes rising and sinking movements and the Sagittal plane includes forward and backward movements. We describe body extension according to the three planes by computing average distances of all skeleton joints (N joints) with respect to the spine joint (J 1,s ) at initial frame (Figure 5).

-For Spreading/Enclosing movements we compute the average spread/enclose movement relating to spine joint position at initial frame.

D H = 1 T T t=1 N i=1 ([J t,i ] X -[J 1,s ] X ) 2 (10)
-For Rising/Sinking movements we compute the average rise/sink movement relating to spine joint position at initial frame.

D F = 1 T T t=1 N i=1 ([J t,i ] Y -[J 1,s ] Y ) 2 (11)
-For Advancing/Retreating movements we compute the average forward/backward movement relating to spine joint position at initial frame.

D S = 1 T T t=1 N i=1 ([J t,i ] Z -[J 1,s ] Z ) 2 (12)
T is the number of frames and [J t,i ] X,Y,Z is the 3D position of joint i at frame t. The Effort component encodes the change in the intensity of exertion during motion. It is a form of expression, mostly related to the change of mood or emotions. It describes the qualitative aspects of movement by identifying the type of energy used in the movement. For example, pushing a heavy object and closing the door are very slight in terms of body organization. Both require the extension of the arm but the intention of the movement is different. The effort component is divided into four factors, each factor ranges between two opposite qualities: Weight (Light/Strong), Time (Sustained/Abrupt), Space (Indirect/Direct) and Flow (Free/Bound). It is the most important element in LMA to describe the quality and the dynamics of the movement. This component is essential, especially for the emotion recognition process. We will focus on four joints (head, spine, left and right hands) since, in our emotions corpus, all expressive gestures were performed by the upper body part.

-The Time factor differentiates between abrupt and sustained motion. Abrupt movement involves less time consumption, so it is a high-speed and Sustained movement is a continuous with constant velocity. It can be represented as the change of acceleration of movement. So, to quantify this factor we compute the mean, the standard deviation and the range of velocity of upper body joints (head, spine and hands). -The Weight factor is about the level of force or pressure used throughout a motion with the contrasting elements: Strong like a punching action or Light like dabbing paint on a canvas action. Light movement is characterized by an invariant rhythm of motion. We can describe it with zero acceleration. Strong movement requires force and acceleration. We compute the accelerations of head, spine and hands to quantify the weight factor. Like the Time factor, mean, standard deviation and the range of the acceleration are computed to present the Weight Effort factor. -The Flow factor characterizes the attitude towards bodily tension and control, with the contrasting elements Free and Bound. We compute the yaw and pitch range of head and hands motions. For Free motion, we will obtain a higher range compared to a Bound motion. -The Space factor characterizes the quality of attention that the person make in a movement and distinguishes a direct movement from an indirect movement. A direct motion is characterized by a single destination where all your attention is focused on a single spatial possibility. For example, when pointing to a particular object we have a Direct gesture. Indirect motion represents a widening of focus like the waving away bugs action. We describe the directness of the upper body parts movements by computing the Straightness index (S) of head and hands motions as the ratio between the distance between the positions of the articulation at the first and last frame (D) and the sum of the distances between successive frames (L) for head, spine and hands joints.

S = D L (13)
A Straightness Index close to 1 means there is a Direct motion, and is close to 0 in Indirect motion.

After encoding all the LMA components, we obtain a descriptor vector composed of 85 features.

Experiments

This section describes our experiment which is divided in two parts. The first one is about the motion recognition where we evaluated our model with public databases (MSRC-12 [START_REF] Fothergill | Instructing People for Training Gestural Interactive Systems[END_REF], Utkinect [START_REF] Xia | View invariant human action recognition using histograms of 3D joints[END_REF] and MSR Action 3D [START_REF] Li | Action recognition based on a bag of 3D points[END_REF]) and our dataset named CMKinect-10. In the second part, we tested our model for emotion recognition with our dataset composed of expressive motions.

We analyzed the importance of our descriptor vector towards each expressed emotion. Both processes have been carried out with learning and statistical methods.

Motion recognition

CMKinect-10

CMKinect-10 is a dataset composed of 10 control motions as shown in Figure 6. Twenty subjects (10 men and 10 women) from the University of Evry Val d'Essonne, ranged in age from 27 to 36 years old (M=29.85 years, SD=2.47), took part in this study. Each subject is asked to make a gesture ten times. This dataset has in total 2000 sequences (20 subjects × 10 motions × 10 times). For data acquisition, the OpenNI driver under ROS (Robot Operating System) provided a high-level skeleton tracking module. This module requires initial calibration to record the 3D position of skeleton joints at 640 × 480 resolution at 30 fps.

Learning method parameter adjustment

For learning method parameter adjustment, we first used the MSRC-12 dataset [START_REF] Fothergill | Instructing People for Training Gestural Interactive Systems[END_REF] and we computed the F-score with a cross-validation test.

Random Decision Forest is an algorithm for classification that consists of many decision trees. During tree induction, nodes are subdivided progressively into child nodes and the tree is then grown until the node becomes pure or when a maximal depth is reached. There are various functions to measure the quality of a split, such as the information gain [START_REF] Quinlan | Learning With Continuous Classes[END_REF] or the Gini impurity [START_REF] Breiman | Classification and regression trees[END_REF]. We varied the number of trees from 10 to 200. The best result of 94% was obtained with 100 trees. The other parameters were set in their default values: for the maximum depth the nodes are expanded until all leaves are pure or until all leaves contain less than 2. The number of features to consider when looking for the best split is equal to √ 85. The minimum number of samples required to split an internal node is equal to 2, and the minimum number of samples required to be at a leaf node is equal to 1 as recommended in [START_REF] Diaz-Uriarte | Gene selection and classification of microarray data using random forest[END_REF].

Multiclass SVM was originally designed for binary classification. The often suggested implementations for SVM multiclass classification are One Against Multi-Layer Perceptron is a feed-forward neural network, consisting of a number of neurons organized in several layers (one input and one output layer with one or more hidden layers). The input layer receives an external activation vector and passes it through weighted connections to the neurons in the first hidden layer. These compute their activations and pass them to neurons in the following layer. To find the optimal number of hidden neurons, we varied the number starting from 10 to 100 neurons. For the activation function, we compared between three functions (logistic sigmoid, hyper-bolic tangent and Rectified linear unit function). The MLP method utilizes a supervised learning technique called backpropagation. Two Backpropagation parameters were adjusted, the learning rate ( ) which controls the step-size in updating the weights, it should be between 0 and 1 and the momentum (m) to accelerate the convergence network while avoiding instability. Best results were achieved with a single layer of 60 hidden neurons and a logistic sigmoid activation function. For the backpropagation algorithm, optimal parameters were obtained by setting m = 0.9 and = 0.001 which yield a recognition rate of 93%.

Experimental results of motion recognition

MSRC-12 dataset

The performance of the system is measured in terms of precision and recall with the F-score [START_REF] Hripcsak | Technical Brief: Agreement, the F-Measure, and Reliability in Information Retrieval[END_REF]. We performed 5-fold cross-validation: the dataset is randomly divided into 5 equal groups. For each of the 5 experiments, 4 folds are used for training and the remaining one for testing. The cross-validation process is then repeated 5 times, with each of the 5 folds used exactly once as the test set. The performance measure is then obtained by the average of the values computed in the loop. Table 1 illustrates recognition rates obtained in MSRC-12 for each learning method. Best results are Table 1 Recognition rates (%) of our method compared to the state of the art methods on the MSRC-12 dataset.

Methods

Iconic Metaphoric Lehrmann et al. [START_REF] Lehrmann | Efficient Nonlinear Markov Models for Human Motion[END_REF] 90.90 -Song et al. [START_REF] Song | Distribution-sensitive learning for imbalanced datasets[END_REF] 79.77 57.21 Truong et al. [START_REF] Truong | Dynamic Gesture Recognition with Laban Movement Analysis and Hidden Markov Models[END_REF] 88 achieved with the RDF method with recognition rates of 93% in the metaphoric category and 99% in iconic gestures. Figure 7 illustrates the confusion matrix for MSRC-12 dataset when using the RDF method. The diagonal elements represent the number of elements where the predicted values are equal to the expected values, while off-diagonal elements are those that are mislabeled by the classifier. The higher the proportion of values on the diagonal of the matrix in relation to values off of the diagonal, the better the classifier. For example, the "wind up the music" gesture was correctly recognized on 98% of the data, while it was incorrectly recognized as "throw an object" gesture by 2%. From the confusion matrix, we can conclude that our proposed descriptor is very good for discriminating the different gestures. We also compared our result to state of the art methods in Table 1. As we can see, our method outperforms their methods, which confirms the robustness of our descriptor in characterizing both iconic and metaphoric gestures.

MSR Action 3D dataset

To compare the performance of our descriptor with state of the art methods on the MSR Action 3D dataset, we followed the cross-subject test setting where half of the subjects are considered as training data and the rest as testing data. As shown in Table 2, we obtained recognition rates close to the state of the art methods, 89.5%, 89% and 88.1%, with OAO, OAA and MLP, respectively. The dataset is divided in three subsets AS1, AS2 and AS3. The subsets AS1 and AS2 include actions with similar movements, while AS3 groups complex actions. As a learning method, we used the one that gave the best recognition results, the RDF method. Overall accuracies of 90.3%, 88.7% and 93.1% were achieved for AS1, AS2 and AS3, respectively. Figure 8 shows Table 2 Recognition rates (%) of our method compared to the state of the art methods on the MSR Action 3D dataset.

Methods

Recognition rates (%) Xia et al. [START_REF] Xia | Spatio-temporal Depth Cuboid Similarity Feature for Activity Recognition Using Depth Camera[END_REF] 89.30 Slama et al. [START_REF] Slama | Grassmannian Representation of Motion Depth for 3D Human Gesture and Action Recognition[END_REF] 86.21 Oreifej et al. [START_REF] Oreifej | HON4D: Histogram of Oriented 4D Normals for Activity Recognition from Depth Sequences[END_REF] 88.89

Our method RDF 90.5 OAO 89.5 OAA 89 MLP 88.1

the confusion matrices for AS1, AS2 and AS3, respectively, when using the RDF method. The AS1 subset contains high similar actions, which explains some misclassifications that we found in this subset, for example, the confusion between the following actions: Horizontal arm wave and Hammer, Forward punch and High throw. Also, actions in AS2 are likely to be confused, for example, "draw tick" and "draw circle", since they are quite similar. However, we get perfect recognition results for actions in AS3.

UTkinect dataset

To compare our results with the state of the art approaches on the UTKinect dataset, we followed the experiment protocol proposed by Xia et al. [START_REF] Xia | View invariant human action recognition using histograms of 3D joints[END_REF] where they applied Leave-One-Out Cross Validation (LOOCV) [START_REF] Arlot | A survey of cross-validation procedures for model selection[END_REF] and achieved an overall mean accuracy of 90.92% and the best accuracy of 95%. As shown in Table 3, our method outperforms their results by a factor of about 1% when using the RDF classifier and the same when applying MLP and OAA methods, but just 1.8% lower with the OAO method. The resulting confusion matrix with the RDF method is shown in Figure 9. Some confusions occurred between some classes, like the confusion between "throw" and "push" classes. On the other hand, we achieved recognition rates of 100% in the following action classes: stand up, pull, wave hands and clap hands. Fig. 9 Confusion matrix of our proposed approach for UTkinect dataset.

CMKinect-10 dataset

We evaluated our descriptor with our dataset composed of 10 control motions. Using the four learning methods for motions classification. We used the 5-fold cross-validation technique to assess the predictive performance of our model. Table 4 summarizes recognition results obtained for each learning method. As we can see, we have almost the same result when applying the four learning methods, the mean of the F-scores was close to 1. The confusion matrix of our control motion dataset is illustrated in Figure 10. We obtained perfect recognition results for the following motions: dance, stop, turn left and turn right.

In conclusion, we evaluated our descriptor against different public datasets and we can confirm the robustness of our descriptor in human motion recogni- tion. Now, we will evaluate how well it human emotions expressed through body motions.

Emotion recognition and analysis

Based on learning method

The purpose of this study is to build an emotional dataset composed of 4 basic emotions (happy, angry, sad and calm) performed with expressive motions. Such a dataset can be used in a robotic application, for example for robot teleoperation in an expressive way which makes the interaction between human and robot more natural. In such a case, the robot will be able to recognize not only the actions of the human but also his mood. In the experimental part, we will first evaluate the pertinence of our descriptor in characterizing human emotions based on learning methods and viewer ratings. We will also study the relationship between body features and expressed emotions in order to conclude the importance of each Laban feature to characterize each emotion.

Emotion classification: Our emotional dataset is composed of 5 control motions selected from the CMKinect-10 dataset (move, turn left, turn right, wave and stop) described in section 4.1.3. Each motion is acted with 4 different emotions (happy, angry, sad and calm). The same subjects who participated in the previous dataset (CMKinect-10) took part in this study. We proposed scenarios presenting emotional situations. Each participant was asked to read the proposed scenario and take the time to feel the situation. After each participant was asked to stay in front of the kinect and perform each motion while acting the situation. Two examples of scenarios proposed respectively for sad and happy emotions were:

-Someone calls you and informs you that your best friend is dead, so you feel very sad. -You are selected in a game and you will win a fortune, so you feel very happy.

During the recording sessions, the order of scenarios, emotions and motions were randomized from one participant to another. We recorded 400 videos (20 participants × 5 motions × 4 emotions). As classification methods, we used the same four methods applied for motion recognition. RDF trained and tested with 5 fold cross-validation achieves the best result compared to OAO, OAA and MLP with recognition rates of 89%, 78%, 71% and 71%, respectively (Table 5).

Features importance: As shown in the section above, our motion descriptor has successfully classified the movements in the different public datasets and the emotions in our expressive motions dataset when applying the different learning methods. Now, we want to study the importance of chosen features and see if there are redundant features that contribute less to the performance of our descriptor. In the expressive motions classification step, some authors have also discussed this point and tried to characterize expressive gestures through body movement parameters. The most relevant is the one of Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF] which is also based on the LMA model to quantify motion features. Compared to them, we work on short gestures (4 seconds of du-ration) instead of complete dances during between one to two minutes. This is a strong requirement of our application which is in the context of human-robot interaction. As the movement sequence is significatively shorter, it means the amount of data is reduced. To coped with this constraint, we decided to rely on all components of LMA. Whereas it is not the case for Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF]. That is why we made a faithful association with the LMA model. We quantified all LMA factors. For example, in the Shape component, following the definition of LMA, we quantified the three Shape factors. Unlike Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF], we considered the directional movement factor as important since it allows us to describe the attention that the person carries in a movement. Also, for the Shape flow factor, we constructed the skeleton convex hull that offers a more precise fit than a simple bounding box and we computed its volum to characterize the body shape deformation during movement. For the Space Effort factor, the authors considered a movement as direct when the character is moving in the same direction as the head orientation. They computed the angle between the head's orientation and the trajectory of the root joint. However, based on our statistical experiments, if we make an anger pointing gesture with the hand while keeping the head not oriented to the pointing direction means a direct movement. To avoid a such confusion between two extreme qualities of the space factor (direct and indirect), we described the directness of the upper body parts (head, spine, and hands) by computing the Straightness index (S) of each articulations. For the Space component, we quantified the trajectory of the root joint as Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF] but it is also important to know about the trajectories of hands in the space to characterize emotions from body movement. For example, in the pointing gesture a long trajectory of the hand indicates sadness whereas a short trajectory can indicate anger emotion.

In the next part, we will evaluate the importance of our features. Aristidou et al. [START_REF] Aristidou | Emotion Analysis and Classification: Understanding the Performers' Emotions Using the LMA Entities[END_REF] studied the importance of each feature in their descriptor. In our case, we prefer to study the importance of the set of features to find which combinations of features give the best classification result. We used the RDF method to measure the importance of each feature. During the training phase, each tree is grown using a different bootstrap samples from the original training data, leaving 1/3 as OOB (Out Of Bag) to estimate the prediction error. The importance of the feature f i is measured as the difference between the OOB prediction error of each tree before and after permuting f i . We started with the whole set of features and we computed the OOB error rate. After, we sorted the features in descending order of importance, and removed the features with low importance. We recomputed the OOB error rate of the new set of features. We repeated the process until there remained only one feature in the set. Finally, we sorted all OOB error rate results obtained in each subset of features and we selected the one that corresponds to the smallest OOB error rate. As we can see in Figure 11, with the whole set of features (85 features), we obtained a minimal OOB error rate of 0.04, which confirms the importance of the combination of all features to characterize our expressive motion dataset.

Human perception

Emotions classification: We tried to make an external evaluation of the emotions expressed in the videos. We recruited 20 viewers (10 men and 10 women) from the University of Evry Val d'Essonne, ranged in age from 28 to 37 years old (M=30.9 years, SD=3.16). They watched recorded videos and rated the degree to which they perceived the expressed emotion using a 5-item Likert scale (1=strongly disagree, 3=neutral, 5=strongly agree). For a reliable evaluation, expressive motions in videos were reproduced by a virtual avatar as shown in Figure 12 which represents the moving gesture acted with happy and sadness emotions. This helps viewers to make scores without being influenced by some factors like facial expressions, gender, etc. Table 6 illustrated the mean of perception ratings given by all viewers for each expressed emotion. Each cell represented the score (s) computed with the following formula:

s = 1 N 1 V N i=1 V v=1 s iv
N is the number of expressed emotions, V is the number of viewers, and s iv refers to the perception rating given by viewer v for the expressed emotion i, using the 5-item Likert scale. To trust the evaluation of viewers we check whether viewer ratings were due to chance and if there was inter-viewer consistency. So, we computed the Cronbach's alpha, a measure used to assess the reliability of a set of scale, it is expressed as a number between 0 and 1. According to [START_REF] Bland | Statistics notes: Cronbach's alpha[END_REF], the acceptable range is between 0.70 to 0.95 [START_REF] Bland | Statistics notes: Cronbach's alpha[END_REF]. In our case, the mean of Cronbach's alpha was greater than 0.8 for all emotions (happy 0.895, angry 0.916, sad 0.895, calm 0.873) which indicates the high consistency between the viewers in rating emotions.

Fig. 12 Move gesture reproduced by a virtual avatar, expressed with happy and sad emotions. Features evaluation: Once we have finished emotion evaluation we proceeded to feature rating in each observed video in order to study the correlation between emotions and body features. We asked the same 20 viewers to watch the same videos and rate each selected feature using a 7-point Likert scale. Cronbach 's alpha has been used to determine the internal consistency between viewers when evaluating features. We obtained values higher than 0.7 (Time 0.958, Weight 0.874, Space 0.823, Flow 0.885, Shaping 0.914, Shape Flow 0.843). So, we can say that we had a very good reliability between viewers in feature rating, except in the Directional Movement quality where we obtained a value of 0.570 inferior to 0.7 which indicates an unsatisfactory internal consistency reliability between viewers. We can explain this by the type of motions selected which makes the distinction between direct and indi- Table 8 presented the correlation results obtained between LMA factors and emotions. The scores greater than 0.3 are written in bold (a correlation coefficient of 0.3 is considered as a moderate correlation). From this table, we can notice that the Directional Movement quality has the least importance in characterizing human emotions in expressive control motions.

Discussion

The purpose of this statistical part is to study the recognition of emotions through body movement, and subsequently to study the importance of body features to characterize human emotions. For the first study, according to the results in Table 6, we can notice that viewers confuse Happy and Angry emotions both have high arousal and differ in valence. Also, there is a confusion between Sad and Calm emotions sharing a low arousal and differing in valence. But generally the high scores are presented in the diagonal of the confusion matrix of emotion recognition which indicates the success of viewers in classifying emotions. Same results are obtained with learning methods especially with the RDF method achieving a f-score of 0.89. For the second study, according to results illustrated in Table 8, we can notice that Happy and Angry emotions were associated with similar features but with different importance. The Angry emotion was strongly characterized by abrupt, strong and free movements. However, the Happy emotion was significantly characterized by growing shape and extended body members. On the other hand, Sad and Calm emotions had the same feature qualities but with different importance. The Sad emotion was strongly associated with sustained, light and bound movements with shrinking shape and contracted body limbs. The Calm emotion was strongly characterized by bound movements. According to the scores given by the viewers we can say that most of the LMA characteristics are important for the characterization of emotions with the exception of the Directional movement factor. However, with the RDF method we found that the combinaison of all LMA features is important. This can be explained by the type of our gestures where in some cases the human is unable to differentiate between rectilinear and curvilinear movement for a same movement acting with two different emotions.

Despite having recruited non-professional actors to express emotions for the construction of the database, the viewers were able to distinguish the Laban features in the movements. This may be an advantage for the application of our framework to be used even by nonexperts. On the other hand, we still have the opportunity to enhance the consistency between viewer ratings with some training Laban's theories and techniques.

Conclusion and perspectives

In this paper, we have proposed an expressive motion representation based on the Laban Movement Analysis method. Our motion descriptor was inspired by LMA

Fig. 1

 1 Fig. 1 Kinect and local coordinate systems.

Fig. 2

 2 Fig. 2 Body features.

4 Fig. 4

 44 Fig. 4 Gradual angles between successive frames.

Fig. 5

 5 Fig.5Distance variation relating to the spine joint, in the "crouch" gesture from the MSRC-12 dataset[START_REF] Fothergill | Instructing People for Training Gestural Interactive Systems[END_REF].

Fig. 6

 6 Fig. 6 CMKinect-10 dataset.

Fig. 7

 7 Fig. 7 Confusion matrix for MSRC-12 dataset.

Fig. 8

 8 Fig. 8 Confusion matrices for MSR Action 3D dataset.

Fig. 10

 10 Fig. 10 Confusion matrix for CMKinect-10 dataset.

Fig. 11

 11 Fig. 11 Optimal features set giving the minimal value of the OOB error.

Table 3

 3 Recognition rates (%) of our method compared to the state of the art methods on the UTkinect dataset.

	Methods				Recognition rates (%)
	Xia et al. [36]			95
				RDF		96
	Our method	OAO OAA		93.2 95.3
				MLP		95
	walk	0.96			0.04
	sit down		0.96		0.03			0.01
	stand up		1.00
	pick up		0.09		0.91
	carry	0.10			0.90
	throw						0.95	0.04	0.01
	push						0.04	0.96
	pull							1.00
	wave hands							1.00
	clap hands							1.00
		w a lk	s it d o w n s ta n d u p	p ic k u p	c a rr y	th ro w	p u s h	w a v e p u ll h a n d s c la p h a n d s

Table 4

 4 CMKinect-10 dataset: The mean and standard deviation of F-scores results for each learning methods (RDF, OAO, OAA and MLP).

Table 5

 5 Emotions dataset: The mean and standard deviation of F-scores results for each learning methods (RDF, OAO, OAA and MLP).

	Methods	RDF	OAO	OAA	MLP
	Mean	0.89	0.78	0.71	0.71
	F-scores	(+/-0.05)	(+/-0.06)	(+/-0.13)	(+/-0.12)

Table 6

 6 Confusion matrix of emotion recognition based on viewer ratings.

	Expressed		Perceived emotions
	emotions	Happy Angry Sad	Calm
	Happy	3.08	2.16	1.77	2.04
	Angry	2.29	3.27	1.80	1.81
	Sad	1.67	1.81	3.41 2.15
	Calm	2.14	1.95	2.28	3.17

Table 7

 7 Relationship between features and left/right extremities of Likert scale. We used the Pearson correlation to evaluate wether body features are associated with expressed emotions. It ranges from -1 for a perfect negative correlation to +1 for a perfect positive correlation, 0 representing no correlation. We focused on the two factors of LMA, Shape and Effort, which describe the qualitative side of the movement. Table8summarizes the relationship between the four emotions and Effort-Shape features: The Happy emotion was significantly characterized by Shape factors, there was a moderate positive correlation between Happy emotion and Shape factors. It is then characterized by the growth of the shape and the extension of body members. About Effort factors, there was a moderate positive correlation with Abrupt Time quality and a moderate negative correlation with Bound Flow quality. The Angry emotion was characterized by a moderate positive correlation with Shape factors. Between Angry emotion and Effort features, there was a strong positive correlation with Abrupt Time and Strong Weight and a strong negative correlation with Bound Flow quality. The Sad emotion was significantly characterized with Contracted and Shrinking Shape, Sustained, Bound and Light movement. The Calm emotion there was a low correlation between calm emotion and most LMA features. We found a weak negative correlation with Shape factors. There was a moderate negative correlation with Time and Weight factors and a positive correlation with Bound Flow factor.

	Components Factors	1=Left	7=Right
		Shape Flow shrinking	Growing
	Shape	Directional Movement	Curvilinear Rectilinear
		Shaping	Contracted	Extended
		Time	Sustained	Abrupt
	Effort	Weight Space	Light Indirect	Strong Direct
		Flow	Free	Bound
	rect movement sometimes difficult. Some gestures of our
	dataset are limited (stop gesture) compared to free mo-
	tions like dance movements, hence the challenge of rec-
	ognizing emotions through restricted gestures.

Acknowledgements We would like to thank the staff of the University of Evry Val d'Essonne for participating in our datasets. This work was partially supported by the Strategic Research Initiatives project iCODE accredited by University Paris Saclay.

qualities and aiming at characterizing both quantitative and qualitative aspects of human movement. Our descriptor vector was tested for motion and emotion recognition. Four learning methods were used for learning and classification motions (RDF, OAO, OAA and MLP). In the first study, we compared our approach to state of the art methods where we first analyzed the robustness of our descriptor in the recognition of motions with three public datasets and our dataset dedicated to control motions. In a second study, we evaluated the pertinence of our descriptor in recognizing human emotions based on five expressive motions. Each motion was performed with four basic emotions selected from the arousal-valence model (Happy, Angry, Sad and Calm).

The same learning methods were applied for emotion recognition, and a study was proposed to analyze the relationship between our descriptor and the four selected emotions. Different scenarios were proposed to participants in order to be able to express their emotions in a natural way, and a 3D virtual avatar was introduced to reproduce human body motions. A statistical study based on viewer ratings was proposed to evaluate our recognition system. An external judgment was made for the evaluation of emotions and the rating of features across all motions in order to analyze the correlation between body features and the selected emotions and then concluded the importance of each feature to express each emotion.

As perspectives, we will expand our database with other emotions to provide more information about the relationship between body features and human emotions. We will apply our framework in a robotic application in order to control NAO robot with expressive motions from our dataset.


Insaf Ajili

Malik Mallem

Jean-Yves Didier

Relevant LMA Features for Human Motion Recognition

Keywords: Human motion recognition, Discriminative LMA features, Random Forest, Features Reduction

Motion recognition from videos is actually a very complex task due to the high variability of motions. This paper describes the challenges of human motion recognition, especially motion representation step with relevant features. Our descriptor vector is inspired from Laban Movement Analysis method. We propose discriminative features using the Random Forest algorithm in order to remove redundant features and make learning algorithms operate faster and more effectively. We validate our method on MSRC-12 and UTKinect datasets.

I. INTRODUCTION

Human motion recognition is a fundamental topic in the field of computer vision. One of the main challenges with motion recognition is that the same motion may be performed in different ways by different persons, and even by the same person. Although a significant amount of research has been focused on human motion representation, it remains still not enough. First motion features proposed in the computer vision literature were based on interest points [START_REF] Laptev | Space-time interest points[END_REF], [START_REF] Wang | Dense trajectories and motion boundary descriptors for action recognition[END_REF]. After, another type of features were studied based on depth informations provided from depth sensors. The depth cameras in general provide better quality 3D data than those estimated from monocular video sensors. This allows to focus on the analysis of the human motion through the identification of his joints [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF], [START_REF] Zhou | Discriminative key pose extraction using extended lc-ksvd for action recognition[END_REF], [START_REF] Wang | Action recognition based on joint trajectory maps using convolutional neural networks[END_REF], [START_REF] Lehrmann | Efficient nonlinear markov models for human motion[END_REF]. However, such proposed features did not take into account the semantic aspects of motion, for example to classify two actions with the same movement but performed with different intentions or rythms. In such case these features are not relevant to discriminate between these motions. Another key challenge in motion representation is to realize a good compromise between robustness performance and computational costs. The assumption that increasing the number of features can provide more informations about motion is not always valid in practice, because it can be time consuming and may lead to finding a less optimal solution. For feature reduction, several approaches were proposed and could be divided in two groups: methods based on statistical measures [START_REF] Surendiran1 | Feature selection using stepwise anova discriminant analysis for mammogram mass classification[END_REF], [START_REF] Lazar | A survey on filter techniques for feature selection in gene expression microarray analysis[END_REF], and methods based on learning algorithms [START_REF] Weston | Feature selection for svms[END_REF], [START_REF] Yamada | Highdimensional feature selection by feature-wise non-linear lasso[END_REF]. The first category consists in ranking features according to some statistical measures. It is fast and independ of any classifier, but it requires a threshold to select the top ranked features. Finally, in statistical approaches, some important features that are less informative on their own, but they are I.Ajili, M. Mallem and J. Didier are with IBISC, Univ Evry, Universit Paris-Saclay, 91025, Evry, France.

informative when combined with others can be discarded. The second category evaluates the importance of a random subset of features by training a model on it. A learning method is used to evaluate the importance of each combination of features on the classification performance. Despite the effectiveness of these methods, they have the constraint to be computationally more expensive compared to the statistics methods due to the repeated learning and cross validation steps.

In this paper, which extends our preliminary work presented in [START_REF] Ajili | Gesture recognition for humanoid robot teleoperation[END_REF], [START_REF] Ajili | Robust human action recognition system using laban movement analysis[END_REF], we address the problem of analyzing human motion from skeleton sequences captured by depth cameras. Particularly, our work focuses on representing human motions by keeping most salient and complementary features based on Random Forest algorithm. Our descriptor vector is inspired from Laban Movement Analysis method (LMA) to describe quantitative and qualitative representations of motions. The rest of the paper is structured as follows. Section 2 describes our proposed approach with motion recognition steps. Section 3 presents the experimental results on MSRC-12 and UTKinect datasets. Finally, conclusions and future work are stated in Section 4.

II. PROPOSED APPROACH

A. Data Acquisition

We use kinect sensor for data acquisition to extract 3D skeleton joints in real time. The first step is the normalization of all skeletons which consists in aligning all skeletons in the center of the kinect coordinate sytem with the base B at initial frame. Given a motion sequence S = {J j,t }, j ∈ 1, . . . , N , t ∈ 1, . . . , T , J j,t corresponds to the coordinates of the joint j captured at frame t. We define a local coordinate system to the skeleton anchored to the hip center joint (J c ), represented by the base B , equipped with three unit vectors, the left hip joint vector n lh , the spine vector n s and their cross product n c = n lh ∧ n s . For each sequence, we first apply translation to move the skeleton to the center of kinect, and after a rotation to align both coordinate systems (Fig. 1). The transformed joint yields to:

[J j,t ] B = R -1 B←B ([J j,t ] B -[J c,1 ] B ) (1)
R B←B = [ n lh n lh n s n s n c n c ] (2)
n lh = [J lhi,1 ] B -[J c,1 ] B (3) n s = [J s,1 ] B -[J c,1 ] B (4)
Once we applied transformations to all sequences, our system is independent of the initial position and orientation of the subject in the scene. Then, we pass to the next step which consists in converting the skeleton joints data to a descriptor vector based on LMA method.

B. Motions Representation

LMA approach employs a multilayered description of movement, focusing on four components: Body, Space, Shape, and Effort. Body component has the responsibility of highlighting the body part which is moving, making the connection between the moving parts and taking in consideration the issues of locomotion and kinematics. For this category, we describe the organization and connection between the different joints (Figure 2). We consider two parts, the upper and lower part. For the first one, the extension of the different joints is described by computing the following angles in left and right parts respectively: between hands and shoulders (θ l 1 , θ r 1 ), between elbows and hips (θ l 2 , θ r 2 ), between elbows and shoulders in the symmetrical part (θ l 3 , θ r 3 ). We also calculate the distances between the two hands (d Hs ) as well as the distances between the shoulder center and both hands (d shc,lh , d shc,rh ). It allows us to have a more idea on the way of the two hands. For the lower part of the body, the extension of the knees has been described with the angles between the feet and the hips (θ l 3 , θ r 3 ). These two characteristics allow to characterize specific actions like crouch or hide gestures. We also characterize the opening of the legs with the angle computed between the two knees θ Ls . Mean, standard deviation, and range of the body features are computed to quantify the Body component. We compute the length of trajectories (L) made by upper and lower body extremities (head, hands, and feet) to quantify the Space component.

L = T -1 t=1 J j,t+1 -J j,t (5)
In Shape component, we describe the way the body changes shape during movement with three qualities. In the first Shape flow factor, we charachterize the change shape in a self-toself relationship by computing the volume of the smallest convex envelope of the human body based on Quickhull algorithm [START_REF] Barber | The quickhull algorithm for convex hulls[END_REF], as shown in Figure 2. The second factor is the Directional movement, we define the pathway of the movement of upper body extremities (hands and head) through space by computing their curvatures (C).

C = T -1 t=2 arccos( ------→ J j,t-1 J j,t ------→ J j,t-1 J j,t • ------→ J j,t J j,t+1 ------→ J j,t J j,t+1 ) (6)
Finally, we quantify the Carving factor of the Shape component which describes the qualitative changes in the shape relating to spine joint pose at initial frame J s,1 , according to three planes: Horizontal (D H ), Frontal (D F ), and Sagittal (D S ), relating them to bipolar descriptors: spreading/enclosing, rising/sinking, and retreating/advancing, respectively (Figure 2).

D H = 1 T T t=1 ( N j=1 sqrt((J xj,t -J xs,1 ) 2 )) (7)
D F = 1 T T t=1 ( N j=1 sqrt((J yj,t -J ys,1 ) 2 )) (8)
D S = 1 T T t=1 ( N j=1 sqrt((J zj,t -J zs,1 ) 2 )) (9)
Effort component describes how the body concentrates its effort while performing a motion and characterizes expressive behaviors based on four factors: Time: Sudden/Sustained, Weight: Light/Strong, Flow: Bound/Free, and Space: Direct/Indirect (Figure 3). In Effort component we focus on the upper body part (head, hands, and spine), since it was the most expressive part during human motion. Joints velocities and accelerations are computed for quantifying Time and Weight factors, respectively. Three measures of variability (Mean, standard deviation, and range) are used for both features. We quantify the Flow factor by computing the yaw and pitch range of joints motion. For Free motion we will obtain a higher range compared to Bound motion. To describe the direction of the movement in space for Space factor, we compute the Straightness index (S) of joints motion as the ratio of the distance between the first and last frame (D) to the sum of the displacements between two successives frames (L).

S = D L (10)

C. Motions Recognition

For motions training and classification, we apply the Random Forest approach (RF) [START_REF] Breiman | Random forests[END_REF]. This method consists of an ensemble of decision trees, each tree is grown using by a different bootstrap sample from the training data. Let the feature vector be v = {f i }, i = 1, . . . , d, where d is the number of features for each sample. At each node, best split is chosen from a random sample of p features from d. Consider a node k comprising S k samples, splitted into left and right child nodes with subsamples of S kl and S kr , respectively, the tree is then grown by selecting the splitting condition that maximizes the purity of the resulting tree. Gini index I(S k ) is used to select the feature at each internal node k. The amount of homogeneity gain achieved by the splitting node k in feature f can be evaluated in the following equation:

G(f, S k ) = I(S k ) - i∈l,r ( |S ki | I(S ki ) |S k | ) (11)
Where

I(S k ) = 1 - l j=1 ( S j k S k ) 2
, l is the number of classes in node k, S j k denotes the number of learning samples which belong to class j at node k. Therefore, after several selections for f , the one producing the lowest value of Gini index is picked as the split criterion for the node. In the testing step, each test sample is simultaneously pushed through all trees, starting from the root, and assigning the data to the right or left child recursively until a leaf node is reached. Finally, the forest chooses the classification having the majority of votes from each of the decision trees made.

D. Features Reduction

Feature reduction step consists in keeping the smallest subset of most relevant features for motions representation to achieve a good compromise between accuracy and runtime in the classification process. The measure of relevant feature is returned by RF method. During training phase, each tree is grown using a different bootstrap sample from the original training data, leaving 1/3 as OOB (Out Of Bag) to estimate the prediction error of OOB. The importance of the feature f i is measured as the difference between OOB prediction accuracy of each tree before and after permuting f i .

I t (f i ) = j∈O t I(y j = y jt ) |O t | - j∈O t I(y j = y i jt ) |O t | (12)
I(f i ) = T t=1 I t (f i ) T ( 13
)
O t corresponds to OOB samples for a tree t, y j is true class label of the j th training sample. y jt and y i jt are the predicted classes for the j th sample by tree t before and after permuting the feature f i , respectively. Finally, a high decrease in accuracy is an indication of the feature importance. We start with the whole set of features, we compute and record the OOB error rate. After we sort the features in descending order of importance, and we remove the feature of small importance f min . Moreover, we apply the Tukey's test (α = 0.05) to simultaneously remove features that do not give a significant difference of the OOB error rate result.

I(f i ) 8 f min = argmin i {I(f i )} 9
Apply Tukey's test and select set of features {f t } that does not lead to a significant changement of

E k 10 R = f min {f t } 11 v k+1 = v k \R 12 k = k + 1 13 end 14 v * = argmin k {E k (v k ))}
v * is the optimal feature subset with minimal OOB error.

III. EXPERIMENTAL RESULTS

To evaluate the performance of our method, we use two public action datasets, MSRC-12 [START_REF] Fothergill | Instructing people for training gestural interactive systems[END_REF] and UTKinect [START_REF] Xia | View invariant human action recognition using histograms of 3d joints[END_REF], we report two measures: the mean of fscores and the OOB error rate. For the first measure, we adopt the 5-fold cross validation to optimize the RF parameters, and compute the averaged results. We employ the commonly used F-score as the performance measure.

1) Evaluation on MSRC-12 Dataset: MSRC-12: is a dataset composed of 594 sequences, containing the performances of 12 gestures by 30 people. In total, there are 6244 gesture instances. The gesture classes are divided into two groups: metaphoric gestures, and iconic gestures. The motion files contain 3D coordinates of 20 joints captured at a sample rate of 30Hz.

We converted the raw data into a descriptor vectors based on our LMA qualities. Our descriptor vector composed of 85 features was fed into a learning algorithm, RF. Most important parameters of RF, the number of trees (n trees ) and the number of features to consider when splitting a node (max f eatures ) were adjusted. We varied n trees starting from 10 until 200 trees, and we tested three values of max f eatures (85, log 2 (85), and √ 85). Best recognition rate of 94.89% was achieved when setting n trees = 100 and max f eatures = log 2 (85), and almost the same recognition result of 94.88% was obtained for n trees = 100 and max f eatures = √ 85. We also confirmed the RF parameters values by computing OOB error rate while varying n trees and max f eatures . As we can see in Figure 4, the two curves of max f eatures = log 2 (85), and √ 85 are very close with a very little superiority result of max f eatures = log 2 (85). We identified the minimum value of n trees where OOB error stabilize (around 0.008), we found n trees = 100, which confirms the result obtained with the recognition rate measure. Table I illustrates the recognition results of our OOB error rates in terms of RF parameters (ntrees and max f eatures ). method compared to the state of the art methods on MSRC-12 dataset. Our method outperforms the state of the art methods, and is very close to the result obtained in [START_REF] Wang | Action recognition based on joint trajectory maps using convolutional neural networks[END_REF], which confirms the robustness of our descriptor in characterizing both iconic and metaphoric gestures with an accuracy rates of 99% and 93%, respectively. After evaluating our descriptor vector on gestures recognition in MRC12-dataset, we applied our feature

Methods

Recognition rates (%) Hussein et al. [START_REF] Hussein | Human action recognition using a temporal hierarchy of covariance descriptors on 3d joint locations[END_REF] 91.70 Zhou et al. [START_REF] Zhou | Discriminative key pose extraction using extended lc-ksvd for action recognition[END_REF] 90.22 Wang et al. [START_REF] Wang | Action recognition based on joint trajectory maps using convolutional neural networks[END_REF] 94.86 Lehrmann et al. [START_REF] Lehrmann | Efficient nonlinear markov models for human motion[END_REF] 90.90 Our method 94.89 reduction algorithm (Algorithm 1) in order to keep only most discriminant features according to this dataset. To obtain more stable results and better estimations for the expected OOB error rate, we repeated this procedure 30 times and average the results. In Table II, we make a comparison between results obtained before and after feature reduction process, in terms of M ean F -score, OOB error rate (errOOB), and number of features (N ). We notice that the number of relevant features is decreased about 20% achieving the same F-score results and decreasing the OOB error rate. We obtained a low OOB error rate of 0.004 with a number of relevant features N = 67 < 85 (see Fig. 5).

2) Evaluation on UTKinect Dataset: We also evaluated our descriptor with UTkinect dataset which is composed of 10 subjects performing 10 different activities in varied views namely walk, sit down, stand up, pick up, carry, throw, push, pull, wave hands, and clap hands. A total number of 199 sequences are available. Each action is repeated twice by the actor. Sequences are captured using one Kinect in indoor settings and their length ranges from 5 to 120 frames. This is a challenging dataset due to variations in the view point and high intra-class variations where each actor performs

IV. CONCLUSION

In this paper, we presented an efficient method for extracting most relevant motion descriptors for human motion recognition. Our descriptor was inspired from LMA technique to combine both quantitative and qualitative characteristics of motion. Furthermore, an effective feature reduction algorithm was applied to keep only the most informatives features which had a great impact on the computational latency while maintaining or even improving the reported results. Based on these results, we plan to recognize expressive motions and study the importance of each LMA features to characterize human emotions.

Fig. 1 .

 1 Fig. 1. Kinect and local coordinate systems.

Fig. 2 .

 2 Fig. 2. Some LMA features.

Fig. 3 .

 3 Fig. 3. Effort factors.

Algorithm 1 : 7 Sort

 17 Feature reduction process. Input : v 0 = {f i }, i = 1, . . . , p v 0 is the whole feature set. Output: v * = {f j }, j = 1, . . . , p * v * subset of most relevant features. 1 k = 0 2 while |v k | ≥ 1 do 3 Compute and record OOB error rate: E k (v k ) 4 for i = 1 to p do 5 Compute I(f i ) Importance of each feature in v k . 6 end {f i } in descending order according to values of

Fig. 4 .

 4 Fig. 4.OOB error rates in terms of RF parameters (ntrees and max f eatures ).

Fig. 5 .

 5 Fig.5. Optimal feature subset with minimal OOB error using Tukey's test (α = 0.05).

TABLE I RECOGNITION

 I RATES OF OUR METHOD COMPARED TO THE STATE OF THE ART METHODS ON MSRC-12 DATASET.

TABLE III COMPARISON

 III OF (MEAN-FSCORES, OOB ERROR VALUES, NUMBER OF FEATURES) BETWEEN BEFORE AND AFTER FEATURES REDUCTION STEP INUTKINECT DATASET. We applied both methodes, feature extraction with LMA approach and after RDF method for actions recognition. We measured the recognition rate using same validation method as MSRC-12 dataset, the 5-fold cross validation technique. We obtained as result the mean of fscores 0.96(+/ -0.01)/. With the same RDF parameters setting in MSRC-12 dataset, we applied features reduction step. TableIIIsummarizes recognition results (M ean F -score, OOB error rate (errOOB), and number of features (N )) before and after applying features reduction step. With 36 features we obtained same mean f-score and a lower OOB error value. So we can say that our method managed to reduce features while keeping most relevant features and same recognition results.

		M ean F -score	errOOB	N
	Before reduction	0.96 (+/-0.02)	0.0075	85
	After reduction	0.96 (+/-0.01)	0.006	36
	actions in different views.