Insaf Ajili

Zahra Ramezanpanah
email: zahra.ramezanpanah@gmail.com

Malik Mallem
email: abdelmalik.mallem@univ-evry.fr

Jean-Yves Didier
email: jeanyves.didier@univ-evry.fr

Expressive motions recognition and analysis with learning and statistical methods

Keywords: Expressive motions recognition, Laban Movement Analysis, Random Decision Forest, human perception, Features importance

This paper proposes to recognize and analyze expressive gestures using a descriptive motion language, the Laban Movement Analysis (LMA) method. We extract body features based on LMA factors which describe both quantitative and qualitative aspects of human movement. In the direction of our study, a dataset of 5 gestures performed with 4 emotions is created using the motion capture Xsens. We used two different approaches for emotions analysis and recognition. The first one is based on a machine learning method, the Random Decision Forest. The second approach is based on the human's perception. We derived the most important features for each expressed emotion using the same methods, the RDF, and the human's ratings. A comparison between RDF and observers classifiers was made in the discussion section.

Introduction

Body language often has a great impact in transmitting messages to the listener by providing social meaningfulness to daily human-human interactions. Among the various type of body languages, facial expressions, body posture, and gestures are very important to convey feelings and contribute to the overall effectiveness of communication. The expressive quality of movement presents the way in which a gesture is performed. It is an important factor for transmitting high-level pieces of information during human being communication (e.g., emotions, moods). Generally, the state of the art research has focused on the decoding of emotions from facial expressions and vocal cues. A smile or raised inner eyebrows can convey happiness and amusement. Eyebrow lower and chin raise can characterize sad emotion. More than 95% of the literature in the field of identification of human emotions has used the facial expressions as a modality for emotion perception [START_REF] De | Why bodies twelve reasons for including bodily expressions in affective neuroscience[END_REF]. Besides that, few researchers have studied how body motions can provide reliable characteristics of emotions across the lifespan. In this work, we aim to develop an automated expressive motion recognition system which characterizes human behaviors (motions and emotions) based on his body movement without taking into consideration the facial expressions. In daily activities, happiness is always illustrated by jumping, sadness by bending, anger by punching, etc [START_REF] Wallbott | Bodily expression of emotion[END_REF]. But on the other hand, the same motions performed with different intentions can also give us pieces of information about human psychological states. According to psychology research, a prerequisite to analyzing the human behavior is to perceive which motion characteristics lead to express a specific type of emotions. For example, sadness was always characterized by smooth and slow movements. However, Happiness was associated with faster, and more expanded movements [START_REF] Montepare | The use of body movements and gestures as cues to emotions in younger and older adults[END_REF]. However, human physiology and the limitation in the physical behavior of human movement cause the emotion recognition to be a difficult task.

The main purpose of our work is to extract a robust motion descriptor in order to characterize and recognize human emotions performed with different motions. On the other hand, we interpret the relationship between motions parameters and emotions. Many motion features were proposed in the literature with different combinations. However, there is always a lack of significant formalization in the direction of evaluating their perceptual effect. To address the above issues, we used the Laban Movement Analysis (LMA) technique, which is a language for describing and understanding human movement. This method captures both quantitative and qualitative aspects of the movement. Based on its four factors (Body, Space, Shape, and Effort), the LMA helps to make a link between expressive motion qualities and emotions which avoids the arbitrary features selection. This paper builds on our preliminary works on the Microsoft Kinect-based gesture recognition based on the LMA method [START_REF] Ajili | Robust human action recognition system using laban movement analysis[END_REF][START_REF] Ajili | Gesture recognition for humanoid robot teleoperation[END_REF], where gestures were performed without considering human states or emotions. We only use kinematic features to characterize human motions for a robotic application. Now, we add Effort LMA component which describes the qualitative aspects of the movement by identifying the type of energy used in the movement. Then, we evaluate our descriptor on both motion and emotion recognition through our dataset. Our dataset is composed of 5 gestures (Dancing, Moving, Waving, Pointing, and Stopping) performed with 3 emotional expressions (Happy, Sad, Angry) and Neutral state. Each gesture is repeated 5 times by 11 different subjects. The dataset is acquired from the MVN Awinda system [6]. It's about a 3D motion capture data composed of 17 motions trackers attached to the different body segments. The main contributions of our work are:

-An LMA-devoted motion representation for classifying motion and emotion with the RDF method.

-A publicly accessible motion dataset of 5 expressive motions performed with 4 different emotions selected from the Arousal-Valence model (Happy, Angry, Sad, and Neutral). -Studying the importance of our motion descriptor to characterize each emotion via a machine learning method. -Evaluating our recognition system which is based on an automatic learning method against a human perception approach.

The rest of the paper is organized as follows: Section 2 presents the related works in human expressive motions field. Section 3 provides the technical details of the proposed approach. Section 4 presents two main experiments based on machine learning and statistical methods, respectively. The paper ends with conclusion and perspectives for future work.

Related works

Emotions are the intrinsic moods which are not independent of physiological changes. In line with recent developments in computer vision and multimedia technologies, there has been much interest in emotion understanding. In this regard, the facial expression is one of the popular methods which has been used in various studies [START_REF] Kanade | Comprehensive database for facial expression analysis[END_REF]. However, the impact of the age was the main drawback of this method. Several studies have been done to reduce the errors that occurred due to the age factor such as [START_REF] Ho Kwon | Age classification from facial images[END_REF][START_REF] Ramanathan | Face verification across age progression[END_REF][START_REF] Lanitis | Comparing different classifiers for automatic age estimation[END_REF]. Another modality was been considered in the emotions analysis field which is the speech. Kamaruddin et al. [START_REF] Kamaruddin | Driver behavior analysis through speech emotion understanding[END_REF] have detected emotions through the vocal channel. They studied the correlation between driver behavior states and speech emotions for analyzing driver behavior. In their work, the extraction of the Mel Frequency Cepstral Coefficient accompanied with Multi-Layer Perception and fuzzy neural network classifiers are used. However, this modality is not always available, for example, when affective expressions are estimated from a long distance, or in a noisy environment. So in this regard, few researchers have tried to enhance the accuracy of emotion recognition systems by using affective multiple data [START_REF] Pantic | Toward an affect-sensitive multimodal humancomputer interaction[END_REF]. Several studies incorporate facial expression and audio information either as a feature extraction [START_REF] Chen | Emotional expressions in audiovisual human computer interaction[END_REF] or as a feature selection [START_REF] Silva | Bimodal emotion recognition[END_REF]. In the recent years, more attention has been paid to use the body movement for transmitting emotions. Body movements were represented by a set of activities usually used for 3D human action recognition [START_REF] Liu | Action2activity: Recognizing complex activities from sensor data[END_REF][START_REF] Liu | From action to activity: Sensor-based activity recognition[END_REF]. However, another field was appearing that used the body movement to convey human emotions. Many authors have confirmed the advantage of this modality in the emotional field, such as, Argyle et al. [START_REF] Argyle | Bodily communication[END_REF] who have shown that the body movement is as strong as facial expressions in conveying emotions. Gelder et al. [START_REF] De | Why bodies twelve reasons for including bodily expressions in affective neuroscience[END_REF] have suggested that body expressions allow for successful emotion conveying across long distances. Kleinsmith et al. [START_REF] Kleinsmith | Affective Body Expression Perception and Recognition: A Survey[END_REF] have compared between bodily and facial expressions, and demonstrated the importance of bodily expressions for non-verbal communication in Human-Computer and Human-Robot interactions. To understand the body ways of expression, Laban [START_REF] Von Laban | The mastery of movement[END_REF] formalized movement qualities relying on a body movement analysis system, called LMA. It is particularly useful in describing qualitative aspects of the movement crucial to expression and communication. The LMA categorizes movement descriptions into four main components: Body, Effort, Shape, and Space.

Several researchers have applied this system for emotion analysis through body movement. The LMA was used for several purposes, such as generating expressive motions which consists in regulating LMA factors in some gestures to generate an expressive motion. Chi et al. [START_REF] Chi | The emote model for effort and shape[END_REF] developed an EMOTE system and applied Effort and Shape components to generate expressive gestures for virtual agents. Same as Zhao et al. [START_REF] Zhao | Acquiring and validating motion qualities from live limb gestures[END_REF] used the same model to produce natural gestures for embodied communicative agents. They extracted motion qualities inspired by Effort and Shape components. Amaya et al. [START_REF] Amaya | Emotion from motion[END_REF] introduced a model to generate an emotional movement from a neutral movement. They extracted emotional transforms using a signal processing technique relying on two components (the speed and the spatial amplitude transform). These parameters were applied to existing motions and generate two emotions (anger and sadness). Masuda et al. [START_REF] Masuda | Motion rendering system for emotion expression of human form robots based on laban movement analysis[END_REF] proposed a motion rendering system that adds a target emotion to arbitrary movements. They extracted movement features based on the LMA method. They studied the correlation between Laban's features (Space, Time, Weight, Inclination, Height, and Area) and four emotions (Pleasure, Anger, Sadness, and Relaxation). Duripinar et al. [START_REF] Durupinar | Perform: Perceptual approach for adding ocean personality to human motion using laban movement analysis[END_REF], improved the EMOTE model proposed by Chi et al. [START_REF] Chi | The emote model for effort and shape[END_REF]. They grounded the association between the personality model OCEAN and the body motion and applied this association to animate expressive characters with personality. Shafir et al. [START_REF] Shafir | Emotion regulation through movement: Unique sets of movement characteristics are associated with and enhance basic emotions[END_REF] asked people familiar with LMA to identify a set of motor characteristics that predict the enhancement of specific emotions like anger, happiness, and sadness. The purpose behind it is to be able to regulate their emotions through their motor behavior. Also, the LMA method was used for motion retrieval which consists in indexing and searching complex motions in large motion databases. Kapadia et al. [START_REF] Kapadia | Efficient motion retrieval in large motion databases[END_REF] proposed a method for indexing and retrieving motions from a large motion database. To search for complexes motions, they extracted motion cues based on LMA components composed of structural, geometric, and dynamic features. Aristidou et al. [START_REF] Aristidou | LMA-Based Motion Retrieval for Folk Dance Cultural Heritage[END_REF] introduced an approach for content-based retrieval of dance motion capture data inspired from LMA qualities. Also, the LMA model was used for style transfer which consists in carrying over the style from one avatar motion onto another. Airstidou et al. [START_REF] Aristidou | Style-based motion analysis for dance composition[END_REF] used an iterative time warping to map from the input to the output sequence. They transformed a motion sequence into a new motion style while retaining its original content. In another context, for emotion recognition the LMA model was applied by several authors, such as Zachartos et al. [START_REF] Zacharatos | Emotion recognition for exergames using laban movement analysis[END_REF] who extracted motion features based on only the Effort LMA component for emotion recognition in a game scenario. They used the Multi-Layer Perceptron classification algorithm to recognize four emotions (Meditation, Concentrate, Excitement, and Frustration). Recently, the LMA technique was used for characterizing emotions through a specific action, such as waving [START_REF] Lourens | Communicating emotions and mental states to robots in a real time parallel framework using laban movement analysis[END_REF][START_REF] Barakova | Expressing and interpreting emotional movements in social games with robots[END_REF]. The authors asked participants to perform the gesture "waving" with four emotions, angry, happy, sad, and polite. They extracted three Effort qualities, time, weight, and flow and studied the relationship between these qualities and the four emotions when performing the waving gesture. For "walking" task, Cimen et al. [START_REF] Cimen | Classification of human motion based on affective state descriptors[END_REF] quantified LMA components to extract posture, dynamic, and frequency-based descriptors for human motion. They used the SVM method for emotions classification when walking in different emotional states (Angry, Happy, Sad, and Relaxed). These studies mostly focused on emotions expressed in a single movement.

Most of the researchers who have studied expressive gestures have focused on a specific action such as waving, walking, Knocking, etc. and focused on the LMA Effort component as it is the same movement and only the intention of the movement differs. For our case, we have proposed a generic motion descriptor that can describe the quantitative and qualitative aspects of the movement so allows to differentiate different movements and also a same movement performed with different emotions. Moreover, most researchers who have adopted the LMA method worked in the field of music and dance and were forced to ask professional people for the construction of their dataset and sometimes make some training courses [START_REF] Aristidou | Style-based motion analysis for dance composition[END_REF]. For our case, we chose to create a publicly accessible motion database easy to use by everyone, so we did not need to professional people to construct our dataset or to test an unknown expressive motion. Our work advances the previous researches in human expressive motions recognition. We proposed a descriptor motion to characterize both human motions and emotions. We created a dataset composed of limited gestures (not free) performed with different emotions selected from the Circumplex model of affect (Happy, Sad, Angry, and Neutral). We studied the importance of our motion features in characterizing each emotion. We proposed a human approach based on human perception and ratings to evaluate both emotions and motion descriptor. The obtained results from human classifier were used to evaluate the accuracy and the adequation of our recognition system based on automatic learning method.

Methods

The whole process of our method consists of four main parts. The first part is data collection using the Xsens capture motion for our emotional dataset. The second part is the feature extraction using the LMA method to convert raw data into a robust motion descriptor. The third part is the classification of expressive motions. Finally, we study the importance of each motion feature to express each emotion. These last two steps are performed using two different methods, the first one is based on a machine learning algorithm (RDF), and the second one is based on a statistical study based on viewers' perceptions.

Data collection

Participants 11 subjects (five males and six females) from the University of Evry Val d'Essonnes ranged in age from 27 to 36 years old (M=29.85 years, SD=2.47) participated in this study. They are master and phd students in the information technology field. They were paid for taking part in our research. They expressly allowed us to publish their movements on the internet and to use their motion capture data for research purpose. In order to keep away from having any overstatement feeling emotions, we did not choose professional actors, whom, thanks to theatrical training, know the manner of using their body to demonstrate effect through actions. Our dataset has then the ability to be enriched by everyone without the need for professional actors. Expressive gestures Our emotional dataset is composed of five expressive motions (Dancing, Moving, Waving, Pointing, and Stopping), illustrated in Figure 1. We choose such motions for our future application dedicated to the teleoperation of NAO robot with expressive control motions. For each category of movement, we captured neutral motions, as well as motions performed with emotions: Happy, Angry, and Sad. Unlike other affects such as surprise or disgust which could not be easily identified especially by learning method, the common point amongst these three chosen affects is their ability to express over a relatively long time, a mood or an emotional state which could be recognized easier by the population and also by the learning method. To help the participants to express well their emotions, we play different music corresponding to each situation, and we propose scenarios presenting emotional situations. Each participant was asked to read the proposed scenario, take the time to feel the situation, and perform the action 5 times. We were careful to give the participants only verbal instructions for expressive motions, rather than performing the movements ourselves. Examples of scenarios proposed to participants when performing an expressive gesture (Figure2): Happy emotion:

-You have passed all your courses in the University and the same day that you get your results you have a phone call from a company that interviewed you, and you understood you got also your favorite job.

-You are in a good relationship and the birthday of your partner is close and you are planning to have a party but you do not have enough money, suddenly you receive an email which informs you of winning in a lottery.

Sad emotion:

-For a popular festival you booked your ticket and hotel from 6 months ago but because of a friend you miss your flight and you can not arrive at your favorite festival. -You are in a new country that you have never been there and you use your phone to find your address but suddenly it sticks and stops working, you do not know their language and they can not speak English or your mother tongue at all.

Angry emotion:

-You have a job interview which you have been waiting for a long time. Today is 24/03/2018 and you forget that tomorrow, the date of your interview, clocks are turned forward an hour because of Daylight Saving Time. So, you wake up late, nearly an hour, you dress up quickly and get in your car but it does not work. You have no other choice than public transport. Finally, in the subway station, you find that the drivers are on a strike. -You have your own restaurant but you find out that your business is not making enough money. You decide to discover the problem, so you introduce yourself as a new employee to your employees. On the first week, you notice that the restaurant is opened lately and, as a result, there is always a long delay for customer's orders. On the other hand, you see the unpleasant behavior of waiters with customers which is due to their dissatisfactions with their incomes.

In the end, you are very angry about the inefficiencies of the managers who are hired by you.

During the recording sessions, the order of scenarios, emotions, and motions was randomized from one participant to another. For each participant, we collected the following files: (1) a .avi video recorded by a camera containing the video of the participant's motion, (2) a .csv file including all motion information produced by the Xsens, and (3) an avatar animation video reproducing the motion of the character. In total, we obtained 1100 motion capture sequences (11 participants × 5 motions × 4 emotions × 5 times).

Motion capture For each performed gesture, 3D motion sequence of the person's full body was recorded by the Xsens sensor, as shown in Figure 2. The Xsens estimates the 3D position and orientation of each body joint using 17 inertial trackers.

Feature extraction

In the direction of describing human behavior, Laban Movement Analysis (LMA) method is used. The extracted body features are made up of four LMA components (Body, Space, Shape, and Effort) and each of them is described through a specific explanation that quantifies and qualifies various attitudes of dynamic motions (Figure 3). Body component has the responsibility of highlighting the body part which is moving, making the connection between the moving parts and taking into consideration the issues of locomotion and kinematics. For this category, we describe the organization and connection between the different joints (see Figure 4). We consider two parts, the upper and lower part. For the first one, the extension of the different joints is described by computing the following angles in the left and right parts respectively: between hands and shoulders (θ l 1 , θ r 1 ), between elbows and hips (θ l 2 , θ r 2 ), between elbows and shoulders in the symmetrical part (θ l 3 , θ r 3 ). We also calculate the distances between two hands (d Hs ) as well as the distances between the shoulder center and both hands (d shc,lh , d shc,rh ). For the lower part of the body, the extension of the knees has been described with the an- gles between feet and hips (θ l 3 , θ r 3 ). These two characteristics allow to characterize specific actions like crouch or hide gestures. We also characterize the opening of the legs with the angle computed between two knees θ Ls . Mean, standard deviation, and range of body features are computed to quantify the Body component. Space component describes the trajectory performed by the participant's body parts during an action. We compute the length (L) of the trajectory made by the upper extremities including head, left and right hands joints (J t is the 3D joint position captured at frame t, and T is the number of frames in a motion sequence.

L = T -1 t=1 J t+1 -J t (1)
Shape component can be defined as a collection of the properties that appear from body and space components. This category expresses the changing form of the body that is either motivated by self or the environment. The three factors of Shape are Shape Flow, which represents a relationship of the body to itself; Directional movement, which defines a relationship of the body toward some part of the surrounding environment; and Shaping, which describes the qualitative changes in the shape according to three-dimensional planes. For the Shape Flow, we compute the convex hull volume (V ) of the 3D skeleton based on Quickhull algorithm [START_REF] Barber | The quickhull algorithm for convex hulls[END_REF] in order to describe body extension. Mean, standard deviation, and range of the 3D convex hull volume feature are computed to quantify the Shape Flow factor.

For Directional factor, we focus on the upper extremities including, head, left and right hands. We describe their pathway by computing the gradual angular change φ occurring between two successive frames, defined as follow:

φ J t = arccos( ----→ J t-1 J t ----→ J t-1 J t • ----→ J t J t+1 ----→ J t J t+1 ) (2)
This equation describes the local curvature of joints trajectory. From this angle we derived the curvature feature (C) defined as:

C = T -1 t=2 φ J t (3)
This index tends to 0 in straight-Line trajectories cases and changes to a very high value in curved paths. For Shaping factor, we describe movements according to three planes, Horizontal (sideways movements), Sagittal (forward/backward movements), and Frontal (upward/downward movements). We describe body inflation according to the three planes by computing average distances of all skeleton joints [J t,i ] X,Y,Z (t = 1, . . . , T and i = 1, . . . , N ) with respect to the spine joint (J 1,s ) at initial frame.

D H = 1 T T t=1 ( N i=1 sqrt(([J t,i ] X -[J 1,s ] X ) 2 )) (4)
D F = 1 T T t=1 ( N i=1 sqrt(([J t,i ] Y -[J 1,s ] Y ) 2 )) (5)
D S = 1 T T t=1 ( N i=1 sqrt(([J t,i ] Z -[J 1,s ] Z ) 2 )) (6)
The intention and dynamic qualities of the movement, the feeling tone, the texture and the manner of consumed energy during movement are explained by the Effort component. This component is generally associated with the change in emotion or mood, hence, it is useful for motion expressivity. This factor is required for the description of emotions. We focus on the upper body part since this is the part that moves the most when expressing an emotion, especially the following four joints: head, spine, left and right hands. The Effort component consists of four factors: Space, Weight, Time, and Flow which are explained as follows: Space expresses the quality of active attention to the surroundings. It is divided into two categories, Direct (when the action is direct the attention is on a single point in space, focused and specific) and Indirect (giving attention to multiple directions in the space, multi-focused and flexible). We compute the Straightness index (S) of upper body joints trajectories (Sh for head, Ss for spine, Sl for left hand, and Sr for right hand) as the proportion between the Euclidean distance of the straight trajectory between the positions at the first and last frame (D), and the sum of distances between successive frames (L) defined in the Equation 1.

S = D L (7)
In the direct movement, we obtain a straightness index close to 1, and close to 0 in indirect case. Time is the interior attitude of the body toward the time, not the period or the length of the movement. Time category can be expressed as sudden (quick, urgent, unexpected, surprising) and sustained or stable (stretching in the time, continues, slowly). So to quantify this factor we compute the mean, the standard deviation, and the range of velocity (v) of upper body joints (head (vh), spine (vs), left and right hands (vl, vr)). Weight involves the strength or power with which a movement is performed. It can be strong or light. Strong movement requires force and acceleration while light movement is characterized by an invariant rhythm of motion, so lower acceleration values. We compute the mean, standard deviation, and the range of the acceleration (a) of the head (ah), spine (as) and hands (al, ar) to quantify the Weight factor. Flow involves the continuity of the movement. Its categories are bound (controlled movement) and free (unstoppable, liberated). We compute the yaw and pitch ranges of the head (range yawh, range pitchh), spine (range yaws, range pitchs), left hand (range yawl, range pitchl), and right hand (range yawr, range pitchr) motions. Bound movement can be described as a movement which can be stopped at any moment, whereas free or smooth movement presents a continuity in the motion. Once we have quantified all LMA factors, we obtain a descriptor vector composed of 85 features which will be the input of the classification method presented in the next section.

3.3 Expressive motion recognition and analysis using the learning method

Recognition

For expressive motions classification, we used the Random Decision Forest method (RDF) [START_REF] Breiman | Random forests[END_REF], since it has shown its effectiveness in characterizing human actions in our preliminary work [START_REF] Ajili | Robust human action recognition system using laban movement analysis[END_REF]. This is also explained by the fact that this method is part of the family 'Ensemble od methods' whose general principle is to build a collection of classifiers, then to aggregate the set of predictions, which increase classifier accuracy as explained by Dietterich [START_REF] Thomas | Ensemble methods in machine learning[END_REF]. Also, the RDF method uses the randomization in the induction process, with the two methods: bagging [START_REF] Breiman | Bagging predictors[END_REF] and random subspaces [START_REF] Tin | The random subspace method for constructing decision forests[END_REF], in order to build base classifiers different from each other, and thus introducing diversity among them. This makes the model more general and helps to combat the overfitting problem which may have appeared in other models such as SVM and its enhancements [START_REF] Liu | Fortune teller: Predicting your career path[END_REF][START_REF] Liu | Urban water quality prediction based on multi-task multi-view learning[END_REF]. Another great quality of the random forest algorithm is its ability to measure the relative importance of each feature on the prediction. Let the feature vector of training data be v = {f i }, i = 1, . . . , d, where d is the number of features for each sample. At each node, the best split is chosen using a random sample of p features from d. Consider a node k comprising S k samples, split into left and right child nodes with subsamples of S kl and S kr , respectively, the tree is then grown by selecting the splitting conditions that maximize the purity of the resulting tree. Gini index I(S k ) is used to select the feature at each internal node k. The amount of homogeneity gain achieved by the splitting node k in feature f can be evaluated in the following equation:

G(f, S k ) = I(S k ) - i∈l,r ( |S ki | I(S ki ) |S k | ) ( 8
)
Where

I(S k ) = 1 -l j=1 ( S j k S k ) 2
, l is the number of classes in node k, S j k denotes the number of learning samples which belong to class j at node k. Therefore, after several selections of f , the one producing the lowest value of Gini index is picked as the split criterion for the node. In the testing step, each test sample is simultaneously pushed through all trees, starting from the root, and assigning the data to the right or left child recursively until a leaf node is reached. Finally, the forest chooses the classification having the majority of votes of each decision trees.

Features importance

The features importance step consists in keeping the smallest subset of most relevant features for motion representation to achieve a good compromise between accuracy and runtime in the classification process. The measure of relevant features is returned by the RDF method. During the training phase, each tree is grown using a different bootstrap sample from the original training data, leaving 1/3 as OOB (Out Of Bag) to estimate the prediction error of OOB. The importance of the feature f i is measured as the difference between the OOB error rate of each tree before and after permuting f i . Our feature selection method is presented in the Algorithm 1.

Algorithm 1: Feature selection process.

Input : v 0 = {f i }, i = 1, . . . ,
v k+1 = v k \R 12 k = k + 1 13 end 14 v * = argmin l {E l (v l ))}
v * is the optimal feature subset with minimal OOB error.

Results

Recognition: After converting raw data into LMA features we obtained a descriptor vector composed of 85 features. The RDF method was used for motions and emotions classification.

For motions classification we consider all data, 1100 sequences (11 persons×5 gestures×4 emotions×5 repetitions). In this experiment, we evaluated the robustness of our descriptor to classify motions with different intentions. We used the 3 folds cross-validation method, so data is divided into 3 groups. We loop through 3 groups and each time we use one group as the testing set and the remaining groups as the training set. Finally, the performance measure is obtained by the average of the values computed in the loop. As performance measures, we used the accuracy, precision, recall, and the F-score (F ), where parameters are: the number of trees (ntrees) and the number of features to consider when splitting a node (maxf eatures). We varied ntrees starting from 10 to 300 trees, and we tested three values of maxf eatures (85, log2(85), and √ 85). The best F-score of 0.83(+/ -0.07) was achieved when setting ntrees = 150 and maxf eatures = log2(85). We also confirmed the RDF parameters values by computing the OOB error rate while varying ntrees and maxf eatures. As we can see in Figure 5, the two curves of maxf eatures = log2(85), and maxf eatures = √ 85 are very close with a very little superiority result of maxf eatures = log2(85). We identified the minimum value of ntrees where the OOB error stabilize (around 0, 06), we found ntrees = 150, which confirms the result obtained with the recognition rate measure.

For emotions classification, we divided the dataset into five groups per gesture in order to study emotions recognition for each gesture. Each group is composed of 220 sequences (1 gesture×4 emotions×11 persons×5 repetitions). In each group, we consider 7 subjectif for the training data and 4 subjects for the testing data. Table 1 shows the recognition results obtained in emotions recognition experiment for each gesture (waving, moving, dancing, stopping, and pointing) and for all gestures.

Confusion matrices of the different emotions are presented in Figure 6. The results show that our descriptor vector can distinguish between different motions and even between same motion performed with different emotions. Some expressive motions have been successfully recognized (100%) such as: Waving and Moving gestures with Happy and Angry, and Neutral states, Stopping and Pointing ges- tures with Angry emotion, and Pointing gesture with a Neutral state. Otherwise, some confusions were obtained in the same motion especially between Happy and Angry emotions, like Dancing gesture, and between Sad and Neutral states in the Stopping gesture. According to these results, we can approve the effectiveness of our motion descriptor in recognizing human motions and emotions through body movement. We also computed the confusion matrix of all expressive motions in Figure 7. So we have 20 classes (5 gestures×4 emotions). As we can see, the highest values are marked in the diagonal of the matrix, which confirms the performance of the recognition results. We have no confusion between different gestures, but we can find some confusions in the gesture itself when it was performed with different emotions. Such confusions are obtained in Moving, Waving, and Stopping gestures when feeling Sad and Neutral. Generally, we can say that our descriptor motion has successfully characterized both quantitative and qualitative aspects of motions.

Features importance: In the second experiment, we aimed at studying the importance of each motion features to contribute to each emotion. We used the RDF method to select the most relevant features for each emotion based on OOB error rate measures. Here, we are more interested in the qualitative aspect of the movement. Since we consider at each time the same gesture acting with several emotions, so we have the same kinematic features importance. The only difference appears at the level of motion dynamics. For this, we keep only Effort-Shape factors, which describe the different expressive qualities perceived in the motions. Also, to study motion features importance in expressing emotions, we have to consider at each time the neutral emotion as a reference to other emotions (Happy, Angry, and Sad). So, with the RDF method we classified each time one emotion with the Neutral state. Then, we evaluated the importance of each feature in characterizing each emotion by applying our algorithm presented in Algorithm 1 which consists in keeping only the most important features. We obtained the fol- lowing results illustrated in Table 2. From this table, we can draw the following conclusions:

The Happy emotion is characterized more by the strength and the urgency of the movement. So, as important features for this emotion, we have the velocity and the acceleration of body articulations in almost all gestures. For example, in Moving gesture with raising both hands, the proposed features importance algorithm gives as results, the velocity and the acceleration of the right and left hands are the most important features in characterizing the Happy emotion (Figure 8).

The Angry emotion is more characterized by the strength of the movement in all gestures and the urgency in two gestures (Waving and Moving). Figure 8 illustrates the acceleration feature of the right hand in the Pointing gesture performing with the right hand when feeling Angry comparing to the Neutral state. Another point, according to features importance results, the shape characteristics do not appear significant on the expression of Happy and Angry emotions.

The Sad emotion is characterized by Shape and Effort characteristics in all gestures. For example, in the Moving gesture, we obtained as important features the following Shape characteristics: the convex hull volume of the skeleton, as well as the body inflation according to the sagittal and frontal planes. In the Effort component, urgency (Time), strength (Weight), straightness (Space), and smoothness (Flow) of the movement were considered as important characteristics in expressing the Sad emotion in the Moving gesture. In conclusion, based on the machine learning algorithm we can draw the following relationships between the three emotions and Effort-Shape factors. The Happy emotion was described by Time and Weight Effort factors. However no shape factors is important for describing the Happy emotion, we can explain this by the choice of the gestures. In our case, our challenge is to be able to recognize the emotions through limited gestures so for the Happy emotion expressed through a limited gesture we can not have a remarkable difference in the body shape when performing the gesture with happy and neutral states. The only difference can appear in the speed and the strength of the gesture. For the Angry emotion, the most important factor is the weight followed by the Time factor. For the Sad emotion, Shape and Effort factors are important. Unlike Happy and Angry emotions, shape factors participate well in characterizing the Sad emotion. This can be explained by the fact that performing limited gesture when feeling sad can contribute to a strong contraction in the whole body, so comparing to the Neutral state, we can notice a difference between them. Finally, the Directional movement factor did not have any importance in expressing the three emotions We can say that this factor is not important in such case when performing limited gestures with emotions.

3.4 Expressive motion recognition and analysis using the statistical method

Recognition

The purpose of this study is to analyze human emotions based on the human perception. We will see how can a person recognize human emotions through her body movement. We tried to make an external evaluation of the emotions expressed in the videos. We recruited 10 viewers (5 men and 5 women) from the University of Evry Val d'Essonne ranged in age from 28 to 37 years old (M=30.9 years, SD=3.16). They watched recorded videos and evaluated each emotion using the 5item Likert scale (1=strongly disagree, 3=neutral, 5=strongly agree) to determine whether they recognized or not the expressed emotion. For a reliable evaluation, expressive motions in videos were reproduced by a virtual avatar as shown in Figure 1. This help viewers to make scores without being influenced by some factors like facial expressions, gender, etc.

Features importance

Once the evaluation of the emotions was completed, we proceeded to the evaluation of motion features in each observed video in order to study the correlation between emotions and body features. In this part, we focused on the Effort-Shape components because they are the two major direct specifications or indications of expressive human movements. So, we have 7 factors to evaluate (Shape Flow, Shaping, Directional Movement, Time, Weight, Space, and Flow). We asked another group of 10 viewers from the University of Evry Val d'Essonnes (age from 28 to 31 years) to watch the recorded videos and evaluate each selected features using a 7 point Likert scale as follows:

-Shape Flow: convex hull volume of the body (1=Very shrinking, 7=Very Growing) -Shaping: distance between spine joint and body extremities according to three planes (1=Very Contracted, 7=Very Extended) -Directional Movement: the pathway of the body joints (1=Very Curvilinear, 7=Very Rectilinear ) -Time: speed of movement (1=Very Sustained, 7=Very Quick) -Weight: the strength of movement (1=Very Light, 7=Very Strong) -Space: straightness of movement (1=Very Indirect, 7=Very Direct) -Flow: smoothness of movement (1=Very Free, 7=Very Bound) For example, to rate the Shape flow factor we should look at the development of the volume of the 3D skeleton convex hull. If you perceived that the skeleton volume has been strongly increased throughout movement you rate the Shape Flow factor with 7, and if you perceived that it has been strongly decreased you make 1 as a score.

Results

Inter-viewer reliability

The verification of the consensus among viewers in their decisions is needed in the direction of the emotions perception and the motion features rating with Likert scale. Thus, we used the Cronbach's alpha coefficient to estimate the reliability of the scales given by the viewers when evaluating emotions and motion features. In other words, the Cronbach's alpha is one way of measuring the strength of consistency between viewers decisions. The formula used to compute the Cronbach's alpha coefficient is:

α = n n -1 (1 - n i=1 V i V T ) ( 9
)
where n is equal to the number of items, V i refers to the variance of scores on each item, V T represents the total variance of overall scores. As we can see in Figure 9, the Cronbach's alpha coefficient is always higher than 0.8 (Happy (0.897), Angry (0.894), Sad (0.879), Neutral (0.814)) which means, according to [START_REF] Tavakol | Making sense of cronbach's alpha[END_REF], there is a high consistency in the ratings given by the viewers for the perception of emotions. We also measure the raliability of scales given by the viewers for Effort-Shape factors (Time, Weight, Space, Flow, Shape flow, Shaping, and Directional movement). According to results illustrated in Figure 10, Cronbach's alpha coefficient is higher than 0.7 (acceptable level) for all features rating. So, we can say that viewers were self-consistent when evaluating motion features in our emotional dataset, especially, Time and Weight factors with a reliability coefficient of 0.958. This result shows a strong homogeneity between the viewers when estimating Time and Weight factors. However, there is a less important but acceptable correlation between the viewers when rating Space, Flow and Directional movement factors with Cronbach's alpha of 0.754, 0.763, and 0.703, respectively. For Shaping and Shape Flow factors, there is a good agreement in the ratings given by the viewers with Cronbach's coefficients of 0.830 and 0.839, respectively.

Recognition: For emotions recognition based on viewers ratings, we take the 1100 videos for each emotion perception. After finishing the emotions rating by the viewers using the 5-Likert scale, for each emotion perception we consider a recognized emotion when the scale is superior to 3 (neutral). We present the confusion matrix between expressed emotions and perceived emotions in Figure 11. The diagonal cells correspond to the number of correct emotions recognition reported by the viewers (scale > 3). Off-diagonal cells contain frequencies of incorrect classifications. We can conclude that the viewers have correctly classified most emotions in each gesture with some confusions occurred especially between Happy and Angry emotions where they both have high arousal but only differ in the valence, and between Sad and Neutral states. So, based on viewers evaluation we found the same results obtained with the RDF classifier.

After, we considered all expressive gestures and we computed the mean of viewers ratings in each expressed emotion. As we can see in Figure 12, the Happy emotion was successfully recognized by the viewers with some confusions with the Angry emotion and the lowest mean rating was associated with the Sad emotion. For the expressed Angry emotion, the highest mean rating was achieved in the perception of the Angry emotion followed by the perception of the Happy emotion. Sad and Neutral states were well recognized by the viewers in different motions with a bidirectional confusion between the two emotions by certain viewers.

Features importance: To study features importance in characterizing human emotions, we measured the correlation between Effort-Shape factors and emotions mean ratings by the viewers using the Pearson correlation coefficient. Pearson coefficients are used in statistics to measure how strong a relationship is between two variables and referred to as Pearson's r, with a value ranging between -1 for a perfectly negative correlation and +1 for a perfect positive correlation, with 0 represents no correlation. The Pearson correlation between the variables X and Y is calculated as: [START_REF] Masuda | Motion rendering system for emotion expression of human form robots based on laban movement analysis[END_REF] have also confirmed in their experiment the same relationship between Happy emotion and Effort qualities.

r XY = n i=1 (X i -X)(Y i -Ȳ ) n i=1 (X i -X) 2 n i=1 (Y i -Ȳ ) 2
For the Angry emotion, there is a strong positive correlation with the following three Effort factors (Time r = 0.622, p < 0.001; Weight r = 0.640, p < 0.001; Space r = 0.682, p < 0.001) and a positive moderate correlation with the Directional Movement shape factor r = 0.568, p < 0.001. The Flow factor was negatively associated with the Angry emotion. So, the Angry emotion was strongly characterized by abrupt, strong, direct, rectilinear, and free movements. Some qualities are consistent with those found by Masuda et al. [START_REF] Masuda | Motion rendering system for emotion expression of human form robots based on laban movement analysis[END_REF]. The authors have resulted in Angry emotion being correlated with quickness and powerfulness. For Happy and Angry emotions, we found a similar relationship with Effort-Shape factors but with different correlations importance. In Effort qualities rating, the Angry emotion was evaluated as significantly faster, stronger and freer. However, in Shape factors rating Happy emotion was evaluated with a more expanded shape.

The Sad emotion was negatively correlated with all Effort-Shape factors expect the Flow quality: Shape factors (Shape Flow r = -0.524, p < 0.001; Shaping r = -0.613, p < 0.001; Directional Movement r = -0.505, p < 0.001), Effort factors (Time r = -0.795, p < 0.001; Weight r = -0.780, p < 0.001; Space r = -0.566, p < 0.001). According to Shape factors rating, the Sad emotion was significantly characterized by shrinking shape, contracted body extremities, and curved path. According to Effort factors rating, the Sad emotion was characterized by light, bound, sustained and indirect movement.

The Neutral emotion was associated with the same Effort-Shape qualities as the Sad emotion but with a lower correlation importance. Masuda et al. [START_REF] Masuda | Motion rendering system for emotion expression of human form robots based on laban movement analysis[END_REF] found that both emotions (relaxed and sadness) correlated with same Effort qualities (slowness and weakness).

Discussion

The purpose of this study is to characterize and recognize expressed human emotions through body movement first based on machine learning method. Experimental results confirmed the performance of the classifier as well as the proposed descriptor movement. Secondly, we considered the human classifier approach as a form of baseline. So, we studied the robustness of the RDF method in recognizing human emotions and studying discriminate body features comparing to humans perceptions method. We obtained the following conclusions:

-In emotions recognition study, both RDF and human classifiers have confused between Happy and Angry emotions, and between Sad and Neutral emotions. But, as presented in the confusion matrix when using RDF (Figure6) and human (Figure11), we note that the machine learning method was more accurate than observers. This may be due to the limited number of observers who participated in the rating of the different sequences. Also, it can be explained by the type of gesture chosen. Our approach is very challenging, we try to recognize emotions through limited gestures. For example, through performing the pointing gesture via Sad emotion or Neutral the same movement with a somewhat stable ruthme is performed. So, visually it will not be easy for observers to distinguish between the two types of movement. -In features importance with the human approach, each observer have evaluated the importance of each LMA factor in characterizing each emotion. With the RDF classifier, the algorithm consists in studying the importance of the features in characterizing emotions comparing to the neutral state. Moreover, we applied the Tukey's test to remove redundant features. As presented in the Table 2, the Sad emotion was characterized by Effort-Shape factors, except the directional movement factor. The same result was obtained with the evaluation of the observers in the Table 3. However, for Happy and Angry emotions, the RDF classifier found that the two most discriminative features in characterizing both emotions are the Time and the Weight factors. These results confirm that the proposed method allows us to characterize the emotions and define the important characteristics while optimizing our motion descriptor by keeping the most robust and significant features.

Conclusion and perspectives

The present study set out to investigate the perception of 4 emotions expressed through 5 different motions as well as the importance of the features for each expressed emotion. We exploited the LMA method to represent and analyze the expressive content of movements through several description levels. Two studies are examined: the recognition of emotions through predefined gestures and the selection of most important motion features in characterizing each expressed emotion. Two different methods are proposed, a machine learning method based on the RDF algorithm and a statistical method based on viewers ratings. We used our own dataset composed of 5 expressive gestures performed by 11 subjects captured by the MVN Awinda system. The participants in our dataset are all from the University of Evry Val d'Essonnes. We did not choose professional actors in order to achieve a system accessible to everyone. Our dataset has then the ability to be enriched by everyone without the need for professional actors. The result of this work according to both methods is detailed as follows: For emotions recognition step, different emotions are successfully recognized by two methods, learning and statistical. We also confirm the robustness of our descriptor vector in discriminating both gestures and emotions. For features selection step both methods, RDF, and viewers rating, demonstrate that Happy and Angry emotions are significantly characterized by Time and Weight qualities. They involve strong and fast movements with higher intensity in Angry emotion. The Sad emotion was characterized with all Effort-Shape factors expect the Directional movement factor. Our proposed machine learning method which combines RDF and Tukey's test has successfully kept the most discriminative features which lead to the optimization of our descriptor motion. In the statistical approach, despite recruited people non-expert in the Laban Movement Analysis field in order to express and analyze emotions, they were able to distinguish the Laban features in the movements, which allows our framework to be used even by non-experts. On the other hand, we still have the opportunity to enhance the consistency between the viewer's ratings with some training Laban's theories and techniques. Notwithstanding the above limitations, this study confirms the robustness of our motion descriptor to discriminate expressive motions and give us useful information about the relationship between human states and motion features which allows enhancing interactive applications by regulating body features for each desired emotion.

Fig. 1

 1 Fig. 1 Expressive gestures dataset in order from top to bottom: Dancing, Moving, Stopping, Waving and Pointing.

Fig. 2

 2 Fig. 2 Dancing gesture expressed with Happy and Sad emotions reproduced by a virtual avatar.

Fig. 3

 3 Fig. 3 LMA Components (Body, Space, Shape, and Effort).

Fig. 4

 4 Fig. 4 Body characteristics.

F = 2 Fig. 5

 25 Fig. 5 Optimal RDF parameters (ntrees, maxf eatures).

Fig. 6

 6 Fig.6Confusion matrices between expressed emotions (in rows) and perceived emotions (in columns) for each gesture using RDF method.

Fig. 7

 7 Fig. 7 Confusion matrix of all expressive gestures, 5 gestures (D dancing, M moving, W waving, S stopping, and P pointing) performed with 4 states (H happy, A angry, S sad, and N neutral).

  (a) v l , vr in Moving. (b) a l ar in Moving. (c) ar in Pointing.

Fig. 8

 8 Fig. 8 Motion features variation across states (purple/orange curves for left/right hands in Happy emotion; yellow/blue curves for left/ right hands in the Neutral state; black curve for the right hand in Angry emotion).

Fig. 9

 9 Fig.9Inter-Viewer reliability of the emotions perception using the Cronbach's alpha.

Fig. 10

 10 Fig.10Inter-Viewer reliability of Effort-Shape features rating using Cronbach's alpha.

Fig. 11

 11 Fig. 11 Confusion matrices between expressed emotions (in rows) and perceived emotions (in columns) for each gesture based on viewers rating.

Fig. 12

 12 Fig. 12 Mean ratings of emotions perception by 10 viewers.

  p v 0 is the whole feature set.Output: v * = {f j }, j = 1, . . . , p * v * subset of most relevant features. 1 k = 0 2 while |v k | ≥ 1 doCompute and record OOB error rate: E k (v k )

	3
	4	for i = 1 to p do
	5	Compute I(f i )	Importance of each feature in v k .

6 end 7 Sort {f i } in descending order according to values of I(f i ) 8 f min = argmin i {I(f i )} 9 Apply Tukey's test and select set of features {ft} that does not lead to a significant changement of E k 10 R = f min {ft} 11

Table 1

 1 Emotions recognition results (Accuracy, Precision, Recall, and F-score) on the different gestures of our dataset.

	Gestures	Accuracy (%)	Precision (%)	Recall (%)	F-score
	Waving	90	90	89	0.89
	Moving	78	77	76	0.75
	Dancing	62	78	76	0.75
	Stopping	86	88	86	0.86
	Pointing	95	95	94	0.94
	All gestures	81	86	81	0.80

Table 3 summarizes

 3 Pearson coefficients results obtained when measuring the correlation between the four emotions and Effort-Shape factors ratings: For the Happy emotion, we found a positive correlation with the Shape factors (Shape Flow

Table 3

 3 Pearson's correlation coefficients r between Effort-Shape factors and expressed emotions ratings (**. correlation is significant at the 0.001 level). Shaping r = 0.542, p < 0.001), and the following Effort factors (Time r = 0.555, p < 0.001; Weight r = 0.543, p < 0.001). So, the Happy emotion was significantly characterized by growing shape and extended body members. It was also associated with quickness and powerfulness qualities. Masuda et al.

			Happy	Angry	Sad	Neutral
		Shape Flow	0.541**	0.132	-0.524**	-0.316**
	Shape	Shaping	0.542**	0.194**	-0.613**	-0.328**
		Directional mvt	0.269**	0.568**	-0.505**	-0.397**
		Time	0.555**	0.622**	-0.795**	-0.554**
	Effort	Weight Space	0.543** 0.326**	0.640** 0.682**	-0.780** -0.566**	-0.594** -0.487**
		Flow	-0.316**	-0.665**	0.559**	0.497**
	r = 0.541, p < 0.001;

5

  Compliance with Ethical Standards -Funding: This study was funded by the Strategic Research Initiatives project iCODE, University Paris Saclay. -Conflict of Interest: Author Insaf Ajili declares that she has no conflict of interest. Author Zahra Ramezanpanah declares that she has no conflict of interest. Author Malik Mallem declares that he has no conflict of interest. Author Jean Yves Didier declares that he has no conflict of interest.

Acknowledgements We would like to thank the staff of the University of Evry Val d'Essonne for participating in our dataset. As well, we would like to thank Mrs. Alice Jourdan for help with data gathering and tabulation. This work was partially supported by the Strategic Research Initiatives project iCODE accredited by University Paris Saclay.