Marie Tahon

Laurence Devillers

Towards a

Towards a small set of robust acoustic features for emotion recognition: challenges.

Marie Tahon, Laurence Devillers, Associate Member, IEEE.

Abstract-The search of a small acoustic feature set for emotion recognition faces three main challenges. Such a feature set must be robust to large diversity of contexts in real-life applications; model parameters must also be optimized for reduced subsets; finally, the result of feature selection must be evaluated in cross-corpus condition. The goal of the present study is to select a consensual set of acoustic features for valence recognition using classification and non-classification based feature ranking and cross-corpus experiments, and to optimize emotional models simultaneously.

Five realistic corpora are used in this study: three of them were collected in the framework of the French project on robotics ROMEO, one is a game corpus (JEMO) and one is the wellknown AIBO corpus. Combinations of features found with nonclassification based methods (Information Gain and Gaussian Mixture Models with Bhattacharyya distance) through multicorpora experiments are tested under cross-corpus conditions, simultaneously with SVM parameters optimization. Reducing the number of features goes in pair with optimizing model parameters. Experiments carried on randomly selected features from two acoustic feature sets show that a feature space reduction is needed to avoid over-fitting. Since a Grid search tends to find non-standard values with small feature sets, the authors propose a multi-corpus optimization method based on different corpora and acoustic feature subsets which ensures more stability.

The results show that acoustic families selected with both feature ranking methods are not relevant in cross-corpus experiments. Promising results have been obtained with a small set of 24 Voiced Cepstral coefficients while this family was ranked in the 2nd and 5th positions with both ranking methods. The proposed optimization method is more robust than the usual Grid search for cross-corpus experiments with small feature sets.

Index Terms-emotion recognition, acoustic features, crosscorpus, SVM parameter optimization, Information Gain, Bhattacharyya distance.

I. INTRODUCTION

I N the field of human-robot interaction (HRI), emotion recognition can be useful for many applications, especially with robotic assistants and social companions. Non-verbal information and particularly audio information play an essential role in human communication [START_REF] Devillers | Automatic detection of emotion from vocal expression[END_REF], [START_REF] Devillers | Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs[END_REF], but undoubtedly, complementary emotional information exist in other modalities. The authors are involved in the French project on robotics ROMEO 1 . The robot designed in this project faces rich affective interactions in different contexts (speaker's acoustic environments, tasks, real end-users) and can collect inputs from various sensors, such as audio, video, tactile and physiological. According to Batliner et al. [START_REF] Batliner | Whodunnit searching for the most important feature types signalling emotion-related user states in speech[END_REF], M. Tahon and L. Devillers are with the Human-Machine Communication Department, LIMSI-CNRS, 91403 Orsay, France L. Devillers is with the University Paris-Sorbonne IV, 28 rue Serpente, 75006 Paris, France 1 http://www.projetromeo.com obtaining large amounts of real-life data is currently one of the most important hurdles. Since many situations must be represented in real-life data, the collection of real-life emotions requires having large panels of speakers, several different acoustic environments, different emotional and social contexts, which are expensive to set up. Available corpora for emotion recognition in real tasks are nowadays still sparse and contain few instances. Furthermore, most of the available corpora are not yet in the public domain for privacy reasons. Cross-corpus conditions allow to deal with the issue of emotional data availability. Cross-corpus experiments face different challenges: real-life data, rich affective contexts and small corpora.

Because of all these challenges in real-life emotional data cross-corpus experiments, researchers must generalize emotional models as much as possible. The authors' hypothesis is that an appropriate feature space reduction allows generalizing the models, then improves the recognition rate for unseen data [START_REF] Huang | A ga-based feature selection and parameters optimization for support vector machines[END_REF]. Of course, feature space reduction has also the advantage of being computationally faster, which is highly positive for real-time recognition systems such as robots. Indeed, feature space reduction allows to avoid over-fitting and models tend not to learn by heart the training corpus. Different techniques have already been experimented in affective computing for selecting the best acoustic features. However, few experiments are carried on real-life data in multi-corpus conditions. Automatic feature selection techniques select a subset of features according to a criteria (usually the classification rate itself). This technique has the advantage of being easy to perform in cross-corpus conditions, however it is very time-consuming. Feature ranking techniques are very interesting since they are easy to analyze, but, as of now, such techniques have been used with only one corpus in cross-validation conditions. The originality of the present work lies in the use of two ranking methods to find the most relevant families of features: individual ranking with Information Gain and group of features ranking with Gaussian Mixture Models and the Bhattacharyya distance. Random combinations of features are tested as baselines. The families of features found with these methods are tested under cross-corpus emotion recognition experiments.

In section II, the state of the art on emotional databases, acoustic features and multi-corpus experiments are described. In section III, the authors present the five databases used in their experiments. Section IV summarizes the set of acoustic cues. Section V presents feature space reduction protocols and results. Section VI presents a multi-corpus optimization of SVM parameters and cross-corpus valence recognition experiments with different subsets of acoustic features. Results and conclusions are drawn in the last section.

II. STATE OF THE ART

In this section, the following three topics are addressed: the existing emotional databases (section II-A), the current state of the art on acoustic features for emotion recognition (section II-B) and previous experiments that use more than one database (section II-C).

A. Emotional databases

This section presents the definition of emotion used in the article, the collection of realistic databases and HRI databases, and finally, segmentation and annotation issues for emotional databases.

1) Emotion definition: Emotion is a complex and multidisciplinary field of research and finding a consensual definition is a hard task. According to Scherer [START_REF] Scherer | Vocal communication of emotion: a review of research paradigms[END_REF], the number of human emotions occurring in the context of social interactions is infinite, subtle and often mixed. The complex nature of emotion can be described by discrete labels [START_REF]What are emotions ? and how can they be measured ?[END_REF] and continuous dimensions [START_REF] Russell | Evidence for a three-factor theory of emotions[END_REF], notably valence (positivity), arousal (responsiveness) and control (dominance).

2) Acted versus realistic databases: The collection and annotation of emotional databases are crucial issues in emotion recognition and many studies have already been realized in the framework of HUMAINE2 [START_REF] Douglas-Cowie | The humaine database : Addressing the collection and annotation of naturalistic and. in induced emotional data[END_REF], [START_REF] Douglas-Cowie | The humaine database[END_REF]. A decade ago, emotion recognition models were trained using acted artificial data, such as the Danish Emotional Speech (DES) [START_REF] Engberg | Design, recording and verification of a danish emotional speech database[END_REF] or the Berlin Emotional Speech-Database (EMO-DB) [START_REF] Burkhardt | A database of german emotionnal speech[END_REF]. The two main drawbacks of the standard corpora used in the community are the very small size of audio corpora and the data variability in terms of task, speaker, age and audio environment which compromise the significance of results and improvements [START_REF] Schuller | Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge[END_REF]. As a consequence, there is a critical need for data collection with end-users (with different types of speakers, ages) and real tasks for emotion recognition systems since realistic emotions could not be found in acted databases [START_REF] Batliner | Laryngealizations and emotions: How many babushkas?[END_REF]. However, in real-life contexts, emotions are quite sparse, neutral speech is predominant and emotions are shaded [START_REF] Devillers | Challenges in real-life emotion annotation and machine leraning based detection[END_REF], [START_REF] Batliner | The automatic recognition of emotions in speech[END_REF]. Some prototypical databases are also collected. They contain consensual emotions defined as "utterances that are consistently recognized by a set of human evaluators" [START_REF] Mower | Interpreting ambiguous emotional expressions[END_REF]. Therefore, their collection with large panels of speakers and different acoustic environments is simplified.

3) Human-Robot Interaction emotional data: In order to implement emotional models in systems such as robots, new data must be collected during interactions with these systems. So far, very few HRI databases have been collected. Among them, AIBO database [START_REF] Batliner | you stupid tin box» -children interacting with the aibo robot: A cross-linguistic emotional speech corpus[END_REF] was collected during a child-robot interaction with the AIBO Sony pet. The SEMAINE database [START_REF] Mckeown | The semaine database: annotated multimodal records of emotionally coloured conversations between a person and a limited agent[END_REF] was recorded during emotionally rich interactions with an automatic agent. The Herme database [START_REF] Han | Speech & multimodal resources: The herme database of spontaneous multimodal human-robot dialogues[END_REF] was collected in the Science Gallery of Dublin, during conversations between visitors and a robot. Some French emotional HRI databases have also been collected with visually-impaired people (IDV-HR [START_REF] Tahon | Real-life emotion detection from speech in human-robot interaction: Experiments across diverse corpora with child and adult voices[END_REF]) and children (corpus NAO-HR [START_REF] Delaborde | A wizard-of-oz game for collecting emotional audio data in a children-robot interaction[END_REF], [START_REF]Affective links in a child-robot interaction[END_REF]) interacting with the robot Nao. The collection of such data is more challenging than acted data (protocol, speakers, annotation) and are usually not freely available.

4) Segmental annotations:

Annotations can be realized at different time level: whole sentence, segment (homogeneous emotion among the segment), word, phoneme, etc. [START_REF] Batliner | Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach[END_REF]. Therefore the first task is a segmentation task. Segments boundaries can be defined automatically and/or manually. Annotation can be either supervised: categories and/or dimensions are defined before the annotation task, or unsupervised: the annotator chooses the word which best fits the emotion he or she perceives. In the first case, emotional states are constrained and the annotation scheme is highly context-dependent, but finding a consensus between annotators is easy. In both cases, due to the scarcity of data, precise emotion labels have to be mapped onto macro-classes. For example, hot anger is remarkable and is easily annotated as anger, but other anger micro-classes (such as boredom, irritation, ...) are specific to each corpus. Macro-classes are usually driven by emotional theories such as the "big six" theory from Ekman [START_REF] Ekman | Handbook of cognition and emotion[END_REF]. Valence and Arousal are often annotated on a continuum, but categories are also used [START_REF] Delaborde | A wizard-of-oz game for collecting emotional audio data in a children-robot interaction[END_REF].

B. Acoustic features

This section deals with the extraction and normalization of acoustic features. The "holy grail" is to find which features are the best for modeling the emotional speech [START_REF] Eyben | The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing[END_REF]. The section also presents a review on multi-corpus experiments.

1) Features reviews:

In the last years, a lot of acoustic features have been developed for speech signal analysis and musical information extraction. Most of them are also used for emotional speech [START_REF] Batliner | Whodunnit searching for the most important feature types signalling emotion-related user states in speech[END_REF], [START_REF] Schuller | Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge[END_REF], [START_REF] Schuller | Paralinguistics in speech and language, stateof-the-art and the challenge[END_REF]. Acoustic feature sets used in emotion recognition models mainly contain prosodic (pitch, energy, rhythm), timbre and voice quality features. The HUMAINE association also took an inventory of acoustic features in the CEICES initiative [START_REF] Batliner | Ceices : Combining efforts for improving automatic classification of emotional user states: a ń forced co-operation ż initiative[END_REF]. Some features are segmental (they are computed at the frame level), others are supra-segmental (they are computed on a whole sentence, or emotional instance). Some are computed on voiced, others on unvoiced signals [START_REF] Clavel | Fiction supports for realistic portrayals of fear-type emotional manifestations[END_REF].

2) Reduction of the feature space: The most difficult task in building emotion detection systems is to find the most appropriate acoustic features [START_REF] Schuller | The relevance of feature type for the automatic classification of emotional user states: low level descriptors and functionals[END_REF]. The "best feature set" would be small enough to be implemented in a real-time system (not much time-consuming) and robust enough to detect real-life emotions. There are two main approaches for this task. The first one consists in extracting all state-of-theart acoustic features and apply automatic feature selection algorithms to reduce the feature space. Another approach consists in studying and analyzing precisely each feature to find relevant acoustic features. Precise analysis is carried on clean and sparse data and will not be treated in the present paper, whereas feature selection is used on corpora which are collected in different situations. Both approaches are complementary for affective speech study. Techniques which reduce the feature space dimensionality, consist in a mapping of the input acoustic space onto a less dimensional one, while keeping as much information as possible. The Principal Analysis Component (PCA) is one of the most common technique. PCA reduces the feature space without degrading emotion recognition rate [START_REF] Lee | Toward detecting emotions in spoken dialogs[END_REF]. The interpretation of the selected variables is not easy with PCA since variables are projected variables and do not necessarily have a physical sense.

3) Feature ranking and selection algorithms: Feature ranking and selection algorithms have been extensively developed and studied in literature. Filters founded on information theory such as the Information Gain or Best First algorithms have been also used for feature selection even if they perform individual feature ranking [START_REF] Polzehl | Approaching multi-lingual emotion recognition from speech -on language dependency of acoustic/prosodic features for anger recognition[END_REF], [START_REF] Kostoulas | Enhancing emotion recognition from speech through feature selection[END_REF]. Filters usually have the advantage of being computationally simple. Wrappers selection employ a target classifier's rate as optimization criterion. The Sequential Forward Search technique has been used for emotion recognition [START_REF] Brendel | A quick sequential forward floating feature selection algorithm for emotion detection from speech[END_REF]. Four feature selection algorithms have been compared by Altun & Gökhan [START_REF] Altun | Boosting selection of speech related features to improve performance of multi-class svms in emotion detection[END_REF], showing that the Least Square Bound Feature Selection algorithm is superior in terms of reducing the average cross-validation error. This study also shows that prosodic and sub-band energy features are the most selected ones by all algorithms. The group-Lasso algorithm does variable selection on (predefined) groups of variables in linear regression models [START_REF] Meier | The group lasso for logistic regression[END_REF]. The advantage of this algorithm is to perform ranking using all features together. Most of the experiments described in literature are carried on one single corpus. Even if some general trends appear among the selected features, it is quite hard at that time to find a consensus. However, prosodic features -pitch, sound intensity and duration -and cepstral features seem to be highly relevant for emotion recognition [START_REF] Wu | Acoustic feature analysis in speech emotion primitives estimation[END_REF], [START_REF] Rong | Acoustic feature selection for automatic emotion recognition from speech[END_REF], [START_REF] Tao | Features importance analysis for emotional speech classification[END_REF]. One of the main problem is that the selected features are usually different from one corpus to another and from one task to another and also depend on the selection technique used.

C. Multi-corpus experiments

1) Cross-corpus protocols: Experiments on a single corpus do not enable estimation of the influence of the sources of variability on emotion recognition because they are kept constant. Many studies are carried on one corpus only and researchers cannot generalize the results to other corpora. Cross-corpus experiments consist in using one corpus as training set and using another one as test set. By this way, recognition rates are low but more realistic [START_REF] Schuller | Cinemo -a french spoken language resource for complex emotions: facts and baselines[END_REF]. Three protocols are predominant in cross-corpus experiments (with N corpora). The leave-one-corpus-out protocol (the "unit" protocol) consists in training on N -1 corpus, testing on the last one [START_REF] Schuller | Cross-corpus acoustic emotion recognition: variances and strategies[END_REF], [START_REF] Eyben | Cross-corpus classification of realistic emotions: some pilot experiments[END_REF]. The model can be trained with a few corpora and improved with an unsupervised adaptation [START_REF] Zhang | Unsupervised learning in cross-corpus acoustic emotion recognition[END_REF]. It allows merging different data: fixed or variable linguistic information, realistic or acted data, children or adults. In Marchi et al. [43], the authors conclude that accumulation of speech data similar to the testing conditions in the training set improves the recognition performances. The second protocol consists in training on one corpus and testing on the N -1 other corpora independently. Then, the recognition rate for each corpus is obtained via the majority voting (the "vote" protocol [START_REF] Schuller | Using multiple databases for training emotion recognition: to unite or to vote ?[END_REF], [?]) or average classification rate ( [START_REF] Tahon | Real-life emotion detection from speech in human-robot interaction: Experiments across diverse corpora with child and adult voices[END_REF]) of the N -1 classifiers trained independently on the remaining corpora.

Since the Interspeech 2009 Emotion Challenge [START_REF] Schuller | The interspeech 2009 emotion challenge[END_REF], the evaluation of emotion detection systems has been standardized with the UA (Unweighted Average recall) rate. Schuller et al. [START_REF] Schuller | Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization[END_REF] performed a cross-corpus test with the "vote" protocol using the acoustic set available for the Interspeech 2012 Challenge using z-normalized (or mean variance normalized) features. For binary valence recognition, the UA results on seven corpora -acted (DES, EMO-DB), induced (eNTERFACE) and realistic (VAM, SAL and SUSAS) -are from 50% to 55%. Such an experiment is very interesting to validate a full cross-corpus protocol on merging corpora with a large number of database. Another strategy consists in finding which corpus (or group of corpora) is the best for training the emotional model and which features best fit the final application [START_REF] Wu | Acoustic feature analysis in speech emotion primitives estimation[END_REF], [START_REF] Yildirim | An acoustic study of emotions expressed in speech[END_REF]. The goal of this strategy is to analyze corpora and features, keeping in mind a specific application. In [START_REF] Devillers | Real-life emotion-related states detection in call centers: a cross-corpora study[END_REF], the authors show with cross-corpus experiments (on medical emergency and electricity supply call-centers recordings) that it is possible to generalize a corpus recorded in a certain context to the recognition of emotions elicited in other situations. Indeed, feature selection results or classification performances obtained with a training set and a test set extracted from the same corpus are usually not robust enough for other corpora. French corpora are rarely used in crosscorpus experiments. Most of the cross-corpus experiments are carried within the same language category because cross-language recognition rates may be even more worse.

2) Multi-corpus model parameters optimization: As far as the authors know, there are very few studies on model parameters optimization in emotional cross-corpus experiments. Most studies use Support Vector Models (SVM) and Sequential Minimal Optimization (SMO) function [START_REF] Schuller | Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization[END_REF], [START_REF] Busso | Iterative feature normalization scheme for automatic emotion detection from speech[END_REF]. The kernel parameter γ and the complexity c may be optimized using the GRID algorithm or set to fixed values. Optimization is usually performed on the training data, but a development database may be used for optimization purposes [START_REF] Chastagnol | Personality traits detection using a parallelized modified sffs algorithm[END_REF]. However, Huang and Wang say that "obtaining the optimal feature subset and SVM parameters must occur simultaneously" [START_REF] Huang | A ga-based feature selection and parameters optimization for support vector machines[END_REF].

3) Analyses of acoustic feature distributions: Acoustic feature distributions across all instances for each affective state of a given corpus, are full of information. In a precedent study, the authors of the present paper introduced a measure of the distance between anger and other emotions for a given acoustic feature [START_REF] Tahon | Acoustic measures characterizing anger across corpora collected in artificial or natural context[END_REF]. According to this distance, several corpora have been ordered in function of their spontaneity degree. Acoustic features distributions are usually modeled under Gaussian distributions. In Godin & Hansen [START_REF] Godin | Analysis of the effects of physical task stress on the speech signal[END_REF], Gaussian mixtures are used to model physical task stress and neutral speech. In this study, the symmetric Kullback-Leibler distance is used to measure changes in distribution from neutral to physical task stress. The Bhattacharyya distance could measure distance between GMMs to guide phone clustering [START_REF] Mak | Phone clustering using the battacharyya distance[END_REF]. Such a protocol is also used for image retrieval [START_REF] Sfikas | An analytic distance metric for gaussian mixture models with application in image retrieval[END_REF]. The Bhattacharyya distance is easy to compute for Gaussian distributions and is a powerful tool for dependent ranking features. Such a study is presented later in this paper.

III. DATABASES

This section presents the five databases used in the following experiments. Four of them are not available for privacy reasons. Therefore, the addition of a publicly available database such as AIBO will enable the scientific community to compare the results. Three of them are collected in the framework of the French project on robotics ROMEO; the prototypical JEMO corpus is collected in the course of an emotion detection game; the AIBO corpus is collected with children interacting with the robot AIBO. This choice guarantees real-life and humanmachine interactional data. All lexical contents are free. Two languages are spoken: French and German.

A. Acquisition protocols of ROMEO databases

The French ROMEO project aims at designing a social humanoid robot. This robot aims to assist elderly and disabled persons at home in everyday life activities, but should also be able to play games with children. The three corpora described below were recorded with a high quality lapel-microphone at the sample frequency of 16 kHz. All participants were native French speakers. The use of these databases carefully respects ethical conventions and agreements, ensuring the anonymity of the speakers, the information privacy and the non-diffusion of corpora and their annotations.

1) Human-System Interaction with visually impaired people (IDV-HS) [START_REF] Tahon | A corpus for identification of speakers and their emotions[END_REF]: IDV-HS contains 71 minutes of emotional speech. 28 speakers (11 males and 17 females) were recorded. The recordings took place in a relatively empty room (apart from some basic pieces of furniture) in an apartment block for visually impaired people, who are potential end-users for the ROMEO project. The originality of this corpus lies in the selection of speakers: for a same scientifically controlled recording protocol, we can compare young and elderly voices (from 23 to 89 years old). The experimenter presented six scenarios in which the participant had to pretend to be interacting with a domestic robot called ROMEO. The six scenarios were: medical emergency, suspicious noises, awaking in a good mood, with a bad health, in a bad mood, and finally a visit from close relations. The robot's voice was synthesized on a computer. For each presented scenario, the participant had to picture himself or herself in this context and to speak until the robot detects the right emotion. The emotion detection system was remotely controlled by an experimenter.

2) Human-Robot Interaction with visually impaired people (IDV-HR) [START_REF] Tahon | Real-life emotion detection from speech in human-robot interaction: Experiments across diverse corpora with child and adult voices[END_REF]: The corpus IDV-HR features visually impaired people (young and elderly people) interacting with the humanoid robot NAO from Aldebaran Robotics. 22 speakers from 20 to 80 years old (11 males and 11 females) were recorded in a furnished medical apartment in the IDV, for an amount of 4 hours of recordings. This corpus is collected for the study of both affective states and interactional behaviours. The speaker was asked to play three series of five scenarios (15 scenarios) in which he/she pictured himself/herself in a situation of waking up in the morning. The robot would engage him/her in conversation about either his/her health, or the program of the day, etc. Each of these five scenarios was devoted to a different affective state of the speaker: well-being, minor illness, depressed, medical distress, happiness. Each series of five scenarios differed from the other, by the social attitude of the robotthree attitudes among the followings: friendly, empathetic, encouraging, directive, doubtful or machine-like. The goal of having different social attitudes was to study interactional and social markers. The robot was remotely controlled by an experimenter who selected the social attitudes and the utterances which matched the content of the speaker's speech the best.

3) Human-Robot Interaction with children (NAO-HR) [START_REF] Delaborde | A wizard-of-oz game for collecting emotional audio data in a children-robot interaction[END_REF]: In the NAO-HR corpus, we focused on the role of the robot NAO as a game companion. Ten French children (5 males, 5 females), from 8 to 13 years old, were recorded for a total amount of about 30 minutes of recordings. As a game companion, NAO had to be able to supervise a game, while also being sensitive to the emotions of the children. It should for example be able to detect through the expressed emotions if a child is sad, or if there is a tension between the children. In the NAO-HR corpus, two children by session were recorded as they played with the robot at the LIMSI laboratory. Children were offered to play three games with the robot: a question-answer game, a song game and an emotion-acting game [START_REF]Affective links in a child-robot interaction[END_REF]. This corpus has also been collected to study affective and interactional markers during children-robot interactions [START_REF] Delaborde | Use of nonverbal speech cues in social interaction between human and robot: Emotional and interactional markers[END_REF], [START_REF]Impact of the social behaviours of the robot on the user's emotions: Importance of the task and the subject's age[END_REF].

B. Segmentation and annotation scheme of ROMEO databases

Instance's boundaries are defined on each speaker's track. An instance is emotionally homogeneous, i.e. the emotion is the same and of a constant intensity along the segment [START_REF] Devillers | Coding emotional events in audiovisual corpora[END_REF]. Since instances do not refer to word or sentence level, their durations vary quite a lot as shown in table II. Segmentation is done manually with the Transcriber tool [START_REF] Barras | Transcriber: development and use of a tool assisting speech corpora production[END_REF] since automatic segmentation using algorithms based on vocal activity detection often makes mistakes [START_REF] Batliner | Whodunnit searching for the most important feature types signalling emotion-related user states in speech[END_REF]. Some parasite noise happened to be audible in the studio (guide dog walking around, people working outside, etc.). When overlapping the speaker's speech, these parts were discarded during the manual segmentation.

Two expert coders perceptively annotated -as far as possible they did not take into account the semantic content -the segment along an annotation scheme inspired by the MECAS (Multi-level Emotion and Context Annotation Scheme) [START_REF] Devillers | Challenges in real-life emotion annotation and machine leraning based detection[END_REF]. This scheme enables the representation of complex and realistic emotions using a major and a minor emotion label for each instance. The corpora are annotated with affective state labels which describe each emotional instance (a major and a minor emotional label) and valence and arousal ratings (from -2 to 2). Valence dimension describes the sensation expressed by the speaker (positive or negative) and arousal dimension describes the strength of the expressed emotion (weak or intense). The annotation scheme also includes interactional dimensions (for other studies relative to the HRI), the audio quality and whether the participant is acting or talking spontaneously.

In the present paper, only the valence label is used. Valence is known to be one of the most challenging emotional dimension to recognize (in comparison to arousal for example) [START_REF] Schuller | Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization[END_REF]. It is defined with three different states: neutral, positive and negative (see table I). The negative state contains diverse emotions such as anger and sadness. The "neutral" state contains emotions which have not been labeled as either positive or negative. Only consensual instances are kept for our experiments, therefore the agreement between coders is perfect. The number of emotional segments per class is balanced not to favor a class more than the others. The number of instances used in the present paper and the characteristics of each corpus (total number of consensual instances before balancing, minimum, and mean instance duration, number of speakers and their age, and finally the number of segments after balancing classes) are summarized in table II.

A characteristic of importance is the instance duration after segmentation. Indeed, acoustic features need a minimum number of samples to be computed. Real-life instances' duration lasts less than 1 s whereas acted instances' duration lasts usually more than 1 s, therefore acoustic feature extraction on real-life data usually makes more mistakes. Many studies do not consider instances which last less than 1 s. This is the case in the AIBO corpus, as shown in table II. However these short instances often have strong emotional contents, generally called affect bursts (laughs, cries, etc.) [START_REF] Scherer | Affect Bursts[END_REF]. C. Benchmarks corpora 1) JEMO corpus [START_REF] Brendel | Building a system for emotions detection from speech to control an affective avatar[END_REF]: The JEMO corpus was collected in the course of the ANR (French National Research Agency) Affective Avatar project. It was recorded in order to obtain prototypical emotions. The speakers were asked to play an emotion-detection game during which the system had to recognize their emotion. The lexical content was totally free and the language is French. The system used a speech segmentation based on silences and pauses and an emotion recognizer based on four emotions: neutral, anger, sadness and positive (or joy). The system detected the emotions from the audio signal and presented the detected emotion on a screen visible to the player. 64 speakers were recorded (29 men, and 35 women) for a total duration of about 30 minutes. This corpus contains mainly prototypical emotions, i.e. emotions clearly recognizable by coders. In the present paper, the four emotional macro-classes have been grouped into three valence states according to table I. The number of instances used in the following experiments and the characteristics of the corpus are presented in table II.

2) AIBO corpus [START_REF] Batliner | you stupid tin box» -children interacting with the aibo robot: A cross-linguistic emotional speech corpus[END_REF]: The FAU AIBO corpus was collected during an interaction between a child (from 10 to 13 years old) and the Sony dog robot called AIBO. The German lexicon of the AIBO corpus comprises words that occur very frequently. The recordings took place in two different schools: in a class room (Mont) and in a playground (Ohm). In order to have similar acoustic environments in each corpus, the recordings of the Ohm school (AIBO-O) and the Mont school (AIBO-M) are split in two. The authors use two different configurations of the AIBO corpus. The CEICES configuration [START_REF] Batliner | Ceices : Combining efforts for improving automatic classification of emotional user states: a ń forced co-operation ż initiative[END_REF] is used for multi-corpus experiments (family selection, model parameters optimization and cross-corpus experiments). In this configuration, emotional instances are annotated with anger, motherese, empathy and neutral labels. The balanced number of emotional segments per class used in the present paper, is reported in table II. The authors also use the Interspeech 2009 configuration [START_REF] Schuller | The interspeech 2009 emotion challenge[END_REF] as a baseline to compare the reference acoustic feature set OpenSmile 2009 and the acoustic feature set used in this paper (see section IV-C). In the 2-class Interspeech challenge, only anger and idle (other emotional states) labels are used. In this configuration, classes are unbalanced. If then, the Weighed Average recall (WA) may be biased. For example all instances are recognized as neutral, the biggest class. This point is very important to be as close as possible to final applications.

IV. ACOUSTIC CUES

The authors developed their own feature set (Li-174), all experiments on acoustic family selection are carried on this specific set. However, for comparison purposes, they also use the baseline OpenSmile set (Os-384). Both feature sets are described in this section.

A. The 174 acoustic features set

The robot ROMEO will be endowed with a segmentation and feature extraction tool able to detect voice activity, and to compute acoustic features. Since the final goal is to embed the robot with an emotion recognition tool, this tool should extract the minimum number of acoustic features. Each segment is segmented on 30 ms frames every 10 ms using a Hamming window. Low-Level Descriptors (LLD) are extracted from the speech signal at the frame-level. Voiced and unvoiced features have been shown to be relevant for fear-type emotion recognition [START_REF] Clavel | Feartype emotion recognition for future audio-based surveillance systems[END_REF], LLD are extracted on voiced, unvoiced and the full instance. Statistical functions, or functionals, are applied to LLD, thus giving acoustic features at the segment-level. All acoustic cues are normalized to speaker with the Mean-Variance normalization.

174 acoustic features are referenced in the Li-174 set (table III). The choice of the features is inspired by existing feature sets (OpenSmile3 , Yaafe4 , etc.) but also from musical information extraction (perceptive features) [START_REF] Peeters | A large set of audio features for sound description (similarity and classification) in the cuidado project[END_REF]. They are grouped in seven acoustic families according to the LLD from which they are extracted. For example the Pitch family contains all functionals computed on the pitch: mean, standard deviation (std), maximum and minimum.

B. Detailed description of the acoustic set 1) Pitch (F 0 ): is extracted with Praat5 [START_REF] Boersma | Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound[END_REF] each 10 ms. The fundamental frequency is converted in semitones (st) since this scale is close to the human ear perception of frequencies. Praat tool also gives the pulses. Pulses define voiced and unvoiced parts of the speech signal. To limit the number of pitch errors, voiced parts which last less than 40 ms are discarded. For each voiced part, the Li-174 computes mean, std, minimum and maximum statistics on pitch. The mean values of these four statistics are given for each emotional instance.

2) Energy: has different definitions. To be as close as possible to the perceptive energy, the global loudness [START_REF] Peeters | A large set of audio features for sound description (similarity and classification) in the cuidado project[END_REF] is extracted in Li-174 with Matlab. Global loudness corresponds to the energy of the signal convolved with a perceptive filter (here a Bark filter). For voiced, unvoiced parts and the full signal, the Li-174 computes mean, std, minimum and maximum statistics on loudness.

3) Spectral features (SPE): usually describe the spectral envelope. Many methods exist for extracting the spectral envelope (true-envelope [START_REF] Röbel | Efficient spectral envelope estimation and its application to pitch shifting and envelope preservation[END_REF], Hilbert-Huang transforms [START_REF] Ringeval | Hilbert-huang transform for non-linear characterization of speech rythm[END_REF], etc.). The authors of the present paper chose to compute the LPC envelope, since it has shown its interest for emotion recognition [START_REF] Ruiz | Advanced voice analysis of pilots to detect fatigue and sleep inertia[END_REF]. Roll-off frequencies, spectral Centroid and spectral Slope which describe the shape of the envelop [START_REF] Peeters | A large set of audio features for sound description (similarity and classification) in the cuidado project[END_REF], are defined as follows (a(f ) is the amplitude of the signal at the frequency f ):

• Roll-off frequency f ro at x%: for emotion recognition [START_REF] Wu | Acoustic feature analysis in speech emotion primitives estimation[END_REF], [START_REF] Mower | Interpreting ambiguous emotional expressions[END_REF], [START_REF] Dumouchel | Cepstral and long-term features for emotion recognition[END_REF].

f RO 0 a 2 (f ) = x fe/2 0 a 2 (f ). • Spectral centroid f c : f c fe/2 0 a(f ) = fe/2 0 f • a(f ). • Spectral
6) Formants: are usual features in emotion recognition. They have been implemented in the 2010 Interspeech Challenge [START_REF] Schuller | The interspeech 2010 paralinguistic challenge[END_REF]. They are also present in many studies on emotions [START_REF] Devillers | Challenges in real-life emotion annotation and machine leraning based detection[END_REF], [START_REF] Vlasenko | Vowels formants analysis allows straightforward detection of high arousal acted and spontaneous emotions[END_REF]. Formant-based features such as articulation distance (which consists in an approximation of the area of the vocalic triangle), have been studied for emotion recognition in [START_REF] Yildirim | An acoustic study of emotions expressed in speech[END_REF], [START_REF] Lee | Emotion recognition based on phonem classes[END_REF], [START_REF] Beller | Articulation degree as a prosodic dimension of expressive speech[END_REF].

7) Voice quality (VQ): consists in many high-level features which were developed for voice transformation, speech synthesis, clinical purposes and emotion recognition. Among those descriptors, Harmonic to Noise Ratio (HNR), jitter, shimmer and the voiced vs. unvoiced ratio (punvoiced) have been shown to be relevant for valence recognition [START_REF] Tahon | Usual voice quality features for emotionnal valence detection[END_REF]. In the Li-174 feature set, HNR, jitter, shimmer and punvoiced are extracted with Praat. The harmonic number corresponds to the ratio:

f ro (95%) f ro (5%)
. This feature estimates the number of harmonics in voiced parts of speech energy.

C. Baseline

Since the Interspeech 2009 Emotion Challenge, the 384 OpenSmile acoustic set (Os-384) is considered as a reference. It consists in 16 LLD (fundamental frequency, RMS energy, Zero-Crossing Rate, Harmonics to Noise Ratio and 12 cepstral coefficients), their derivatives and 12 functionals (mean, variance, kurtosis, skewness, extremes functions and slope regression coefficients).

A comparison between the reference set Os-384 and the authors' set Li-174 is realized based on the challenge IS09 configuration: two non-balanced classes NEG (2465 instances) and IDL (5792 instances) with WEKA 6 . The training is performed with Ohm school instances and testing with Mont school instances. Results are shown employing the whole feature sets Os-384 and Li-174 with support vector classification (Sequential Minimal Optimization learning, linear kernel, normalization of training data): UAR (Os-384) = 63.8%, WAR (Os-384) = 73.0%, UAR (Li-174) = 60.5%, WAR (Li-174) = 71.7%. Results obtained with Os-384 are slightly better than with Li-174. One of the reason of the difference may be the number of features in each set. Because the number of features in Li-174 is twice less than in Os-384, the classification rate obtained with Li-174 is considered as relevant and will be used in the following experiments.

V. MULTI-CORPUS FEATURE SPACE REDUCTION

In order to reduce the feature space dimensionality, the authors choose to run two non-classification based ranking protocols and one random ranking protocol. The Information Gain (IG) protocol treat each feature independently while a combination of Gaussian Mixture Models and Bhattacharyya distance (G+B) treat family features as dependent groups. Each ranking experiment are applied independently on each of the six sub-corpora (cross-corpus) and on all merged corpora (merged).

A. Ranking feature families with Information Gain (IG) 1) Protocol: Each of the 174 attributes are continuous values. The WEKA's Information Gain Attribute Evaluation function uses the Minimum Description Length Principle Criterion [START_REF] Fayyad | Muli-interval discretization of continuous-valued attributes for classification learning[END_REF] to find the best cut-points. The Information Gain is the reduction in entropy obtained with the optimized cut-points. Best attributes lead to little cut-points and a small entropy; their information gain is maximum: IG = 1 [START_REF] Yang | A comparative study on feature selection in text categorization[END_REF].

The feature ranking is ran on the six sub-corpora at our disposal. This algorithm is applied to a three states valence and gives one value per segment and per attribute. For each corpus, the information gain IG(a) is attributed to each feature a.

The IG(f ) of an acoustic family is obtained while summing all IG(a) of the features belonging to the family (eq. 1). The relative information gain IG r (f ) is normalized to the sum of the 174 IG(a). The main drawback of this presentation is that the number of features per family differs, but what is timeconsuming is the extraction of the LLD, not the computation of functionals. Therefore the number of features belonging to the same family is not as important as the LLD. That is why the authors chose to compute the sum and not the mean information gain of all features belonging to the same family. Therefore, this protocol allows to identify best LLDs, not best features.

IG(f ) = N (f ) a=1 IG(a) (1)
2) Results: Multi-corpus (corpus-independent and merged corpora) results are reported in table IV. Differences in ranking between Energy Band (Bark and Harmonic bands) and Cepstral families is very small for IDV-HS, JEMO and AIBO-O. For the other sub-corpora, Energy band family clearly reaches the first place. Unexpectedly, Voice Quality family has a small IG r , while it was shown to be useful for valence recognition [START_REF] Tahon | Usual voice quality features for emotionnal valence detection[END_REF]. Formants and Pitch have also a small IG r . Pitch family is slightly more important in JEMO (IG r = 9.5%) and NAO-HR (IG r = 7.1%) compared to other corpora, probably because in both corpora, speakers were playing games which elicited more excitation. The trends observed on merged corpora results are similar to the ones observed on each corpus independently. More experiments show that Unvoiced and All features are often badly ranked for all corpora.

B. Ranking feature families with GMM and Bhattacharyya distance (G+B) 1) Protocol: Assuming that acoustic feature distributions are under Gaussian distributions, each valence class may be modeled using Gaussian mixtures (GMM) with random initialization. Then, the variance of the Gaussian model is a relevant factor to estimate if emotional classes are acoustically homogeneous or not. Experiments with 1 to 256 Gaussians show that 8 Gaussians seem to be enough for the present study. The Bhattacharyya distance [START_REF] Coleman | Image segmentation by clustering[END_REF] is a theoretical distance measure between two Gaussian distributions. The Bhattacharyya distance between two Gaussians d B,i,j (f ) for a set of features f (feature family), is given by equation 2, where M i , M j (resp. Σ i , Σ j ) are the mean matrices (resp. variances matrices) of the two Gaussian distributions i and j. The first term is linked to the well-known Mahanalobis distance which is null if the means are similar, even if the variances are different. On the contrary, the Bhattacharyya distance is not null when variances are different. It has the properties of being computationally simple, and to be extensible to Gaussian mixtures. The distance between classes C1 and C2 is the sum of the different Gaussians i and j according to their weights W i (equation 3) [START_REF] Sfikas | An analytic distance metric for gaussian mixture models with application in image retrieval[END_REF].

d B,i,j (f ) = 1 8 (M j -M i ) T Σ i + Σ j 2 -1 (M j -M i ) + 1 2 ln     det Σ i + Σ j 2 det(Σ i ) • det(Σ j )     (2) d B,12,GM M (f ) = N 1 GM M i=1 N 2 GM M j=1 W i • W j d B,i,j (f ) (3)
The final distance d B (f ) is the logarithm of the mean of the three distances Neu/Neg, Neu/Pos and Neg/Pos. The relative Bhattacharyya distance d r B (f ) is the Bhattacharyya distance d B (f ) normalized by the sum of all family distances. Because the GMM initialization is performed randomly, all results are averaged over 5 folds. For both ranking method, there is an impact of the feature family f dimension on the the family rank.

2) Results: The obtained results using the Bhattacharyya distance are presented in table IV. Acoustic family that reaches the highest relative distance is the Spectral family for all corpora independently and merged corpora. Loudness, Cepstral and EBB families just follow the Spectral family. It appears that Pitch and Voice Quality have not a high mean distance, it probably means that emotional classes are not acoustically homogeneous among these families. Formant family is relatively badly ranked when corpora are taken separately, whereas it reaches the fourth position, just before the Cepstral family (fifth position) when corpora are merged. Further results show that Unvoiced features are very interesting to discriminate valence classes in the two AIBO sub-corpora. Unvoiced features are slightly better ranked than Voiced features for all the four French corpora and all merged corpora. Some general trends exist among the six corpora, but each corpus has its own acoustic specificity. This confirms the fact that acoustic patterns of each valence class are different from one corpus to another, which was expected because of the differences in the recording and annotation protocols.

C. Subset of features

Since they aim for to feature space dimensionality reduction, the authors chose to select the 3 first-ranked families extracted on Voiced and Unvoiced signals, which are similar with both IG and G+B ranking algorithms: Cepstral (48 features, coefficient 0 is removed for comparison purposes with OpenSmile feature set), Spectral (14 features) and Energy bands (47 features). Since the number of feature per family highly differs, the dimension is reduced again while selecting only voiced features. In order to compare a selection based on acoustic families of features and independent features, 50 [resp. 25] best-ranked features with IG, are selected for each corpus, forming the Li-B50 [resp. Li-B25] feature set. For testing the hypothesis "the system is over-fitting when using too much features", random features sets have been tested on both Li-174 and Os-384 full sets. The authors chose to add random feature sets to have a baseline study [START_REF] Alam | Comparative study of speaker personality traits recognition in conversational and broadcast news speech[END_REF]. The number of feature in random sets is defined by the number of features in selected sets. The different subsets of features are reported in table V. They consist in full sets, selected family and best independent features sets, random sets, selected from Li-174 and Os-384 sets. The robustness of these subsets will be validated in the next section with cross-corpus experiments.

VI. CROSS-CORPUS EXPERIMENTS

In the following experiments, the cross-corpus emotion classification is performed on the combination of features described in the previous section. Two multi-corpus optimization methods are tested on the six sub-corpora with two acoustic feature sets (Li-174 and Os-384) and randomly reduced feature subsets. Auto-coherence and cross-corpus results are presented, underlying the relevancy of Li-CEP24 subset.

A. Cross-corpus protocol

Classification is realized with Support Vector Machine (SVM) and Sequential Minimal Optimization function (SMO) since this configuration has been widely used. The authors use LibSVM tool 7 . To better understand the obtained results and to estimate the generalization power of each of the six subcorpora, the authors add a confidence interval CI. It is defined by the relation 4, where p is the prediction score and N the total number of instances [START_REF] Chollet | Recent Advances in Speech Understanding and Dialog Systems[END_REF].

P = p ± CI = p ± 1.96 p(1 -p) N (4)
In

B. Multi-corpus optimization of SVM parameters

As Huang and Wang pointed out [START_REF] Huang | A ga-based feature selection and parameters optimization for support vector machines[END_REF], "obtaining the optimal feature subset and SVM parameters must occur simultaneously", this section is thus dedicated to optimization of cross-corpus valence recognition. Obtaining the optimal and minimal feature subset and optimizing of SVM parameters for modeling emotion classes are jointly studied.

1) Optimization with the Grid search algorithm (OPT1):

The Grid search algorithm is a method for finding the optimal C and γ when using the RBF kernel function. Although this method is very well-known, it also has disadvantages: it is time-consuming and do not perform well in all situations [START_REF] Huang | A ga-based feature selection and parameters optimization for support vector machines[END_REF]. The Grid search selects the couple of parameters (C; γ) that leads to a maximum of accuracy (accuracy is averaged over n-folds classification tests) for a given training database. However, other local maxima could be eligible if a confidence interval was given. Non-standard values of (C; γ) tend to be reached by the Grid search algorithm when the number of feature is too small (typically less than 50) with both Li-174 and Os-384 full sets. Selected (C; γ) are often non-standard values with ramdomly reduced feature subsets, but it is not the case with appropriate feature family. Such non-standard values usually lead to a serious drop in cross-corpus classification performance.

2) Fixed optimized values (OPT2): In order to ensure SVM parameter stability and to avoid non-standard values of (C; γ), other optimization techniques must be tested. One could use a development database and optimize (C; γ) on the accuracy obtained while testing on this development database. The obtained results with the corpora described in section III are not successful, probably because development data is not close enough to the final tested data. Moreover, using such a technique in real-life applications implies to optimize and train again the model on line at each changing situation. Eyben et al [START_REF] Eyben | The geneva minimalistic acoustic parameter set (gemaps) for voice research and affective computing[END_REF] propose to average the accuracy over different values of complexity C for the results to be more stable. The authors discarded non standard (extreme) values since "performance drops significantly with the smaller feature set". This confirms the fact that optimization with small feature sets usually lowers classification because the selected (C; γ) parameters are often non-standard values. This technique is not adapted in real-life applications either.

The optimization technique (OPT2) proposed in this paper should ensure more stability of the parameters across corpora and avoid non-standard values, thus leading to good performances in cross-corpus. (C; γ) couple is chosen among the optimized values found by the Grid search over a large number of experiments including different feature sets and corpora. Figure 1 shows histograms of optimized values for both Li-174 and Os-384 and 50, 25 randomly selected features -7 × 2 subsets. The most frequent values are (C; γ) = (2; 0.125). Such a technique has the advantage of being easy to use in real-life applications. The fact that randomly selected feature sets have similar performance than full feature set gives a piece of evidence that the system is over-fitting when using too many features. Therefore, an intelligent feature selection should significantly outperform performances obtained with full sets.

C. Auto-coherence results

In this section, the authors use the same corpus for training and testing under auto-coherence experiments. Such an experiment gives a baseline of the most important rate a system could achieve and how this system recognizes the data it has already seen [START_REF] Devillers | Inference of human beings' emotional states from speech in human-robot interactions[END_REF]. Figure 2 shows that the highest the number of features, the better the recognition rates. With few features the auto-coherence is closer to cross-corpus results. In this specific case, optimization of SVM parameters significantly improves the recognition rates. The IG individually best ranked features Li-B50 [resp. Li-B25] gives the best recognition rate at 63.6% [resp. 55.7%] among the reduced feature subsets with almost 50 features [resp. 25 features]. One of the reasons is that the IG individually best features were ranked on each training corpus.

D. Cross-corpus results

Since the authors' goal is to find a small acoustic feature set, the following experiments are realized in cross-corpus conditions, SVM parameters are set to the fixed optimal values obtained with the OPT2 method. Cross-corpus results are reported in figure 3. IG individually best ranked subset of features Li-B50 [resp. Li-B25] gives good recognition rate at 42.0% [resp. 41.6%] among the reduced feature subsets with almost 50 features [resp. 25 features], however improvements from random subsets and full sets are not significant (< 2.5%). IG and G+B best ranked families (Li-EBB47 best ranked with IG and Li-Spec14 best ranked with G+B) do not give interesting improvements from full sets. However such reduced feature sets do not degrade recognition rates. It means that ranking such as Information Gain and Gaussian Models with Bhattacharyya distance is not powerful enough for selecting features in cross-corpus experiments.

Cepstral family which was ranked in second position with IG and fifth position with G+B, gives the best recognition rates (Li-CEP48: 42.8% and Li-CEP24: 43.4%) and significant improvements from full feature sets. The result obtained with the Li-CEP24 set (43.4%) is significantly higher than the results obtained with the full sets Li-174 (40.0%) and Os-384 (40.0%), and with the cepstral reduced Os-CEP24 set (35.2%). In both Li-174 and Os-384, cepstral features are extracted with the same algorithm but the Mel scales slightly differ: OpenSmile (eq. 5) is based on HTK toolkit [START_REF] Young | The HTK Book Version 3.4[END_REF] and Li-174 (eq. 6) is based on Rabiner and Juang [START_REF] Rabiner | Fundamentals of speech recognition[END_REF].

The authors conclude that the implementation used in the Li-174 for extracting cepstral coefficient gives promising results for cross-corpus valence recognition.

With a large number of features, models fit perfectly with the training data; auto-coherence rates are very high and crosscorpus rates are quite low. With a smaller set of features, models tend to generalize; auto-coherence rates degrade while cross-corpus rates may be improved. This result confirms Huang and Wang [START_REF] Huang | A ga-based feature selection and parameters optimization for support vector machines[END_REF] and heighten the relevancy of small feature sets for automatic real-life valence recognition. Of course, this result is pertinent once model parameters are correctly optimized. Similar trends occur when adding the two AIBO sub-corpora, the only difference is that performance rates are lower.

VII. CONCLUSION

The search of a small acoustic feature set for emotion recognition is an important issue. Looking towards such an optimal set faces three main challenges. First of all, such a set must be robust to real-life applications, in other words, the search has to be conducted on real-life corpora. Next, emotion models are evaluated with an optimal set of acoustic features, simultaneously with parameters optimization. Finally, the result of feature selection must be evaluated in cross-corpus condition. The goal of the present study is to select the most consensual acoustic family for valence recognition via dependent (Gaussian Mixture Models and the Bhattacharyya distance) and independent (Information Gain) non-classification based feature ranking and cross-corpus experiments, and to optimize emotional models simultaneously.

Reducing the number of features goes in pair with optimizing model parameters (here SVM parameters). Experiments carried on randomly selected features from two acoustic feature sets (Li-174 and Os-384) show that a feature space reduction is needed to avoid over-fitting. A Grid search tends to find non-standard values with small feature sets. In order to ensure more stability of the SVM parameters, the authors propose a multi-corpus optimization method which consists in finding the most frequent optimized (C; γ) values found on different corpora and acoustic feature subsets. Reducing the number of features implies to down perform the auto-coherence results but this reduction improves crosscorpus recognition rates. The results show that feature ranking and distance between GMMs are interesting methods to select acoustic families taking into account the mutual dependency of the features within the family. However, best individual feature or best family selected with the Information Gain method (Energy bands family) or best family selected with the G+B method (Spectral family) do not yield relevant results in cross-corpus experiments. Promising results have been obtained with a reduced set of 24 Voiced Cepstral coefficients while this family was ranked in 2nd (with IG) and 4th (with G+B) positions with both ranking methods. These results outperform the results obtained with the standard OpenSmile.

The present paper emphasizes many perspectives and further works. The results presented here show some interesting trends, that could be validated on larger real-life emotional corpora. Nonetheless, collection of more recorded spontaneous data is still a challenge in the emotion recognition field. In addition, further work is needed in selecting features (from different acoustic sets), improving modeling techniques and feature extraction and optimizing the parameters of the models simultaneously in the field of affective computing, especially for real-time recognition applications in the wild.

4 ) 2 ; (n + 1 ) • F 0 2 with n from 1 to 5 [ 5 )

 42155 slope s is defined according to the linear regression of the spectrum: a(f ) = s • f + c te for f ∈ [0; f e /2]. Energy bands features (EBB): are recently used in emotion recognition. Harmonic bands correspond to the spectral energy in the band n • F 0 69]. Bark bands correspond to the spectral energy in the first 21 Bark bands [64]. Bark scale is interesting because it relies on human perception. Harmonic bands are interesting because they focus on the spectral part of most energy. Mel Cepstral coefficients or MFCC (CEP): and derivative cepstral coefficients are usual features for emotion recognition. The Mel-cepstrum is the Discrete Fourier Transform of the logarithm of the Melbands spectrum. The authors use the following Mel scale: M (f ) = 1000 1 + log 10 f 1000 for f > 1000 Hz, M (f ) = f for f < 1000 Hz [70]. They are known to be robust to noisy signal. Many recent studies show their interest

Figure 1 .

 1 Figure 1. Histograms of C and γ values obtained with Grid search on Os-384 (weight:3), Os-R50 (3 random sets), Os-R25 (3 random sets) and Li-174 (weight: 3), Li-R50 (3 random sets), Li-25 (3 random sets) over the six subcorpora.

Table IV FAMILY

 IV RANKING OVER THE SIX SUB-CORPORA AND ALL MERGED CORPORA USING IG AND G+B WITH LI-174 SET.

		Corpus	NAO-HR	IDV-HS	IDV-HR	JEMO	AIBO-O	AIBO-M	Merged
		(#features)			Relative Information Gain IG r
		F 0 (4)	7.1	2.2	4.4	9.5	2.5	2.5	5.2
		Loudness (12)	14.2	10.5	13.1	10.8	12.3	11.1	12.5
		Spectral (14)	5.5	10.9	7.5	12.5	11.2	10.7	12.4
		Cepstral (78)	16.4	31.2	18.3	31.1	32.9	29.9	24.7
		EBB (47)	55.1	37.9	52.0	30.1	34.7	39.9	39.1
		Formant (14)	< 0.001	4.1	1.4	2.3	2.1	2.8	2.0
		VQ (5)	1.8	3.1	3.3	3.8	4.4	3.2	4.0
		Voiced (81)	63.1	46.0	72.8	57.8	69.1	74.5	70.3
		Unvoiced (58)	20.3	33.6	13.2	15.9	17.1	12.3	9.9
		All (35)	16.6	20.5	14.0	26.3	13.8	13.3	19.8
				Relative Bhattacharyya distance d r B,GM M
		F 0 (4)	6.8	4.1	6.1	6.7	-2.8	-2.8	8.3
		Loudness (12)	16.9	17.5	17.3	15.7	22.5	20.4	16.1
		Spectral (14)	19.7	23.4	21.0	23.4	31.6	34.7	26.7
		Cepstral (78)	19.3	19.7	19.7	18.9	26.2	25.4	14.5
		EBB (47)	19.4	20.1	19.6	18.7	26.0	25.0	16.8
		Formant (14)	12.5	11.6	10.9	10.7	-0.9	-0.8	14.9
		VQ (5)	5.5	3.7	5.5	6.0	-2.6	-2.0	2.7
		Voiced (81)	40.5	43.9	43.1	38.2	35.9	17.6	40.6
		Unvoiced (58)	49.1	45.0	47.9	40.6	59.7	76.1	42.0
		All (35)	10.4	11.0	9.0	21.2	4.3	6.2	17.5
					Table V
			DESCRIPTION OF THE FEATURE SUBSETS
	Name	Origin	Description
	Os-384	OpenSmile	Full OpenSmile set
	Os-R50		Mean random combination of 50 features (3 sets)
	Os-R25		Mean random combination of 25 features (3 sets)
	Os-CEP24		24 OpenSmile Cepstral amean coefficients (1-12 and derivatives) on voiced
	Li-174	Limsi	Full Limsi set
	Li-R50		Mean random combination of 50 features (3 sets)
	Li-R25		Mean random combination of 25 features (3 sets)
	Li-B50		Best 50 features obtained on the training corpus with InfoGain individual ranking (6 sets)
	Li-B25		Best 25 features obtained on the training corpus with InfoGain individual ranking (6 sets)
	Li-EBB47		47 Energy Bands (Bark and Harmonic bands) on voiced and unvoiced
	Li-CEP48		48 Cepstral coefficients (1-12 and derivatives) on voiced and unvoiced
	Li-EBB26		26 Energy bands on voiced
	Li-CEP24		24 Cepstral coefficients (1-12 and derivatives) on voiced
	Li-SPE14		14 Spectral features on voiced and unvoiced

  Table VI summarizes results obtained in cross-validation (10 folds, training and testing are realized with the same corpus) and cross-corpus conditions (training and testing are realized with different databases) with different subsets of features and the two SVM parameters optimization techniques described previously. For each subset of feature and optimization technique, 4 × 3 cross-corpus tests and 4 cross-validation tests are run. Average, maximum and minimum values are reported in table VI. All the trends described in this section are similar with AIBO, but performances are lower. Figure 2. Auto-coherence results on different acoustic sets without the two AIBO sub-corpora. CI = ±2.3%. Results are given in terms of mean UA (%) over the corpora. OPT1 -Os-384 gives 43.7% [resp. Li-174 gives 41.2%] -than with OPT2 -Os-384 gives 40.0% [resp. Li-174 gives 40.0%]. XC accuracy degrades with OPT1 while reducing the number of features, whereas it upgrades with OPT2 with both feature sets. OPT1 does not perform as well as OPT2 when the set of features is reduced especially with the Limsi set, for the reasons already mentioned in section VI-B1. With Os-384 [resp. Li-174], cross-corpus performance with 25 randomly selected features and OPT2 is 42.6% [resp. 40.9%] are almost similar to the result obtained with full set and OPT1: 43.7% [resp. 41.2%]. With an appropriate optimization technique (here the OPT2), a random selection of 50 and 25 features performs almost as well as the full sets.

	Table VI COMPARISON BETWEEN TWO SVM OPTIMIZATION TECHNIQUES. CROSS-VALIDATION (CV) AND CROSS-CORPUS (XC) ACCURACY IN % (WITHOUT AIBO). CI = ±2.5% OPT1 OPT2 Set CV XC CV XC avg avg min max avg avg min max between emotion classes -the results are presented with the four French corpora only. Os-384 Li-174 Li-B50 Li-EBB47 Li-CEP48 Li-B25 Li-EBB26 Li-CEP24 Li-SEP14 Os-CEP24 40 50 60 70 80 90 100 92.7 86.2 63.7 58.3 62.6 55.7 55.0 53.6 41.9 Auto-coherence mean UA (%). Combination of features results. 55.0 Figure 3. Cross-Corpus results on different acoustic sets without the two AIBO sub-corpora. CI = ±2.5%. Results are given in terms of mean UA (%) over the corpora. 38 42 44 Cross-corpus mean UA (%). Combination of features results. 42.0 42.8 41.6 43.4 40 40.0 40.7 40.2 39.9 40.0
	36
				35.2
	34
	Os-384	Li-174	Li-B50 Li-EBB47 Li-CEP48	Li-B25 Li-EBB26 Li-CEP24 Li-SEP14 Os-CEP24
	Cross-validation (CV) performances are slightly better with
	OPT1 optimization than with OPT2 (differences < 2.5%). For
	instance with Os-384 [resp. Li-174], OPT1 gives a CV rate of
	57.0% [resp. 58.4%], while OPT2 gives a CV rate of 57.1%
	[resp. 57.7%]. Cross-corpus (XC) performances are better
	on full sets with

http://emotion-research.net

http://opensmile.sourceforge.net/

http://yaafe.sourceforge.net/

http://www.praat.org

http://www.cs.waikato.ac.nz/ml/weka

https://www.csie.ntu.edu.tw/~cjlin/libsvm/

ACKNOWLEDGEMENT

This work was partially financed by the French projects: FUI ROMEO and BPI ROMEO2. The authors thank coders and co-workers who participated in elaborating protocols and annotating emotional states.


Marie Tahon
email: marie.tahon@limsi.fr

Agnes Delaborde
email: agnes.delaborde@limsi.fr

Laurence Devillers
email: laurence.devil@limsi.fr

Real-life Emotion Detection from Speech in Human-Robot Interaction: Experiments across Diverse Corpora with Child and Adult Voices

Keywords: emotion detection, human-robot interaction, cross-corpus, realistic corpus

We focus in this paper on the detection of the emotions in the voice of a speaker in a Human-Robot Interaction context. This work is part of the ROMEO project, which aims to design a robot for both elderly people and children. Our system offers several modules based on a multi-level processing of the audio cues. The affective markers produced by these different modules will allow to pilot the emotional behaviour of the robot. Since the models are built with recording data and the system will test real-life data, we need to estimate our emotion detection system performances in cross-corpus situations. Cross-validation experiments on a three class detection show that derivatives and energy features may be removed from our feature set for this specific task. Cross-corpora experiments on anger-positive-neutral data suggest that detection performances may be better with two different models: one for child voices, one for adult voices.

Introduction

Our study focus is Human-Robot Interaction. We are involved in the French ROMEO project 1 , which aims to design a 1m40 high social humanoid robot which will be able to assist elderly and disabled persons at home in everyday life activities, but also which will be able to play games with children (for example with the grand-children of the user). So as to interact as naturally as possible with the user, the robot will be endowed with an emotional system. It will allow it to be able to adapt its behaviour according to its own emotional state, and will be sensitive to the user's emotional state as well. The robot will evolve in real-life conditions and then face a rich multimodal contextual environment which needs to be processed. In the community, robotic emotional systems are endowed with a processing of visual, tactile or audio inputs. Non-verbal cues are useful at different levels: in maintaining the communication (backchannel), in the comprehension of the message (signs of agreement/disagreement) but also in an interpersonal affective dimension: in the short term (positive/negative emotion) or longer term (affective dispositions). We can refer to works related to WE-3RV by Miwa et al. [START_REF] Miwa | Introduction of the Need Model for Humanoid Robots to Generate Active Behavior[END_REF] dealing on visual and audio inputs (low or loud sounds), studies on iCat focused on the facial expressions of the participant [START_REF] Castellano | Affect recognition for interactive companions: challenges and design in real world scenarios[END_REF] or Kismet's color, face and motion detection [START_REF] Breazeal | Robot in Society: Friend or Appliance?[END_REF]. Nonetheless, emotions carried by speech are seldom used in Human-Robot Interaction (HRI).

We focus on designing an audio detection system for a HRI interface, which will process the audio cues at different levels. The system is based on several modules, each focusing on different specific affective measures: emotion label, activation, valence, speaking rate, presence of affect bursts, etc. These different affective markers will be used for driving the behaviour of the robot.

In the framework of the final application, the HRI system will be supplied with both adult and child voices. In order to train our models, we need data related to our final applicative context, featuring children and adults. Our system is supposed to detect emotions on an audio file recorded in unknown acoustic conditions. In order to estimate the robustness of our emotion detection system to different recording conditions, we have done several cross-corpora experiments. Another method to improve robustness to different recording conditions would be to adapt our models during the interaction, but this aspect will not be treated in this paper. In the community, there are few available realistic HRI corpora, the best known being the AIBO corpus, which is a collection of 51 children interacting with Sony's pet robot Aibo in a specific context [START_REF] Batliner | You stupid tin box" -children interacting with the AIBO robot: A cross-linguistic emotional speech corpus[END_REF]. In our context, we try to use this corpus for building models for children and we also collected new data corresponding more precisely to our applicative context. The focus of the paper is to study the performances of the anger-positive-neutral model we can build on the databases at our disposal. A next step will be to build all the models to give predictions which can be used in the HRI emotion detection system (active/passive, positive/negative, anger against the remaining emotions, etc.).

The normal approach in emotion detection from speech is to subdivide one corpus in two sets: one for training the model, and the other for testing. When using only one corpus, most variables are constant: microphone, room acoustics, sampling frequency, speaker group, annotations, etc. In the case of realistic corpora, variables vary much more than in acted or prototypical corpora [START_REF] Schuller | Cross-corpus acoustic emotion recognition: variances and strategies[END_REF]. Then, doing cross-corpora using realistic corpora is challenging. One of the first studies on cross-corpora classification of realistic emotions [START_REF] Eyben | Cross-corpus classification of realistic emotions -some pilot experiments[END_REF] shows that significant improvement over random guess is observed in a few cases for valence classification. Normalisation in the context of cross-corpus evaluation is an important challenge, as well as canceling the recording conditions effects. We have tested the normalisation to speaker; as [START_REF] Schuller | Cross-corpus acoustic emotion recognition: variances and strategies[END_REF] shows it was the better. In the context of the ROMEO project, we wonder if one single model for both children and adults is enough for emotion detection, or if we need to have two distinct models. A first approach is to test performances in cross-corpus classification: training on the adults and testing on children, or the contrary. A second method that will not be studied here, would be to test leave-one-speaker-out on both corpora Section 2 deals with the final emotion detection system we intend to build for the HRI. Section 3 presents the new corpora we collected in the final application context. Section 4 summarises first experiments on performances of the three corpora: NAO-HR, IDV-HR, and AIBO, then experiments on cross-corpora classification.

Multi-level Processing of Emotional Audio Cues in Human-Robot Interaction

We presented in a previous study on the emotional and interactional markers [START_REF] Delaborde | Use of Nonverbal Speech Cues in Social Interaction between Human and Robot: Emotional and Interactional markers[END_REF] a modeling of the emotional social interaction between a Human and a Robot. We argue that a multi-level use of audio non-verbal cues contributes to an efficient piloting of the decisions of the robot. Low level cues can be computed from the speech signal [START_REF] Devillers | Challenges in real-life emotion annotation and machine learning based detection[END_REF]: duration of speaker turns, F0, energy, and other acoustic coefficients such as MFCC, etc. Multi-level markers can be derived from these cues and provide a system with emotional information such as positive/negative emotion, activation/non activation behaviour, emotion labels (Joy, Sadness, Fear, Anger), speech delivery, rhythm and duration. On a higher level of analysis, these data can be processed so as to get cues about the emotional and interactional tendencies of the speaker: we can obtain emotional and interactional markers such as ill-at-ease, talkative, shy, or dominant. This multi-level processing is presented in Figure 1. A speaker identification system would also bring sociological metadata such as the age bracket of the speaker, the sex, and to be able to recognise a specific user and then keep an automatic track of his or her emotional and interactional profile.

The detection of the expressed emotions is then organised in two levels: the first level is the single speaker turn, when the emotion is immediately processed, and the other level requires the use of a history gathering the markers of the emotions expressed by the speaker, after several speaker turns and also the use of a history of the reactions of the robot. This emotional and interactional profile will be a basis for the selection of the most desirable behaviour of the robot towards the user, depending on the context of interaction. In this study, we look into the performances of short-term emotion detection.

Data Collection

A first data collection, NAO-HR [START_REF] Delaborde | Use of Nonverbal Speech Cues in Social Interaction between Human and Robot: Emotional and Interactional markers[END_REF], features children playing games with the humanoid robot Nao, and a second study concerns visually-impaired elderly people in the context of an interaction with Nao as a robotic domestic assistant (IDV-HR). In both experiments, different communicational strategies of the robot were applied so as to induce different emotional reactions in the participants.

Recording Protocol

In NAO-HR, each child carries a high quality lapelmicrophone (AKG PT40 Pro Flexx) which records the totality of the session.The sampling frequency is 16kHz. The video recording of the children is used for potential verifications, and as a showcase for our studies. Each participant of the IDV-HR data collection is offered to sit comfortably in front of NAO, which is sitting down on a coffee table. The participant is recorded with the same lapel-microphone. The sampling frequency is 44kHz. A camera is placed behind the robot and films the upper part of the body of the speaker for further studies.

Wizard of Oz

Scenarii with Children

We designed Wizard-of-Oz systems which allow us to gather spontaneous emotional data, in task-related contexts. In NAO-HR, pairs of two children aged between eight and thirteen played games with the robot. In a first game, both the children and the robot play a question-answer game, arbitrated by a human game master (experimenter). The second and third game present the robot as capable of recognizing emotions and songs, and each child has to act emotions or hum songs in such a way as to be recognized by the robot. The communicational strategies applied by the robot in the course of these games are divided into positive (desirable) strategies and negatives (undesirable) ones: depending on the moment of the experiment, the robot encourages or congratulates the children, as well as triggers competition or presents some technical failures (repetitions, crashes). An experimenter controls the robot from and adjacent room and selects the behaviours which are to be played.

Scenarii with Visually Impaired People

The corpus IDV-HR features elderly people interacting with the robot. The speaker is asked to play three sessions of five scenarios in which he pictures himself in a situation of waking up in the morning. The robot would come to him to chat about either his health, or the program of the day, etc. The utterances of the robot are spoken through a Speech-to-Text module, and are based on pre-established and fixed sentences. Each of these five scenarios is devoted to a different affective state, which the speaker is asked by the robot to act: wellbeing, minor illness, depressed, medical distress, happy. Each series of five scenarios differ from the other, by the social attitude of the robot (positive: friendly, empathetic, encouraging, or negative: directive, doubtful, machinelike).The robot is remotely controlled by an experimenter who selects the different social attitudes and the utterances which match the content of the speaker's speech the best.

Annotation Protocol

On each speaker's track, we define segment boundaries. A segment is emotionally homogenous, i.e. the emotion is considered as being the same and of a constant intensity along the segment [START_REF] Devillers | Coding Emotional Events in Audiovisual Corpora[END_REF]. Two expert annotators perceptively annotated (as far as possible they did not take into account the semantic content) the segment along this annotation scheme: • Three affective state labels describe each emotional segment. The resulting annotation of a segment can then represent complex emotions (such as both positive and negative expressions). The affective state labels are grouped together in five macro-classes: positive, anger, sadness, fear, neutral.

•

Valence: On the whole, does the speaker feel a positive or a negative sensation? positive; negative; either positive or negative; positive and negative; valence non decidable • Activation: the strength of the expressed emotion. Scale of -2 to 2, from very weak to very intense

The annotation scheme also includes interactional dimensions for other studies relative to the HRI.

Description of the Corpora

The NAO-HR corpus is made up of 1287 emotional segments, for a total amount of 31'07''. Ten children were recorded (five males and five females from eight to thirteen). 22 speakers were recorded in the framework of IDV-HR (11 males and 11 females for a median age of 59). So far, 8 speaker sessions were emotionally labelled, for an amount of 2198 emotional segments (1h 20'). Table I presents the interspeaker agreement scores for the annotations of the macroclasses for both corpora: the Kappa values are computed first on all the macro-classes. However, the NAO-HR corpus offers only few instances of fear and sadness. Moreover, so as to allow the comparison between our data and the corpus AIBO, we need to restrict our study subcorpus to the macro-classes anger, positive and neutral (as explained in 4.1). We present the Kappa values for these three annotation macro-classes. In the experiments described in this paper, we only keep consensual instances.

Cross-corpus Experiments

Performance comparison for several Human-Robot Interaction audio corpora

All the emotion detection features in this study are based on the OpenEar Interspeech 2009 Challenge [START_REF] Schuller | The INTERSPEECH 2009 emotion challenge[END_REF]. As our final emotion detection system is going to be an embedded system, we would like first to reduce the number of acoustic features for the specific task studied in this paper. We have chosen the AIBO corpus to be able to compare the results obtained on IDV-HR and NAO-HR with a well-known reference. We have separated the two schools of the AIBO corpus in order to have the same recording conditions. AIBO corpus contains Anger, Motherese, Empathy and Neutral. In order to compare AIBO, IDV-HR and NAO-HR together, we are going to use the macro-classes: anger, positive and neutral. Table II summarises the number of instances used in all the experiments. As our sets of instances are nearly balanced, we will report only the Unweighted Average Recall percentage (average percentage of correctly detected instances per class). III shows the cross-validation (10 folds) performances for the four corpora. The confidence ratio is estimated on the basis of 384 features without any normalisation. Optimisation of parameters and classification are done with libSVM tool [START_REF] Chang | LIBSVM : a library for support vector machines[END_REF]. We have tested three different features sets: the first one contains the 384 features of the Interspeech 2009 challenge, the second contains only the 192 basic features (no derivatives) and the third one contains the basic features after having removed energy features (180 features). We have noticed in previous studies that energy features are important for activation detection, but removing them does not make the results fall down on our specific task. We have also tested the performance with normalisation to speaker called NS (z-norm: for each acoustic features, median and standard deviation are computed for one speaker only) compared to no normalisation (NO). As we can see in Table III, performances are quite the same for the three sets of features. As we would like to have the smallest number of features in our final ROMEO application, we will use only basic features (without derivatives) without energy features. 3.34 As our NAO-HR corpus is a small corpus, the results are probably biased: high number of features for small number of instances. Normalisation to speaker (NS) allows us to improve significantly the performances on AIBO-Ohm on crossvalidation test. Normalisation to speaker also improves performances on NAO-HR but the gain remains in the confidence interval.

Experiments on cross corpora emotion detection

The final emotion detection system will be able to recognise emotions expressed by adults and children. In order to improve this detection, we would like to know if we can mix both NAO-HR and IDV-HR corpora, or if we need to have two different training sets. The following experiments are made with the 180 features set and libSVM tool for optimisation of parameters and classification. All features have been normalized using normalisation to speaker. We have made different cross-corpus tests between NAO-HR, IDV-HR and AIBO-Mont and AIBO-Ohm. The results reported in Table IV are the UAR performances (and confidence interval) of cross-corpus tests. For example: training on AIBO-Mont, testing on AIBO-Ohm performs 50.20% UAR. A first result is that cross-corpus between AIBO-Mont and AIBO-Ohm is high performing: both corpora are in German, they have the same task, annotation protocol is similar and speakers belong to the same age group. Training on AIBO (both schools) and testing on NAO-HR is better than testing on IDV-HR. NAO-HR and IDV-HR have nearly the same annotation protocol, are both in French, but speakers do not belong to the same age group. We can notice similar trends when training on NAO-HR: tests on AIBO (34.68% and 38.75%) have better performances than tests on IDV-HR (29.02%). Generally speaking, every cross-corpus experiments with IDV-HR are below the random guess (33%). Our conclusion is that we can not mix together IDV-HR and NAO-HR, but it seems feasible to mix together AIBO and NAO-HR. 25.05 (7.48) As we see in section 4.1, IDV-HR presents relatively poor performances on cross-validation. Our hypothesis is that this corpus corresponds to a very specific HRI situation, with visually impaired people whose age varies from 28 to 80; expressed emotions are quite shaded in IDV-HR, contrary to NAO-HR. Therefore, we will probably need a specific model for this particular corpus. Some instances in NAO-HR and IDV-HR are very short (less than 1s). On such durations, the pitch, spectrum and voiced part estimation is not absolutely reliable. It can introduce bias in emotion detection and explain the differences of UAR performances between AIBO (minimum duration is 1s) and NAO-HR and IDV-HR.

Figure 2. Distribution of distances between anger

instances and all the instances Looking at the confusion matrices, we can notice that the anger class is the one which is the worst recognised. This is probably due to the fact that the tasks are different from one corpus to another, and then, anger has a different meaning across corpora. One method to show this point is to compute the distance between anger instances and all the instances [START_REF] Tahon | Acoustic measures characterizing anger across corpora collected in artificial or natural context[END_REF]. Figure 2 shows the distribution of the distances. We can see that anger has quite the same repartition for AIBO and NAO-HR corpora and is very different from IDV-HR anger. This could explain the relatively poor performances in cross-corpus combining IDV-HR with AIBO or NAO-HR.

Conclusion

We focus in this paper on an anger-positive-neutral detection module of an audio HRI emotion detection system, based on a multi-level processing of the audio cues. Due to the real-life condition of the test, we carried out different experiments on emotion detection on cross-corpora.

Our first experiments show that derivatives and energy may be removed of the feature set for our specific angerpositive-neutral detection task. As we have a smaller set of features, the real-time system will probably be faster. The second important point is that normalisation to speaker seems to improve cross-validation performances, which confirms previous studies [START_REF] Schuller | Cross-corpus acoustic emotion recognition: variances and strategies[END_REF]. Our last result is that see that it is possible to mix two audio realistic corpora recorded with children in HRI, but it seems more complex to mix a corpus with adult speakers and a corpus with children speakers. Therefore, we will suppose that two different models would lead to better performances.

Our results on real-life corpora must be validated on larger amount of data. In order to build the final embodied models, we will study the influence of instances duration, other normalisations and last but not least, the real-time audio segmentation. Further studies will need to be carried out to develop the other modules which will supply our HRI system with data: emotion detection, activation detection, speaking rate, affect bursts detection, etc. We will notably study the influence of instances duration on emotion detection.

Figure 1 :

 1 Figure 1: Multi-level detection of the emotional and interactional cues from speech (from [7])

Table I .

 I Annotation agreement scores for the two corpora

	Corpus	# Coders	Kappa values	Macro-classes
	NAO-HR	2	0.4	positive, anger, fear, sadness, neutral
	NAO-HR	2	0.7	positive, anger, neutral
	IDV-HR	2	0.7	positive, anger, fear, sadness, neutral
	IDV-HR	2	0.8	positive, anger, neutral

Table II .

 II The number of instances used in the experiments

	#instances	Neutral	Positive	Anger
	IDV-HR	282	267	282
	NAO-HR	51	51	27
	AIBO (Mont)	372	372	324
	AIBO (Ohm)	501	501	447
	Our first experiment was to reduce the number of acoustic
	features for the specific task: neutral, positive and anger
	recognition. Table

Table III .

 III Cross

		-validation performances (UAR, %) for
	different sets of features and normalisation (NO: no
	normalisation, NS: normalisation to speaker)
	#features	384	192	180	180	Confi
	Normalisation	NO	NO	NO	NS	dence
	AIBO-Mont	62.83	61.22	61.40 59.84	2.90
	AIBO-Ohm	37.18	37.47	37.19 63.20	2.49
	NAO-HR	52.65	61.65	52.29 57.44	8.62
	IDV-HR	40.71	39.09	39.61 41.95

Table IV .

 IV Performances in cross-corpus classification, column for test, line for train

		AIBO-	AIBO-	NAO-HR	IDV-HR
		Mont	Ohm
	AIBO-		50.20	40.25	33.24
	Mont		(2.37)	(8.47)	(3.20)
	AIBO-	53.18		47.06	31.29
	Ohm	(2.99)		(8.61)	(3.15)
	NAO-HR	38.75	34.68		29.02
		(2.92)	(2.45)		(3.08)
	IDV-HR	30.30	30.29
		(2.77)	(2.36)

http://www.projetromeo.com

Acknowledgements

This work is financed by national funds FUI6 under the French ROMEO project labeled by CAP DIGITAL competitive centre (Paris Region).