Marie Tahon

Agnès Delaborde

Claude Barras

Laurence Devillers

A corpus for identification of speakers and their emotions

This paper deals with a new corpus, called corpus IDV for "Institut De la Vision", collected within the framework of the project ROMEO (Cap Digital French national project founded by FUI6). The aim of the project is to construct a robot assistant for dependent person (blind, elderly person). Two of the robot functionalities are speaker identification and emotion detection. In order to train our detection system, we have collected a corpus with blind and half-blind person from 23 to 79 years old in situations close to the final application of the robot assistant. This paper explains how the corpus has been collected and shows first results on speaker identification.

Introduction

The aim of the project ROMEO (Cap Digital French national project founded by FUI6, http://www.projetromeo.com) is to design a robotic companion (1m40) which can play different roles: a robot assistant for dependent person (blind, elderly person) and a game companion for children. The functionalities that we aim to develop are speaker identification (one speaker among N, impostor) and emotion detection in every day speech. The main challenge is to develop a strong speaker detection system with emotional speech and an emotion detection system knowing the speaker. All our systems are supposed to be real time systems.

In the final demonstration, the robot assistant will have to execute some tasks as defined in a detailed scenario. The robot is in an apartment with its owner, an elderly and blind person. During the whole day, the owner will have some visitors. The robot will have to recognize who are the different characters: his little children (two girls and a boy), the doctor, the house-keeper and an unknown person. In the scenario the robot will also have to recognize emotions. For example, Romeo would be able to detect how the owner feels when he wakes up (positive or negative) and to detect anger in the little girl's voice.

To improve our detection systems (speaker and emotion)

we need different corpora, the closer to final demonstration they are, the better the results will be. We focused on blind or half-blind speakers (elderly and young person) and children voices while they interact with a robot [START_REF] Delaborde | A Wizard-of-Oz game for collecting emotional audio data in a children-robot interaction[END_REF] in order to have real-life conditions. However, emotions in real-life conditions are complex and the different factors involved in the emergence of an emotional manifestation are strongly linked together [START_REF] Scherer | Vocal communication of emotion: a review of research paradigms[END_REF].

In this paper, we will describe the IDV corpus which was collected with blind and half-blind person: acquisition protocol, scenarii involved. Then we explain the annotation protocol. And in section 4, we give our first results on speaker identification (identify a speaker from a set of known speakers).

IDV corpus

The part of the final scenario that concerns IDV corpus, we aim to demonstrate at the end of the project consists in:

identify a speaker from a set of known speakers (children or adults), recognize a speaker as unknown and in this case, provide its category (children, adult, elderly) and gender (for adults only), and detect positive or negative emotion.

Speaker identification and emotion detection are real time tasks. For that objective, we have collected a first corpus called IDV corpus with blind and half-blind French people from 23 to 79 years old. This corpus has been collected without any robot but a Wizard-of-oZ which simulates an emotion detection system. This corpus is not fully recorded yet; further records are scheduled with the IDV. A second corpus will be collected in the context of the scenario: at the IDV (Institut de la Vision in Paris) with the robot ROMEO.

Corpus characteristics

So far, we recorded 10h48' of French emotional speech.

28 speakers (11 males and 17 females) were recorded with a lapel-microphone at 48kHz. In accordance with the Romeo Project target room, the recordings took place in an almost empty studio (apart from some basic pieces of furniture), which implies a high reverberation time.

The originality of this corpus lies in the selection of speakers: for a same scientifically controlled recording protocol, we can compare both young voices (from 20 years old) to voices of older person (so far, the oldest in this corpus is 89).

Acquisition protocol

Before the recording starts, the participant is asked some profile data (sex, location, age, type of visual deficiency, occupation and marital status). An experimenter from the LIMSI interviews the volunteer following three sequences described below in 2.3. Some parasite noise happened to be audible in the studio (guide dog walking around, people working outside, talking, moving in the corridor, …). When overlapping the speaker's speech, these parts were discarded.

Sequences description

Each recording is divided into three sequences.

The first one is an introduction to the Romeo project: we explain the participant that we need him to provide us with emotional data, so that we can improve our emotion detection system in a future robot. We take advantage of this sequence to calibrate the participant's microphone.

Since there is no experimental control over the emotions that could be expressed by the participant, this part is discarded in the final corpus and will not be annotated.

In the second sequence, called "words repetition" (table 1), the experimenter asks the participant to repeat after him orders that could be given to the robot. The participant is free to choose the intonation and the expression of his or her production. This sequence gives us a sub-corpus where lexicon is determined and emotions mainly neutral.

Viens par ici! (come here!) Mets le plat au four! (put the dish in the oven!) Arrête-toi! (stop here!) Descends la poubelle! (Bring down the bin!) Stop! (stop!)

Va chercher le courrier! (Bring back the mails!) Ecoute-moi! (listen to me!) Va chercher à boire! (Bring back some water!) Approche! (come near!)

Aide-moi à me lever! (help me to get up!) Va-t-en! (go away!)

Aide-moi à marcher! (help me to walk!) Donne! (give it!) Roméo, réveille-toi! (Romeo, wake up!) Ramasse ça! (pick it up!)

Table 1: List of words and expressions in French

In the third sequence, called "scenarii", the experimenter presents six scenarii (see table Scenarii) in which the participant has to pretend to be interacting with a domestic robot called Romeo. For each presented scenario, the experimenter asks the participant to act a specific emotion linked to the context of the scenario :

for instance Joy, "Your children come to see you and you appreciate that, tell the robot that everything is fine for you and you don't need its help", or Stress, "You stand up from your armchair and hit your head in the window, ask Romeo to come for help", or Sadness, "You wake up and the robot comes to ask about your health. You explain it that you're depressed". The participant has to picture himself or herself in this context and to speak in a way that the emotions are easily recognizable. He or her knows that the lexicon he uses is not taken into account;

the emotion has to be heard in his or her voice.

At the end of each of his or her performance, the experimenter runs a Wizard-of-Oz emotion detection tool, that tells aloud the recognized emotion.

Corpus annotations

Emotion labels

Segmentation and annotation of the data are done with the Transcriber annotation tool1 on the scenario sequences.

The participant utterances are split into emotional segments. These segments mark the boundary of the emotion: when a specific emotion expression starts, and when it comes to an end.

On each segment, three labels describe the emotion. The first label corresponds to the most salient perceived

IDV emotional content

As the emotional annotation of the IDV corpus is not finished yet, all results on emotion annotation are based on a set of 15 speakers.

IDV corpus is divided in two different corpora: spontaneous and acted, according to the task (as defined in part 3). The results of the emotion scores are reported in table 4.

The spontaneous corpus contains 736 instances of 0.5s to 5s. The most important emotional label is "interest" (51%). This corresponds to the agreement of the volunteer with what the interviewer asked him to do.

Positive emotions (18%) are more numerous than negative emotions (6%). The volunteer has accepted to be recorded, so he is not supposed to express displeasure, he will more probably be nice with the LIMSI team.

Macro-class "fear" is also quite important (10%). It corresponds to embarrassment or anxiety, playing the actor is not an easy task.

The acted corpus contains 866 instances of 0.5s to 6s.

The results corresponds to what was expected: the main emotions are well represented. Positive emotion (21%, mainly "satisfaction"), negative emotion (24%, mainly "irritation"), fear (10%, mainly anxiety) and sadness (8%, "deception" and "sadness").

Label emotion

IDV first results

In this section, speaker identification scores are presented. All the results presented here were obtained with the same method based on GMM (Gaussian Mixture Models) speaker models [START_REF] Reynolds | Speaker verification using adapted Gaussian mixture models[END_REF].

First we have studied the different parameters of the GMM model, then the evolution of scores in function of the sex and the age of speakers.

Global speaker identification scores

This section aims at choosing the experimental setup for

Age influence

In this part, we show that speaker identification is easier A female voice is recognized as well at 96%, a male voice is recognized as well at 82%. Female voices have better identification scores.

Sex influence

Emotional speech influence

The results below are based on the corpus "repeating words" which contains 28 speakers. The results presented in this part are based on both sequences "repeating words" and "scenario", with the 15 speakers corresponding to the emotional annotation of the sequence "scenario".

Conclusion

This corpus IDV is interesting for many reasons. 6.

  studying the influence of the age, gender and emotional expression. Experiments are performed with the "repeating words" sequence of the corpus. It contains 458 audio segments of varied duration. 26-dimensional acoustic features (13 MFCC and their first-order temporal derivatives) are extracted from the signal every 10ms using a 30ms analysis window. For each speaker, a training set is constructed by the concatenation of segments up to a requested duration Ntrain; a Gaussian mixture model (GMM) with diagonal covariance matrices is then trained on this data through maximum likelihood estimation with 5 EM iterations. The remaining segments, truncated to a Ntest duration, are used for the tests. For a given duration, the number of available segments is limited by the number of segments already used for training and the minimal test duration necessary (the higher duration is, the less audio files there are). For each test segment, the most likely speaker is selected according to the likelihood of the speaker models. In order to optimize the number of files of train and test, we have chosen the following set of parameters: -test duration: 1s (225 files), -train duration: 10s (179 files), -speaker model: mixture of 6 Gaussians.The error rate is 34.7% (+/-6.5%) when recognizing one speaker among 28.This extremely short test segment duration is due to constraints on segment counts in the database, and improvement of the performance as a function of the segment length will be studied later in the course of the project.

  on elderly person voices than on young voices. Two subcorpora from IDV corpus composed of the 8 older volunteers (4 male, 4 female, from 52 to 79 years old), respectively the 8 younger volunteers (4 male, 4 female, from 23 to 46 years old) are studied separately. Of course, the number of segments is quite low, which may be a bias of the experiment.

  number of segment and trust interval As a result speaker identification (one speaker among N) is better with elderly person voices. Our hypothesis is that voice qualities are much more different with elderly person voices than with young voices. In figure 1, we have plotted the MFCC2 gaussian model for the first four older person (blue) and for the first four younger person (red). As the red curves are quite the same, the blue one are more separated one from another.

Figure 1 :

 1 Figure 1: Distribution of the 4 th MFCC coefficient according to a gaussian model for old (blue, plain) and young speaker (red, dashed)

Figure 2 :

 2 Figure 2 : Confusion matrix between male (1) and female (2) Based on the whole IDV corpus, we compute the confusion matrix sorted by sex without taking into account the age of the speakers anymore.

  three corporaIdentification scores are better with the "words" corpus (lexically controlled) than with the "acted" corpus. The "spontaneous" corpus gives intermediate results. The scores are always better when the train and the test are made on the same corpus.Speaker models were tested directly in mismatch conditions without any specific adaptation. The very high error rates observed are of course due to the very short train and test durations constraints in our experiments, but also highlight the necessity of an adaptation of the speaker models to the emotional context which will be explored during the ROMEO project.

  First, as it presents a sequence of words, lexically determined by the protocol and quite neutral, and a sequence of emotional speech, with the same speakers, recorded in the same audio conditions, it allows us to compare scores for speaker identification between neutral speech and emotional speech. Secondly, the corpus collection has been made with blind and half-blind volunteers from 23 to 79 years old. Thus we can compare scores across speaker age. Moreover we have the opportunity to work with elderly person who often have specific voice qualities.

Table 2 :

 2 ScenariiTable2summarizes the 6 different scenarii and the emotions asked to the participant.

	can recognize an emotion that is of the opposite valence
	of what the participant was supposed to express (the
	experimenter selects Anger when Joy has been acted); it
	can recognize no emotion at all (the experimenter selects
	Neutral when a strong Anger was expressed, or when the
	emotion has not been acted intensely enough); it can
	recognize an emotion that is close to what is expected,
	but too strong or too weak (Sadness instead of
	Disappointment). The participant is asked to act the
	emotion again, either until it is correctly recognized by
	the system, or when the experimenter feels that the
	participant is tired of the game.
	Emotional data acquired through acting games obviously
	do not reflect real-life emotional expressions. However,
	We can also question the relevancy of having the
	participant imagine the situation, instead of having him
	live it in an experimental setting. We should note that for
	obvious ethical reasons we can not put them in a situation
	of emergency such as "being hurt, and ask for immediate
	help": we can only have them pretend it. Another obvious
	reason for setting this kind of limited protocol is a matter
	of credibility of the settings: currently, the only available
	prototype does not fit the target application
	characteristics (Nao is fifty centimeters high, and its
	motion is still under development).
	Scenarii	Emotions
	Medical emergency	Pain, stress
	Suspicious noises	Fear, anguish, anxiety
	Awaking (good mood)	Satisfaction, joy
	Awaking (bad health)	Pain, irritation, anger
	Awaking (bad mood)	Sadness, irritation
	Visit from close relations	Joy

The system is presented as being under-development, and most of the times it does not correctly recognize the emotion: it the strategies that are being used through our Wizard-of-Oz emotion detection tool allow us to elicit emotional reaction in the participant. An example: the participant is convinced that he expressed Joy, but the system recognizes Sadness. The participant's emotional reactions are amusement, or frustration, boredom, irritation. Our corpus is then made of both acted emotions, and spontaneous reactions to controlled triggers. The distinction between acted and spontaneous expressions will be spotted in our annotations; this distinction is really important to have an estimation of how natural the corpus is

[START_REF] Tahon | Acoustic measures characterizing anger across corpora collected in artificial or natural context[END_REF]

.

Table 5 :

 5 The results are referred in the table 5, error rate, number of segments for test and trust interval (binomial distribution test). Speaker identification, age influence: error rate,

		Old person	Young person
	Error rate	17.00%	38.00%
	Number of segment	66	63
	Trust interval	9.18%	12.24%

Table 5

 5

			below shows the error rate
	for speaker identification (1 among 15) across the 3
	corpora: "repeating words", "scenario spontaneous" and
	"scenario acted". The parameters we have chosen for the
	gaussian model are the followings: 5 gaussians, train
	duration: 10s, test duration: 1s.
				TEST
			"Words" "Spontane	"Acted"
				ous"
		"Words"	28.60%	78.60%	88.00%
	TRA IN	"Spontaneous" "Acted"	X X	45.10% X	60.20% 56.30%

Table 5 :

 5 Error rates for speaker identification across the

http://trans.sourceforge.net/en/presentation.php


Marie Tahon
email: mtahon@limsi.fr

Agnes Delaborde

Laurence Devillers

Corpus of Children Voices for Mid-level Markers and Affect Bursts Analysis

Keywords: Audio Signal Processing, Emotion Detection, Human-Robot Interaction

This article presents a corpus featuring children playing games in interaction with the humanoid robot Nao: children have to express emotions in the course of a storytelling by the robot. This corpus was collected to design an affective interactive system driven by an interactional and emotional representation of the user. We evaluate here some mid-level markers used in our system: reaction time, speech duration and intensity level. We also question the presence of affect bursts, which are quite numerous in our corpus, probably because of the young age of the children and the absence of predefined lexical content.

Introduction

In the context of Human-Robot Interaction, the robot usually evolves in real-life conditions and then faces a rich multimodal contextual environment. While spoken language constitutes a very strong communication channel in interaction, it is known that lots of information is conveyed nonverbally simultaneously to spoken words [START_REF] Campbell | On the use of nonverbal speech sounds in human communication[END_REF]. Experimental evidence shows that many of our social behaviours and actions are mostly determined by the display and interpretation of nonverbal cues without relying on speech understanding. Among social markers, we can consider three main kinds of markers: interactional, emotional and personality markers. Generally-speaking, social markers are computed as long-term markers which include a memory management of the multi-level markers during interaction. In this paper, we focus on specific mid-level and short-time acoustic markers: affect bursts, speech duration, reaction time and intensity level which can be used for computing the interactional and emotional profile of the user. In a previous study, we have collected a realistic corpus (Delaborde, 2010a) of children interacting with the robot Nao (called NAO-HR1). In order to study social markers, we have recorded a second corpus (called NAO-HR2), featuring children playing an emotion game with the robot Nao. The game is called interactive story game (Delaborde, 2010b). So far, there exist few realistic children voices corpora. The best known being the AIBO corpus [START_REF] Batliner | You stupid tin box"children interacting with the AIBO robot: A cross-linguistic emotional speech corpus[END_REF], in which children give orders to the Sony's pet robot Aibo. Two corpora were collected for studying speech disorders in impaired communication children [START_REF] Ringeval | Automatic prosodic disorders analysis for impaired communication children, 1st Workshop on Child, Computer and Interaction (WOCCI)[END_REF]. In both studies, there are no spoken dialogs with robots; only the children are speaking.

Many previous studies focus on one of the three social markers. Interactional markers can be prosodic as in [START_REF] Breazeal | Recognition of affective communicative intent in Robot-Directed speech[END_REF]: five different pitch contours (praise, prohibition, comfort and attentional bids and neutral) learnt from infant-mother interaction are recognised by the Kismet robot. Mental state markers can also be only linguistic as the number of words, the speech rate (Kalman, 2010). Personality markers can be linguistic and prosodic cues [START_REF] Mairesse | Using linguistic cues for the automatic recognition of personality in conversation and text[END_REF]. Emotional markers can be prosodic, affect bursts and also linguistic. The concept of "affect bursts" has been introduced by Scherer. He defines them as "very brief, discrete, nonverbal expressions of affect in both face and voice as triggered by clearly identifiable events" [START_REF] Scherer | Affect Bursts, in Emotions[END_REF]. Affect bursts are very important for real-life interactions but they are not well recognized by emotion detection systems because of their particular temporal pattern. [START_REF] Schröder | Experimental study of affect bursts[END_REF] shows that affect bursts have a meaningful emotional content. Our hypothesis is that non verbal events and specific affect bursts production are important social cues during a spontaneous Human-Robot Interaction and probably even more with young children.

Section 2 presents the protocol for collecting our second children emotional voices corpus. The content of the corpus NAO-HR2 is described in Section 3: affect bursts, speakers and other interactional information. Section 4 summarizes the values we can expect for some mid-level social cues. Finally, Section 5 presents our conclusion and future work.

Data collection 2.1 Interactive Story Game

We have collected the voices of children playing with the robot Nao and recorded with lapel-microphone. Nao told a story, and two children in front of it where supposed to act the expected emotions in the course of the story. A game session consists in 3 phases: first the robot explains the rules and suggests some examples, the second part is the game itself, and the last part is a questionnaire proposed by an experimenter. The children are presented a board, on which words or concepts are drawn and written (such as "house", or "poverty"). Emotion tags are written in correspondence for each of this word. The player number one knows that, for example, if the notion "poverty" occurs in the course of the story, he will have to express sadness. He can express it the way he wants: he can speak sadly, or do as though he was weeping; children were free to interpret the rules as they wanted to. Once the rules are understood by the two players, Nao starts to tell the story. When it stops speaking, one of the players is supposed to have spotted a concept in the previous sentence, and is expected to play the corresponding emotion. If the robot detects the right emotion, the child wins one point.

Semi-automatic Human-Robot Interaction System

The behaviour of the robot changes in the course of the game. It can be neutral, just saying "Your answer is correct", or "not correct". It can also be empathic "I know this is a hard task", etc. Fuzzy logic rules select the most desirable behaviour for the robot, according to the emotional and interactional profile of each child, and their sex. This profile is built according to another set of fuzzy logic rules which process the emotional cues provided manually by the Wizard experimenter. The latter provides the system with the emotion expressed by the child (a label such as "Happiness", "Anger", "Sadness", etc.), the strength of the emotion (low, average or high activation), the elapsed time between the moment when the child is expected to speak and the time he starts speaking, and the duration of the speaking turn (both in seconds). From these manually captured cues, the Human-Robot Interaction system builds automatically an emotional and interactional representation of each child, and the behaviour of the robot changes according to this representation.

The dynamic adaptation of the behaviour of the robot and the design of the profile, based on a multi-level processing of the emotional audio cues, are explained in (Delaborde, 2010b). Table 1 The collected audio data is subsequently processed by expert labellers. On each speaker's track, we define speaker turns called instances. The annotation protocol is described in detail in (Delaborde, 2010b). The annotation scheme consists in emotional information (labels, dimensions and affect bursts), but also mental-state and personality information based on different time windows. In this paper, we focus on the study of affect bursts and others mid-level markers such as reaction time, duration but also the low-level marker intensity.

Contents of NAO-HR2 corpus

Description of the corpus

The NAO-HR2 corpus is made up of 603 emotional segments for a total amount of 21mn 16s. Twelve children (from six to eleven years old) and four adults have been recorded (five boys, seven girls, one woman and three men). For this study, we have selected only the speech instances which occur during the story game (not during the questionnaire). In consequence, we obtain 20 emotional answers per gaming session: 10 emotional answers for each speaker. In that way the number of speaker turns is quite similar from one speaker to another.

Affect bursts

An annotation tag indicates the presence or absence of an affect burst in the instances. We notice that a large majority of the corpus is made up of affect bursts. Table 2 summarizes the number of affect bursts (AB) over the total number of instances (TT) for each group of speaker. We have separated the children in two groups of 5 according to their age: the younger are from 6 to 7 years old, the older over 8 year old.

# AB (TT)

Mean From these results we can conclude that asking a participant to express an emotion without any predefined lexical content leads to a high number of affect bursts.

Children seem to use more often affect bursts than adults and young children even more. It seems that they are not at ease with finding words to express an emotion. Both children and adults express happiness laughing, but only children use "grr" affect bursts for anger in our corpora. Expressions of fear are usually more affect bursts for children than for adults. Affect bursts usually contain only a single phoneme; it is not possible to compute easily a speaking rate.

Results on Social Markers

In this section, we have manually measured the different markers in all game sessions.

An example is shown in Figure 1. Nao says: "a lot of sadness", the word "sadness" is one of the keywords written on the board and the child has to express the corresponding emotional state which is sadness. The four social markers we are studying, are represented in red: reaction time is 4.42s, speech duration is 2.17s, mean intensity is 52.83dB (after normalization: 28.67dB) and mean Harmonics-to-noise Ratio is 10.95dB. Reaction Time is important for this turn; the mean value of this 10 year old boy is 3.07s. Intensity and HNR are also lower than the mean values obtained on his whole session (Intensity mean is 32.43dB and HNR mean is 12.56dB).

Intensity and HNR values correspond to what is expected when acting sadness; a high reaction time probably means that the boy was not at ease with this specific turn.

Reaction Time

The reaction time (RT) represents the interval between the time when the speaker is expected to speak (when Nao stops telling the story), and the time he indeed starts to speak. In the context of our game, the children were not supposed to call up their knowledge, or to think about the best answer. They were supposed to act the emotion written on the board. The longer the reaction time, the more the speaker postpones the time of his oral production. This parameter is one of the parameters used for the definition of the dimension "self-confidence" of the emotional profile. The shorter the reaction time, the more the speaker tends to be self-confident. Table 3 presents the mean and standard deviation of mean reaction times for each child.

Mean RT (s) Std RT (s) 4.62 2.00 Table 3: Reaction Time Some children are not at ease with the game, and their RT is much more important than the other (RT = 7.73 for children n°12, 6 year old). When the RT value is so high it often means that the children did not find any answer to give to NAO in the time he has to (if the child did not answer after 12.5s, the robot continues the story). Hesitation is quite used by children who have an important RT.

Estimation of Speech Duration

The speech duration (SD) is another parameter used for the emotional profile of the speaker. It corresponds to the duration of speech of the speaker, for each speaking turn. Children included small pauses (from 850ms to 1.40s) in their speech. These short silences are not considered as ends of speaking turn: it can be breathing, hesitating, thinking, and the speaker resumes speaking. Mean SD (s) Std SD (s) 2.01 1.30 Table 4: Speech Duration for each turn We notice in table 4 that the mean SD is generally quite short. The turns are mostly composed of one single syllable. As we have seen before the proportion of affect bursts is quite important and most of them have short durations. As the players do not have any lexical support except what Nao have just said, they are not simulated to speak a lot.

Estimation of Intensity

For each session, both children were recorded with separate microphones which have their own gain. We compute the mean intensity (Int) normalized to the noise value for each session. It is also possible to estimate the HNR value on voiced parts only. Hesitation is often expressed with a lower intensity: on hesitation turns, mean intensity is from 45% to 70% lower than the mean intensity for the same child. Figure 2 shows that mean Intensity seems to decrease with RT and HNR to increase with RT. As we have said, a small RT generally signifies a good self-confidence; our data show that it is correlated with a high Intensity and a small HNR. When the child is at ease, he will speak loud.

Conclusion and Future Works

The NAO-HR2 children voices corpus is composed of French emotional speech collected in the course of a game between two children and the robot Nao. A semi-automatic Human-Robot Interaction system built the emotional and interactional representation of each child and selected the behaviour of the robot, based on the emotions captured manually by an experimenter. The data we collected allow us to study some parameters which take part in the setting up of the emotional and interactional profile. We have analysed some of the mid-level cues which are used in our Human-Robot Interaction system. Among those cues, reaction time, intensity level and speech duration do make sense in our child-robot interaction game, but speaking rate does not seem to be relevant in that particular context. Indeed, as the children are quite young (from six to eleven years old), and as they are not given any predefined lexical content, they usually express their emotions with affect bursts. The younger the child, the more he/she will use affect bursts. In a future work, we will also study the speaking rate in longer turns of child speech. For the needs of our data collection, the affective interactive system was used in Wizard-of-Oz (an experimenter captured manually the emotional inputs); in a next collection, we will use it with automatic detection of the emotions in speech, and then collect more data to confirm our analysis.

Acknowledgement

This work is financed by national funds FUI6 under the French ROMEO project labelled by CAP DIGITAL competitive centre (Paris Region).

Figure 1 :

 1 Figure 1: An example of social markers during the story game, the markers are collected with Praat

Figure 2 :

 2 Figure 2: Intensity and HNR in function of the reaction time for the 12 children

Table 1 :

 1 Multi-level cues and social markers

	gives an overview of the different level of
	processing of the emotional audio signal: from low level
	cues computed from the audio signal, to high level
	markers such as emotions, emotional tendencies, and
	interactional tendencies.
	Low-level Cues Mid-level Cues	High Level Social Markers
	• Intensity level • Prosody • Spectral envelope	• Affect bursts (Laughs, hesitation, 'grr') • Speech duration • Reaction Time • Speaking rate	• Emotion (label, dimension) • Interactional tendencies (e.g. dominance) • Emotional tendencies (e.g.
			extraversion)

Table 5 :

 5 Intensity and HNR means and std

	The