Laurent Prévot

Brigitte Bigi

Roxane Bertrand

A quantitative view of feedback lexical markers in conversational French

Keywords: Feedback, Backchannel, Corpus, French Language

This paper presents a quantitative description of the lexical items used for linguistic feedback in the Corpus of Interactional Data (CID). The paper includes the raw figures for feedback lexical item as well as more detailed figures concerning interindividual variability. This effort is a first step before a broader analysis including more discourse situations and featuring communicative function annotation.

Objectives

Conversational feedback is mostly performed through short utterances such as yeah, mh, okay not produced by the main speaker but by one of the other participants of a conversation. Such utterances are among the most frequent in conversational data [START_REF] Stolcke | Dialogue act modeling for automatic tagging and recognition of conversational speech[END_REF]. They also have been described in psycho-linguistic models of communication as a crucial communicative tool for achieving coordination or alignment in dialogue [START_REF] Clark | Using language[END_REF].

The general objective of the project (ANR CoFee: Conversational Feedback) 1 [START_REF] Prévot | Cofee-toward a multidimensional analysis of conversational feedback, the case of french language[END_REF] in which this work takes place is to propose a fine grained model of the form/function relationship concerning feedback behaviors in conversation. The present study is first exploration aiming at knowing better the distribution of these items in one of our corpus. More precisely, we would to verify how much interindividual variability we will face in further study and whether we can identify a structure in this variability (e.g speaker profiles). Second, we tried 1 See the project website: http://cofee.hypotheses.org to check there some strong trends in terms of evolution of use of these items in the course of the conversation. This later point was not conclusive and is not developed in this paper.

Some data-intensive works exist for English [START_REF] Gravano | Affirmative cue words in task-oriented dialogue[END_REF], Japanese [START_REF] Kamiya | Coherent back-channel feedback tagging of in-car spoken dialogue corpus[END_REF][START_REF] Misu | Toward construction of spoken dialogue system that evokes users' spontaneous backchannels[END_REF] or Swedish [START_REF] Allwood | On the semantics and pragmatics of linguistic feedback[END_REF][START_REF] Cerrato | Investigating Communicative Feedback Phenomena across Languages and Modalities[END_REF][START_REF] Neiberg | Semisupervised methods for exploring the acoustics of simple productive feedback[END_REF] but not on many other languages such as French for example. On French, the work of [START_REF] Muller | An empirical study of acknowledgement structures[END_REF][START_REF] Muller | Grounding information in route explanation dialogues[END_REF]) concerned a smaller scale (A hour corpus) and very specific task. [START_REF] Bertrand | Backchannels revisited from a multimodal perspective[END_REF] was focussed on the feedback inviting cues and also on a smaller scale (2 × 15 minutes). They showed that particular pitch contours and discursive markers play a systematic role as inviting-cues both for vocal and gestural back-channels.

The paper is structured as follow. Section 2 presents the conversational corpus used for this study, then section 3 presents how this corpus has been processed. Section 4 is related to general figures for the feedback lexical items, followed by more detailed information about inter-individual variability (section 5).

The corpus

The Corpus of Interactional Data (CID) [START_REF] Bertrand | Le cid-corpus of interactional data-annotation et exploitation multimodale de parole conversationnelle[END_REF][START_REF] Blache | Creating and exploiting multimodal annotated corpora: the toma project[END_REF] 2 is an audio-video recording of 8 hours of spontaneous French dialogues, 1 hour of recording per session. Each dialogue involved two participants of the same gender. One of the following two topics of conversation was suggested to participants: conflicts in their professional environment or unusual situations in which participants may have found themselves. It features a nearly free conversational style with only a single theme proposed to the participants at the beginning of the experiment. This corpus is fully transcribed and forced-aligned at phone level. Moreover, it has been annotated with various linguistic information (Prosodic Phrasing, Discourse units, Syntactic tags, ...) [START_REF] Blache | Multimodal annotation of conversational data[END_REF] which will allow us later to take advantage of these levels of analysis.

Numerous studies have been carried out in prepared speech. However, conversational speech refers to a more informal activity, in which participants have constantly to manage and negotiate turn-taking, topic changes (among other things) without any preparation. As a consequence, numerous phenomena appear such as hesitations, repeats, backchannels, etc. Phonetic phenomena such as non-standard elision, reduction phenomena, truncated words, and more generally, nonstandard pronunciations are also very frequent. All these phenomena can impact on the phonetization, then on alignment.

Processing the corpus

The transcription process is done following specific conventions derived from that of the GARS (Blanche-Benveniste and Jeanjean, 1987). The result is what we call an enriched orthographic transcription (EOT), from which two derived transcriptions are generated automatically : the standard orthographic transcription (the list of orthographic tokens) and a specific transcription from which the phonetic tokens are obtained to be used by the grapheme-phoneme converter. From the phoneme sequence and the audio signal, the aligner outputs for each phoneme its time localization. This corpus has been processed with several aligners. The first and main one [START_REF] Brun | Ants: le système de transcription automatique du loria[END_REF] is HMM-based, it uses a set of 10 macro-classes of vowel (7 oral and 3 nasal), 2 semi-vowels and 15 consonants. Finally, from the time aligned phoneme sequence plus the EOT, the orthographic tokens is time-aligned.

The alignment for this paper is another version that has been carried out using SPPAS 3 [START_REF] Bigi | SPPAS: a tool for the phonetic segmentation of speech[END_REF]. SPPAS is a tool to produce automatic annotations which include utterance, word, syllabic and phonemic segmentations from a recorded speech sound and its transcription.

Alignment of items of the list given in (1) were then manually verified. Largest errors were corrected to obtain reliable alignments.

DM prononciations are the standard ones except [START_REF] Gravano | Affirmative cue words in task-oriented dialogue[END_REF]).

(1) ah (ah), bon (well), ben (well), euh (err, uh), mh (mh), ouais (yeah), oui (yes), non (no), d'accord (agreed), OK (okay), voilà (that's it, right)

Strictly speaking, the list (1) is not exhaustive. However, other items are already in the thin part of the distribution's tail. Moreover, some of the items such as euh / err are not necessarily related to feedback. However, by crossing lexical values with position we expect to get close enough the full set of tokens involved in feedback. For example, initial euh not followed by a feedback related item will not be included in the final dataset. This is also an objective of the present work to identify these situations.

The different markers exhibit very different figures with regard to their location as it can be seen in 1. While some are specialized in isolated feedback such as the continuer mh which is most of the time backchanneled, others are found at the beginning of utterances such as euh, ah. The later makes sense since euh is also a filled pause. In total 197 different combinations of the basic markers were identified. The most frequent are the simple repetitions of items such as ouais (up to nine times) or mh. There are also more complex structures as exhibited in (2) that seem to mix two kinds of items: base ones and modifiers (ah, euh). The base ones seem by default to carry general purpose communicative functions as described in [START_REF] Bunt | Multifunctionality and multidimensional dialogue act annotation[END_REF][START_REF] Bunt | The semantics of feedback[END_REF] while the others can also be produced alone but are generally dealing specific dimension such as turn-taking, attitude expression or time management.

(2) a. ah ouais d'accord ok (ah yeah right okay) b. voilà oui non (that's it yes no) With regard to duration, the data is rather messy concerning the very long items. There are extreme lengthening on these units. Aside that and the filler uh that exhibit a wide spread, the other items are not produced with huge variations. Monosyllabic remain well centered around 150-250 ms while disyllabic and repeated items are distributed in the 250-500 ms range. This is important for our next step in which automatic acoustic analysis of these items will be performed.

• • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • ah

Inter-individual variability

Inter-individual variation is a big issue on the way to the generalizability. We would like to understand some of the feedback producing profiles.

Our intuitions coming from familiarity of the data is that there are strong variation but they correspond to a few different speaking styles. In future work, we would like to see in a second step whether we can identify and characterize these styles. Figure 3 illustrates the total figures of feedback per speaker. As expected variation is huge, from 132 to 425 but with in fact with few outliers with a nice batch of speaker in the 200 -300 range. The wider spread of the distribution in the high range comes from two factors. First of all, there are participants producing a high quantity of feed-back items. They produce a massive amount of light backchannels (mh, ouais) compared to lowquantity feedback producers. The later also produce feedback during the long pauses of the main speaker but they produce much less overlapping backchannels. This should be double checked with a specific measure (adding overlapping as a factor). However, a second effect seems important for at least one speaker (the outlier): the amount to time holding the floor. In fact the speaker producing the most feedback did so because she was rarely the main speaker.

In order to get a global idea of the different uses of these items, Figure 5 represents the proportion of each item per speaker. As expected, the variation is important but one can spot some tendencies. For examples for the most frequent items, the rank seems to preserved across speakers. Based on their feedback profile (proportion of use of each items as illustrated in Figure 5), we attempted to cluster the participants as showed in 5. While the lower parts of the dendrogram are hard to interpret the higher part matches well with the impression acquired by listening to the corpus (no backchannels and rather formal feedback vs. lots of backchannels and very colloquial style).

Current and Future Work

About this first batch of analyses, we will complete the analysis of the evolution during the conversation. More precisely, we will go at the individual level looking for time-based changes In parallel to this work, we are launching independent prosodic and kinesic analyses of the forms, as well as a discourse analysis of the functions. Moreover the work is being extended by adding two corpora in the study in order to allow for a better situation generalisability: A French MapTask; and a third corpus consisting in a less cooperative situation. The idea is later to bring together the observations from the different levels in order to propose a multidimensional model for feedback in French dialogues.

Those are steps toward more extensive studies in the spirit of [START_REF] Gravano | Affirmative cue words in task-oriented dialogue[END_REF] or [START_REF] Neiberg | Semisupervised methods for exploring the acoustics of simple productive feedback[END_REF] on French language and in which we hope to address more directly the issue of discourse situation generalisability.

Figure 1 :

 1 Figure 1: Distribution of isolated vs. initial position for the most frequent lexical items

Figure 2 :

 2 Figure 2: Duration (in seconds) of each lexical type

Figure 3 :

 3 Figure 3: Number of feedback items per speaker

Figure 4 :

 4 Figure 4: Distribution of the lexical items

Figure 5 :

 5 Figure 5: Dendrogram of the participants cluster based on their feedback profile

  There are only two items with non standard cases that are over 2 occurrences: sampa: m.w.e.) that is an hybrid between mh and ouais, and sampa w.a.l.a, a reduction of v.w.a.l.a voilà.The extraction themselves have been realized by the authors with a Python script and all the statistical analyses and plots have been produced with R statistical analysis tool. Isolated items are items or sequence of items surrounded by pauses of at least 200 ms and not including any extra material than the items of this list ; (ii) Initial items (or sequence items) are located in front of some other items (but there is no other material within the sequence). Most of these items also occur in final or even surrounded positions but we did not consider these cases since they do are not clearly related to feed-

	4 Descriptive statistics for the lexical
	markers used in feedback
	All the lexical items of the list given in (1) were
	automatically extracted and categorized into two
	categories: (i)

3 http://www.lpl-aix.fr/∼bigi/sppas/ for a few cases. back. More precisely surrounded items are mostly consisting in breaks of disfluencies or genuinely integrated construction (e.g j'étais d'accord avec lui / I agreed with him). Final ones can play a role in eliciting feedback or sometimes bring some kind of closure at the end of the utterance (what has been described as Pivot Ending in

http://www.sldr.org/sldr000027/en

Acknowledgment

This work has been realized with the support of the ANR (Grant Number: ANR-12-JCJC-JSH2-006-01) and exploited aligned data produced in the framework of the ANR project (Grant Number ANR-08-BLAN-0239). We would like to thank all the members of these two projects.


Brigitte Bigi
email: brigitte.bigi@lpl-aix.fr

Cristel Portès
email: cristel.portes@lpl-aix.fr

Agnès Steuckardt
email: agnes.steuckardt@univ-montp3.fr

Marion Tellier
email: marion.tellier@lpl-aix.fr

Multimodal Annotations and Categorization for Political Debates

The paper introduces an annotation scheme for a political debate dataset which is mainly in the form of video, and audio annotations. The annotation contains various information ranging from general linguistic to domain specific information. Some are annotated with automatic tools, and some are manually annotated. One of the goals is to use the information to predict the categories of the answers by the speaker to the disruptions. A typology of such answers is proposed and an automatic categorization system based on a multimodal parametrization is successfully performed.

INTRODUCTION

This work was conducted to analyse political debates in a multimodal perspective. Particularly, this study focused on the answers produced by a main speaker after he was disrupted. The approach relies on the annotations of a debate and on their review. This implies technical and methodological levels to produce high quality multimodal annotations. Then these different types of annotations are merged into a unified framework taking cues from different modalities encoding gestures, prosody, intonation, linguistic information, acoustic information, etc. The aim was to analyse repartee via this multimodal perspective: how MPs can answer to disruptions, what strategies are used, with which modalities and how they are correlated.

Multimodal annotation is often reduced to the annotation of gesture, possibly accompanied by another level of linguistic information (e.g. morpho-syntax). This paper follows a different route. A broad-coverage approach, aiming at annotating a large set of linguistic domains is proposed. This paper reports the methodology used to annotate manually or automatically the corpus (section 2): a video of a political debate at the French National Assembly by Yves Cochet (an ecologist MP). This methodology is based on what was used prior to annotate CID -Corpus of Interactional Data [START_REF] Blache | Multimodal annotation of conversational data[END_REF]. The annotations and conventions were adapted for the purpose of the specific study of answers to disruptions. A typology of these answers was then proposed (section 4) depending on strategies the MP was using. Finally, an automatic categorization system based on a multimodal parametrization was trained from these data (section 5). Classification scores leave us to consider that the proposed typology is relevant.

Before entering into the detail of the annotations for different linguistic phenomena, it is necessary to underline that the different information domains to be annotated (see section 3) do not rely on the same granularity level and then require a different amount of work. As a consequence, this work is still in progress. However, we already have significant results, as it will be shown hereafter.

CORPUS DESCRIPTION

The corpus is a political debate at the French National Assembly, on May 4th, 2010. A subset of about 4 minutes was selected. An ecologist MP, Yves Cochet, debates about a law named "Grenelle II de l'environnement" to be voted. Y. Cochet was speaking about electricity, Greenhouse gas emissions, and the electricity production and the pollution they produce. These subjects generated many reactions and MPs disrupted Y. Cochet by remarks or laughs.

The video was downloaded from the National Assembly web site. It is a standard Flash Video file. The quality is good enough but the definition is small: 320x256. However, it allowed us to annotate gestures, except in some rare cases: sometimes the camera focused on the Assembly instead to focus on the main speaker. The audio track was extracted from the video. The quality is good enough to produce reliable annotations.When MPs in the Assembly are speaking loudly, the audio track contains noises. This hinders the coding of the prosody annotation level but not the other domains, as the main speaker can be heard well enough.

MULTIMODAL ANNOTATIONS

This section is dedicated to the different tools and conventions used to annotate. It illustrates the kind of process to implement in the perspective of obtaining rich and broadcoverage multimodal annotations.

Annotation encoding

Multimodal annotation is faced with the necessity of encoding many different information types, from different domains, with different levels of granularity. Actually, the focus is on video corpus with annotations such as prosody, syntax, gestures, etc. Each of these annotations is aligned on the signal and/or the video. Linguistic annotation, especially when dealing with multiple domains makes use of different tools during the same project. The annotation itself should be done in the most ergonomic and convenient tool for the given annotation task. Then it is up to the person to develop the exploitation tool to create "views" in this very rich and layered object corresponding to the whole annotation set. In this way, analysts, annotators and end-users never face the intrinsic complexity of a rich multi-level annotation framework. Acoustic analysis, phonetics and prosody annotation is very often done using Praat [START_REF] Boersma | Praat: doing phonetics by computer[END_REF]. Gestures, and more generally multimodal studies, now rely on higher level systems such as Anvil [START_REF] Kipp | Anvil -a generic annotation tool for multimodal dialogue[END_REF] or Elan [START_REF] Wittenburg | Elan: a professional framework for multimodality research[END_REF]. Not to speak of orthographic transcription which can be done with many tools. None of these tools are directly interoperable, each using a native format, some of them on top of XML, some others developing an ad hoc markup language.

To merge all the annotations, the TextGrid data file format was chosen. It is a Praat native file format based on an ASCII encoding, and many other tools can import it. The limitation of this choice is that TextGrid has a flat representation, and Elan or Anvil can produce tiers in a tree. On the other hand, Praat can deal with point tiers (often used for prosody). All the automatic annotation tools we developed have a read/write API for this format which will be distributed on-demand. The transcription and the prosody annotation were made using Praat. Gestures were annotated with Elan, and tiers were exported (by our API) to the TextGrid format.

The video, the wav file and the TextGrid annotation file are distributed under the terms of the GPL license on the Speech & Language Data Repository web site 1 . Figure 1 shows a screenshot of the annotation tiers. 3.2 Annotation description

IPU segmentation

The audio signal was automatically segmented in IPUs -Inter-Pausal Units. IPUs are blocks of speech bounded by silent pauses of more than 200 ms (this duration is well suited for French), and aligned on the speech signal.

Enriched Orthographic Transcription

The transcription process takes as input the set of IPUs.

The corpus was then orthographically transcribed following transcription guidelines in line with the French GARS conventions [START_REF] Blanche-Benveniste | Le français parlé[END_REF]. This transcription can be seen in the first tier of Figure 1.

One of the characteristics of speech is the important gap existing between a word's phonological form and its phonetic realisations. Specific realisation due to elision or reduction processes are frequent in speech data. Therefore transcribers were asked to provide an enriched orthographic transcription (EOT), which includes, for example, manually annotated non-standard events such as: mispronunciations, truncated words, some liaisons, elisions, laughs, etc. Some of these specificities have a direct consequence on the phonetisation procedure and so on the syllabification. From the EOT, two transcriptions are generated automatically : (1) the standard orthographic transcription from which the orthographic tokens are extracted to be used for the syntax analysis and its related tools (POS tagger, parser, etc.); (2) a specific transcription from which the phonetic tokens are obtained to be used by the grapheme-phoneme converter.

The time spent for this annotation is 1 hour for 1 minute of speech.

Phonetization

An automatic grapheme-phoneme converter was first applied from the EOT. It produced as output a sequence of phonemes coded in Sampa 2 . An automatic alignment was applied to obtain each phoneme time localization. Then, automatic syllabification was performed using the system described in [START_REF] Bigi | Automatic detection of syllable boundaries in spontaneous speech[END_REF]. Finally, from the time aligned phoneme sequence plus the EOT, the orthographic tokens were automatically time-aligned. Table 1 recaps the main figures about the EOT and the phonetization. Figure 1 shows aligned words on tier 2, syllables on tier 3 and phonemes on tier 4.

Prosody

Prosody can be conceived as the linguistic study of rhythm and melody of speech: metrical variation (rhythm) and intonational variation (melody) play a major role in the organization of speech flow by grouping words in different lev- els of hierarchically structured "meaning groups" [START_REF] Cristo | Le cadre accentuel du français contemporain[END_REF]. The present study focuses on intonational tunes. These melodic schemas associated with meaning groups differ from one language to another but most languages have paradigms of contrastive tunes. These tunes assume pragmatic roles in conversation, namely by carrying the attitudes of the speakers towards the transmitted contents [START_REF] Beyssade | French intonation and attitude attribution[END_REF]. Their argumentative role is therefore crucial. Working on French, we have adapted the intonational inventory proposed in [START_REF] Bertrand | Distribution syntaxique, discursive et interactionnelle des contours intonatifs du français dans un corpus de conversation[END_REF]. This inventory comprises two categories of minor tunes (a rising and a falling one) whose scope is roughly the phrase, and four categories of major tunes (falling, rising, rising-falling and falling from penultimate) whose scope is rather the sentence or the clause. Three additional functional distinctions are made within the rising tune category: the continuation rise, the list rise and the final rise. The data is annotated through auditive identification realized by professional phoneticians.

The time spent for this annotation is about 4 hours for 1 minute of speech. Table 2 recaps the main figures about the intonational tunes annotation. Figure 1 shows "RMC" and "mc" examples in the tier number 5.

Gestures

When we code "gestures" we only consider co-speech gestures which normally co-occur with speech. They are movements of the hands and arms produced by people when they talk. They do not belong to a fixed repertoire as gestures of sign language for instance, on the contrary, they are unique, personal and spontaneous. Therefore we do not take into account gestures that do not relate to speech such as scratching one's face, nervous gestures of self touching, etc. There are several classifications of gestures, McNeill's [START_REF] Mcneill | Hand and Mind: What gestures reveal about thought[END_REF][START_REF] Mcneill | Gesture & thought[END_REF] is currently widely used in the community of gesture researchers even if it sometimes needs to be adapted. The formal model we used for the present annotation of hand gestures is thus adapted from the specification taken from McNeill's work. Iconic gestures bear a close formal relationship to the semantic content of speech [START_REF] Mcneill | Hand and Mind: What gestures reveal about thought[END_REF], they usually illustrate concrete ideas. Metaphoric gestures are very similar to iconics except that they are rather used to depict abstract concepts or illustrate things metaphorically. If one cups their hands when saying the word "pollution" for instance, it is a metaphoric gesture because the cup acts as a symbolic image for the concept of pollution.

Deictics gestures refer to things by pointing with the hand, the finger, the chin, etc. They can be either concrete when pointing to someone, something or somewhere or abstract when referring to something/someone absent or a place or even a moment in time. Finally, beats are rhythmic movements that have no semantic connexion to the speech they accompany. They rather stress important words or phrases.

A typical beat would be a flick of the hand or of the finger. Beats are often superimposed on other gestures [START_REF] Mcneill | Gesture & thought[END_REF].

To this typology we added two types of gestures: emblems which are conventionalized and cultural gestures with a fixed form such as a thumb up to say "OK" and interactive gestures which are used to influence directly the conversational interaction [START_REF] Bavelas | Gestures specialized for dialogue[END_REF]. We also added a possibility to code aborted gestures (i.e. gestures that begin but are left unfinished).

Table 3 summarizes these annotations. Figure 1 shows a main gesture type "metaphoric" and a "beat" in the tier number 6, without a secondary gesture type (tier 7). All types of gestures have been implemented in Elan with a controlled vocabulary. The corpus presents 122 gestures produced by Yves Cochet. Two tiers was used to code gesture type: number 6 represents the main gesture type (that is where all gestures are coded) and number 7 is called "secondary gestures type" and enables us to mention when a gesture has two dimensions and when there are superimposed beats. Finally, handedness was coded (tier number 8) since in this particular discourse it seems relevant. In case of a singlehanded gesture, we coded it in its "Handedness": left or right hand. In case of a two-handed gesture, we coded it as if both hands moved in a symmetric way or if the two hands moved in an asymmetric way (Table 4). The time spent for gesture annotation is about 2 hours for 1 minute of video.

Syntax

The stochastic parser used to annotate syntax is described in [START_REF] Blache | Influence de la qualité de l'étiquetage sur le chunking : une corrélation dépendant de la taille des chunks[END_REF]. It generates automatically morpho-syntactic and syntactic annotations. The parser has been adapted in order to account for the specificities of speech analysis. First, the system implements a segmentation technique, identifying large syntactic units that can be considered as the equivalent of sentences in written texts. A second modification concerns the lexical frequencies used by the parser model in order to capture phenomena proper to speech data. The categories and groups counts for the whole corpus are summarized in Table 5. Figure 1 shows categories on tier 9 and groups on tier 10 (GA: adjective phrase; GN: noun phrase; GP: prepositional phrase; GR: adverbial phrase; NV: verb nucleus; PV: group containing a preposition introducing a verb).

Self-repetitions

Self-repetitions were manually annotated. The source and the repetition were categorized as: DIS (disfluent) ; RHE (rhetoric) ; LIST (list). Counts are presented in Table 6. Sometimes, there are several sources finalized by the repeat. This is the reason why the number of sources is higher than the number of repetitions. This was about 45 minutes annotation for 1 minute of speech. Figure 1 shows repetitions on tier 11.

ANSWERS TO DISRUPTIONS

This annotation consists in the proposition of a typology based on a manual analysis of the debate report. This report includes remarks of MPs in the Assembly that can not be heard in the audio track. This enabled us to align on the signal all parts of the discourse which are concerned by the answers. Figure 1 shows this annotation on tier 12.

The contempt strategy

This strategy is used twice in the selected part of the debate. It consists in ignoring the disruption and continuing the discourse. This strategy can be observed in the video with some converging evidence factors: a short silence, an head movement going to the right (where the disruption come from) and coming back to the front, some specific face features (that we interpret as contempt). These disruptions are mentioned in the report of the debate by "laugths" or by the writing statement of the MP who caused the disruption.

The direct answer

This strategy is divided into two sub-categories. In the first, Yves Cochet just says directly "no" (or another short negative answer), without adding any arguments. The second is made of two steps: Yves Cochet repeats the phrase (which is an other-repetition), and then he argues. Figure 2 illustrates a direct answer 3 with the use of irony. In this case, the main gesture is deictic and coupled with an interactive, using right hand. The intonational tune is a falling.

4.3

The "sideway answer"

This strategy does not consist in answering but in commenting the disruption. This comment can be directed to the Assembly (meta-discourse) or it can be addressed to the chair (meta-interaction) [START_REF] Traverso | Réplique[END_REF]. Figure 3 illustrates this latest case. Yves Cochet turns to the chair and he uses low tones.

AUTOMATIC CATEGORIZATION

A system that performs categorization aims at assigning appropriate categories from a predefined classification scheme to incoming data. Categorization is an important component of many large Information Retrieval or Machine Learning system. It is often defined as the content-based assignment of one or more predefined categories to new observed data. Machine Learning approaches to classification (categorization is a classification task) suggest that the construction This section explores and identifies the benefits of the use of automatic categorization. We suppose that if the proposed typology is consistent with the multimodal annotations, an automatic classification system will be able to model it and to produce correct predictions. Unlike manual annotations that can be subjective, the advantage of the automatic method is that it is objective. The C4.5 algorithm (without pruning) proposed in [START_REF] Quinlan | C4.5: Programs for Machine Learning[END_REF] and implemented in the Weka software [START_REF] Holmes | Weka: a machine learning workbench[END_REF] was chosen. This choice was motivated by work reported in [START_REF] Lavesson | Quantifying the impact of learning algorithm parameter tuning[END_REF] which indicates that parameter tuning is often more important than the choice of algorithm. Experiments were carried out with a 100 fold cross-validation process on the annotated corpus presented in this paper.

One of the difficulties related to the multimodal aspect is that annotations are heterogeneous and unsynchronized. The second difficulty is related to missing values, as they need to be modeled. Another difficulty is that we can not assess the degree of confidence of the annotations, we then suppose that we are dealing with uncertain data. Data uncertainty is a recent challenge in classification [START_REF] Chau | Uncertain data mining: A new research direction[END_REF][START_REF] Ren | Naive bayes classification of uncertain data[END_REF][START_REF] Qin | urule: A rule-based classification system for uncertain data[END_REF][START_REF] Liang | Decision tree for dynamic and uncertain data streams[END_REF]. However, some of these studies simulate uncertainty and all of them are concerned with only one modality.

In order to make a category decision, a representation of categories must be established. One difficulty is to transform annotations, which typically are strings of characters, into a more efficient representation for the learning algorithm and the classification task. Moreover, to cope with the unsynchronized aspect, we oversampled the data as proposed in [START_REF] Zhi | HMM modeling for audio-visual speech recognition[END_REF]. One sample is generated for each of the smallest duration annotation: one sample per phoneme in our case.

The reference-time is the middle of each phoneme. Then we obtained about 2,500 samples. At a given time t, each modality is represented as a vector over the set of possible annotations for this modality. The construction of a categorization classifier for category ci ∈ C consists in the definition of a function that, given a sample st, t ∈ T returns a categorization status value for it. All these vectors are concatenated to obtain a classification vector for each sample: it is an early fusion process as proposed in [START_REF] La Cascia | Combining textual and visual cues for content-based image retrieval on the world wide web[END_REF].

Let s m t be the sample of the modality m at time t. This sample is represented by a vector containing annotations as:

s m t = (a m,1 t , a m,2 t , • • • , a m,n t , • • • , a m,N m t
) where a m,n t = 1 if the annotation is affected to the sample for the modality m and a m,n t = 0 otherwise. Then, the multimodal vector st of a sample st is defined as:

st = ( s 1 t , s 2 t , • • • , s m t , • • • , s M T )
. Experiments using this system produce 90.63% of correct classifications.

This first system produced encouraging results, but it does not differentiate empty annotations and missing annotations. In our data, 117 samples are missing for the prosody annotation and 74 samples are missing for the 3 gestures tiers. The pruning solution implemented in Weka was tested and improved the classification score to 91.08% which is unsatisfactory (the relative gain is of 4.80%). Various pruning solutions for the C4.5 algorithm was compared in [START_REF] Saar-Tsechansky | Handling missing values when applying classification models[END_REF] and results was more or less similar for all methods. We adopted an easier way to take into account missing values and we fixed equal values for each missing annotation as: a m,n t = 1 |N m | . This system produced 94.33% of correct classification, which is a significant improvement (the relative gain compared to the first system is 39.49%).

To deal with uncertainty of multimodal annotations, we propose to assign an ǫ probability to non-observed annotations. In a multimodal context, an ǫm for each annotation in a modality m was manually assigned. A small ǫm values was used to manual annotation (ǫm = 0.001) and a bigger one to automatic annotated domains (ǫm = 0.015). Then if a mass is attributed to unobserved annotations, observed annotations must be pruned. Consequently, s m t is made of annotations estimated as: a m,n t = ǫm if this annotation is not assigned to this sample and a m,n t = a m,n t -λmǫm otherwise. λm is the number of non-observed annotations for the modality m of the sample st. This system produced 94.98% of correct classification. The relative gain compared to the previous system is 11.46%, which is a significant.

Correct classification scores for each individual modality4 are interesting: prosody only is 73.83%; gesture types only (main and secondary) is 79.14%; handedness only is 78.38%; syntax only is 72.91% and repetitions only is 72.71%. Each of these annotations improved the classification score in the multimodal system.

The final high classification score leads us to consider that the proposed typology is relevant because the multimodal classifier is able to model and predict the proposed categories with a high degree of confidence. This confirms that 1/ multimodality is a good way to analyse this kind of data and 2/ annotations are a key point in this field. Particularly, gesture and prosody annotations are the most important.

CONCLUSION

A methodological approach to annotate multimodal data in multiple domains was proposed in this paper. This annotation process was applied to a political debate. The video was recorded at the French National Assembly during a debate about ecology. The analysis of the debate report leads us to propose a typology of the strategies used to answer to disruptions. We trained a specific classification system from the multimodal annotations to validate the typology and validated the approach. Our perspective is to annotate the rest of the video. We plan to use the classification system to annotate the answers to disruptions and to manually validate this annotation.

Figure 1 :

 1 Figure 1: Annotation example

Figure 2 :Figure 3 :

 23 Figure 2: Yves Cochet -"[plus d'avions c'est le volcan islandais]"

Table 1 :

 1 Words and phonetic

	Category	Count
	Words	746
	Filled pauses	4
	Silences	137
	Phones	2356
	Syllables	1030

Table 2 :

 2 Prosody

	Category Count Category Count
	RF1	25 RF2	2
	RL	5 RT	2
	RMC	60 mc	16
	F	25 L	1

Table 3

 3

		: Gesture Types
	Gesture	Main Count Second. Count
	beat	22	50
	metaphoric	73	7
	deictic	12	7
	emblem	7	2
	interactive	2	5
	iconic	4
	aborted	2

Table 4

 4

	Hand	: Handedness Count
	left		72
	right		26
	both, symmetric	23
	both, asymmetric	1

Table 5

 5

		: Syntax
	Category	Count Group Count
	adverb	94 GA	42
	adjective	52 GN	101
	auxiliary	4 GP	68
	determiner	77 GR	84
	conjunction	58 NV	110
	interjection	9 PV	4
	preposition	77
	pronoun	110
	noun	136
	verb	107

http://sldr.org/sldr000729/en

The minimum score is 72.71% by assigning each sample to the most frequent class


Basarab Matei

Sylvain Meignen

A new optimization based approach to the empirical mode decomposition

Keywords: empirical mode decomposition, adaptive algorithm Mathematics Subject Classification (2010) : 41, 68, 62D99

In this paper, an alternative optimization based approach to the empirical mode decomposition (EMD) is proposed. The principle is to build first an approximation of the signal mean envelope, which serves as initial guess for the optimization procedure. We develop several optimization strategies to approximate the mean envelope which compare favorably with the original EMD on AM/FM signals.

Introduction

To analyze signals in time and frequency domains at the same time is a challenging issue and many methods exist to this aim: short-time Fourier transform, Wigner-Ville distribution and wavelets [START_REF] Cohen | Time Frequency Analysis[END_REF], [START_REF] Mallat | A Wavelet Tour of Signal Processing[END_REF]. This issue was one of the motivation for the development of the empirical mode decomposition (EMD). The EMD expands a given signal into a set of functions defined by the signal itself, the intrinsic mode functions (IMFs) computed through the so-called "sifting process" (SP). In the denomination of Huang [START_REF] Huang | The empirical mode decomposition and the Hilbert spectrum for nonlinear and non stationary time series analysis[END_REF], an IMF is a function that satisfies two conditions: (a) in the whole data set, the number of extrema and the number of zero crossings must either be equal or differ at most by one; and (b) at any point, the mean value of the envelope defined by the local maxima and the envelope defined by the local minima is zero.

In spite of its practical achievements, the EMD technique is essentially an algorithmic approach which lacks a well-established theoretical proof and a direct, systematic optimization of the method. One of the problems posed by the EMD algorithm arises from the interpolation of the extrema, namely undershoots and overshoots. To avoid this situation, many alternative approaches to EMD propose to directly compute an approximation of the signal mean envelope. This knowledge enables to build the mean envelope first and then deduce the mode. In that framework, several approaches have been proposed to find out the intersection points of the signal with 129 its mean envelope, see [START_REF] Deléchelle | Empirical mode decomposition: an analytical approach for sifting process[END_REF], [START_REF] Peng | Adaptive signal decomposition based on local narrow band signals[END_REF], [START_REF] Chen | A B-spline approach for empirical mode decompositions[END_REF], [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF] and [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF]. In [START_REF] Deléchelle | Empirical mode decomposition: an analytical approach for sifting process[END_REF], the signal mean envelope is a steady state of some partial differential equation which intersects inflection points of the signal. In [START_REF] Chen | A B-spline approach for empirical mode decompositions[END_REF], the mean envelope is a B-spline with knots defined using the signal extrema while, in [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF], the mean envelope is a spline intersecting signal local means. In [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF], the mean envelope is computed directly through an optimization procedure by imposing a series of linear constraints. In this paper, we first compare these methods to asses of these points of interest for AM/FM signals and we bring about potential improvements to the mean envelope estimation proposed in [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF]. We then derive new strategies to approximate the mean envelope using an optimization procedure. The proposed method appears to behave better than the original EMD on AM/FM signals. Let us stress that contrary to the original EMD algorithm where the convergence of the SP is not ensured, we here seek the mean envelope in a finite dimensional subspace which makes the proposed procedure convergent. The outline of our work is as follows. In Section 2, we recall some basic notions about EMD and also present the results of EMD on a sum of AM/FM signals. In Section 3, motivated by the results obtained using the EMD, we propose a different method to compute a mean envelope approximation. We next introduce, in Section 4, a new optimization procedure to approximate the mean envelope. We finally end the paper with some numerical examples on a sum of AM/FM signals.

EMD basis and limitations

The EMD decomposes a signal into a sequence of AM/FM components. Let us first briefly recall the principle of the EMD algorithm for a onedimensional signal x.

Initialization (EMD

): r = x, k = 1 2. While r is not monotonic (a) Initialization (sifting process): m 0 = r, i = 0 (b) While m i is not an IMF repeat (sifting process) i. Compute the mean envelope e i = 1 2 (u i + l i ) of m i ( where u i (resp. l i ) denotes the upper (resp. lower) envelope of m i ) ii. m i+1 = m i -e i ; i = i + 1 (c) x k = m i , r = r -x k , k = k + 1.
When the decomposition is complete, one can write x = K k=1

x k + r. Apparently, the modes x k are computed in an adaptive way. However, the "sifting process" (SP) that computes the IMF is dependent on a stopping criterion whose choice both influences the number of modes and their shape. Furthermore, in some instances, the SP may not converge. We highlight the limitations of the original EMD (by applying the MATLAB code corresponding to [START_REF] Rilling | On empirical mode decomposition and its algorithms[END_REF]) to a simple AM/FM signal. For the signal x of Figure 3.(A), made of two AM/FM signals, the original EMD (with the default parameters (0.05, 0.5, 0.05), see [START_REF] Rilling | On empirical mode decomposition and its algorithms[END_REF] for details), gives 22 IMFs which are physically not meaningful. A more careful look at the obtained results, shows that the information contained in the modes x 1 and x 2 is spread over more than 10 IMFs. To illustrate this, we depict on Figure 1, seven of these significant modes. The origin of such a behavior is the instability of the SP and this motivates the search for alternative techniques to determine the modes.

First approximation of the mean envelope

In this section, we first investigate several methods that were proposed to compute a signal mean envelope approximation in relation with the EMD algorithm. The common idea of these approaches is to seek particular points of the mean envelope which serve as knots. The approximation is then either defined through splines interpolation [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF] or piecewise cubic polynomials interpolation [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF]. Usually, the knots are approximation of the intersection points of the signal with its mean envelope [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF][3], but in some approaches the knots are located at the extrema of the signal [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF].

Determination of points of interest on the mean envelope

Here, we adopt the point of view of [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF] or [START_REF] Deléchelle | Empirical mode decomposition: an analytical approach for sifting process[END_REF], that is we look for the intersection of the signal with its mean envelope, the mean envelope approximation (2) extrema of x (4) different number of extrema Figure 2. Different regions for the best approximations of θ by t, t (2) , t (4) being defined by spline interpolation. We now propose an improved version of the method developed in [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF] which will compare favorably to the existing ones. In what follows, x(t j ) denotes the set of values corresponding to the extrema of x located at t j . In [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF], the mean envelope is assumed to interpolate ( tj , xj ) defined by:

xj = 1 t j+1 -t j t j+1 t j x(t)dt, tj = t j+1 t j t(x(t) -xj ) 2 dt t j+1 t j (x(t) -xj ) 2 dt . (3.1)
The choice for xj is motivated by the fact that, if the extrema of the signal corresponded to those of the first mode x 1 , putting x = x 1 + e and assuming the integral of a mode is null between its extrema we would obtain: xj = 1 t j+1 -t j t j+1 t j e(t)dt, which is indeed a point of the mean envelope since, from the classical mean theorem for continuous functions, there exist d ∈ [t j , t j+1 ] such that:

1 t j+1 -t j t j+1 t j e(t)dt = e(d).
However, there exist two strong limitations to that model which are the following: the extrema of x do not, in general, coincide with those of x 1 and the second, the integral of x 1 between its successive extrema does not necessarily vanish (think of frequency modulated signals for instance). In order to discuss the above remarks, we consider the following example:

x(t) = cos(2πt) + a cos(2πf t) 0 < f < 1 a ∈ R + . (3.2)
We first focus on the determination of the locations of the extrema of x 1 given x. For the signal x defined in (3.2), x 1 = cos(2πt). We study, in this particular example, the approximation of the locations of the extrema of x 1 by those of even order derivative of x. As remarked in [START_REF] Kopsinis | Investigation and enhancement of the empirical mode decomposition method based on a heuristic search optimization approach[END_REF], the extrema of the second order derivative is more likely to give a better approximation of the location of the extrema of x 1 than the extrema of the original signal. However, the proof given in [START_REF] Kopsinis | Investigation and enhancement of the empirical mode decomposition method based on a heuristic search optimization approach[END_REF] only involves maximal deviation and do not correspond to the general case which we now study. Let us denote by θ the locations of the extrema of x 1 and by t (2m) the locations of the extrema of x (2m) the 2mth order derivative of x. Let us recall that #X denotes the cardinality of the set X. To illustrate our strategy to find the best approximation of the locations θ, we only consider m = 1 and m = 2 the extension to higher order derivatives being straightforward. By using either t,t (2) or t (4) to approximate θ, we use the following procedure: 4) , then we compute : θ = argmin l∈t,t (2) ,t (4) (2) , then we compute :

• If #θ = #t = #t (2) = #t (
1 #θ j |θ j -l j | • otherwise, if #θ = #t = #t
θ = argmin l∈t,t (2) 1 #θ j |θ j -l j |,
and we proceed similarly for the two other cases (#θ = #t (2) = #t (4) and #θ = #t = #t (4) ).

• otherwise, if #θ = #t then θ = t (and similarly if #θ = #t (2) , θ = t (2) or #θ = #t (4) , θ = t (4) )

• otherwise, the cardinality of any of t, t (2) or t (4) is different from that of θ, and we cannot conclude only using x (2) and x (4) and we need to consider higher order derivatives.

The results of the proposed procedure are displayed on Figure 2, for the signal defined in (3.2). These numerical simulations show that the best way to approximate θ strongly depends on a and f . In general, θ is unknown and may not be equal to t. In that case, the EMD algorithm sometimes manages to compute, from x, a first mode having the correct number of extrema, but the underlying process is unclear. The numerical simulations of Figure 2.

show that we can retrieve the number of extrema of x 1 by considering those of higher even order derivatives of x. More precisely, in the above example, to find the number of the extrema of x 1 , one considers the smallest order m 0 such that #t (2m 0 ) = #t (2m 0 +2) , for which we always have #t (2m 0 ) = # θ. In that context, the estimation of θ should be computed using only t (2m) , m ≥ m 0 . With this in mind, we will assume #t = #θ, of which a particular example is given by the signal x, sum of two AM/FM modulated functions x 1 and x 2 , displayed on Figure 3. In this case, the first mode is known to be x 1 while the sought mean envelope is x 2 . We then plot the values of x 1 at t, t (2) and t (4) (see Figure 3 (B)). If we assume that the extrema of x 1 are known, we consider the following error measurement: E(l) = 1 #θ j |θ j -l j |, For the signal of Figure 3.(A), we get E(t) = 15.83, E(t (2) ) = 8.34 and E(t (4) ) = 27.07. A more careful look at these numerical results shows that (B): signal x 1 and its values at t, t (2) and t (4) .

the error using t (4) as an estimator of θ is much larger when the frequency of x 1 is locally lower, which is in accordance with the results displayed on Figure 2. On the contrary, when the frequency of x 1 is locally higher, the estimation is better using the extrema of higher order derivatives (note that such results are obtained because the amplitude modulation is between 0.5 and 1.5 both for x 1 and x 2 , see Figure 2). In what follows, θ will denote the locations of the extrema of the even order derivative chosen to approximate θ.

Initial guess for the mean envelope

The study of the previous subsection suggests that the model given by (3.1) should be replaced by x j,

θ x j, θ = 1 θj+1 -θj θj+1 θj x(t)dt, t j, θ = θj+1 θj t(x(t) -x j, θ) 2 dt θj+1 θj (x(t) -x j, θ) 2 dt .
Note that (t j ( θ), x j, θ) does not necessarily belong to the graph of x. Therefore, we replace t j ( θ) by the abscissa c j obtained by the mean theorem, namely:

∃c j s.t. 1 θj+1 -θj θj+1 θj x(t)dt = x(c j ) = x j, θj . (3.3)
Then, if one defines the mean envelope as the cubic spline interpolant at (c, x(c)) := (c j , x(c j )) j , the signal and its approximative mean envelope coincide at that point. To illustrate these improvements, we now compare the mean envelope computed using (c, x(c)) with those proposed in [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF], in [START_REF] Deléchelle | Empirical mode decomposition: an analytical approach for sifting process[END_REF] and in [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF] (recalled in (3.1)). For the sake of consistency, let us recall the methods defined in [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF] and in [START_REF] Deléchelle | Empirical mode decomposition: an analytical approach for sifting process[END_REF]. In the latter, the mean envelope approximation intersects the inflection points of the signal, while in [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF], the mean envelope approximation satisfies the interpolation conditions that follows. Assume that x(t j ) is a minimum for x and that x(t j ) is an extremum for the sequence (x(t j-2 ), x(t j ), x(t j+2 ). Depending on the cases, the shape of the upper (resp. lower) envelope is used to derive that of the lower (resp. upper). Indeed, Let tj be abscissae of the intersection (when it exists) of straight lines L 1 and L 2 defined by

L 1 : f 1 (t) = x(t j )-x(t j-2 ) t j -t j-2 t + x(t j-1 )(t j -t j-2 )-(x(t j )-x(t j-2 ))t j-1 t j -t j-2
and

L 2 : f 2 (t) = x(t j+2 )-x(t j ) t j+2 -t j t + x(t j+1 )(t j+2 -t j )-(x(t j+2 )-x(t j ))t j+1 t j+2 -t j
. If tj > t j , one imposes : (1/2)(f 1 (t j ) + x(t j )) = e(t j ). Otherwise, one sets (1/2)(f 2 (t j ) + x(t j )) = e(t j ). Note, that when the sequence (x(t j-2 ), x(t j ), x(t j+2 ) is monotonic no interpolation condition is imposed on the mean envelope.

We now carry out the comparison of these methods on the modulated sinus functions displayed on Figure 3.(A) The estimation error is computed through the following quantity:

E f (l, x) = j (x 2 (l j )-x j ) 2 j (x 2 (l j )) 2
. By considering that inf denotes the inflection points of x, the results are as follows: E f (inf, x(inf )) = 0.2712, E f ( t, x) = 0.3423, E(c, x(c)) = 0.1876, (where θ = t (2) ). The method developed in [START_REF] Meignen | A new formulation for empirical mode decomposition based on constrained optimization[END_REF] leads to significantly worse results.

Finally, we assume that the extrema of x 1 are known and we compute (3.3) replacing θ by θ: we obtain a new c such that E f (c, x j,θ ) = 0.1246 which means that x(c) = x j,θ is not necessarily a point of the sought mean envelope. However, the estimation of the intersection points given by (3.3) is helpful in the following sense: our concern is to define a strategy to find the intersection points of the signal with its mean envelope starting from the initial guess (c, x(c)), which we do in the following section.

Optimization procedure for the estimation of the mean envelope

The optimization procedure proposed here computes an approximation of the mean envelope of x. Our approach is based on the fact that, for each c j , (defined by (3.3)), there exists an intersection point of x and of its mean envelope in the vicinity of the points (c j , x(c j )). Note that contrary to the approach of [START_REF] Hong | Local mean-based sifting for empirical mode decomposition[END_REF], where the approximation of the mean envelope serves as initial guess to the SP, we replace the SP by an optimization procedure. In what follows, we will assume that # θ = #θ and that x is sampled at the rate ∆t. We denote for any signal x, by e(x) its mean envelope obtained as u(x)+l(x)

2

, where u(x) (resp . l(x)) is the upper (resp. lower) envelope of x, obtained as usual by cubic spline interpolation of its maxima (resp. minima).

Initialization part

We start from (c 1 , x(c 1 )) = (c, x(c)), which defines the mean envelope e 1 of x through cubic spline interpolation. In relation with the classical EMD algorithm, we notice that e 1 should be such that e(m 1 ) (with m 1 = x -e 1 , having extrema located at τ 1 ), has a L 2 norm which should be close to zero.

Our strategy is then to modify (c 1 , x(c 1 )) to obtain (c 2 , x(c 2 )) leading to e 2 in such a way that the L 2 norm of e(m 2 ) (with m 2 = x-e 2 having extrema τ 2 ) is lesser than that of e(m 1 ). To do so, we seek c 2 j in the interval [c 1 j -p∆t, c 1 j + p∆t] (the choice for p will be discussed later) for each j, following a certain optimization criterion based on the L 2 norm of e(m 2 ). Note that the optimal solution lies in a finite dimensional space which ensures the convergence of our procedure. To explore all the possibilities in this finite dimensional space will be computationally prohibitive. Therefore, we decide to move the points (c 1 j , x(c 1 j )) sequentially using one of the following two strategies:

• one (resp. three) interval(s) strategy find the interval [τ 1 j 0 , τ 1 j 0 +1 ] (resp. [τ 1 j 0 -1 , τ 1 j 0 +2 ]), minimizing locally the L 2 norm of e(m 1 ).

put c 2 j = c 1 j , j = j 0 and find c 2

j 0 ∈ [c 1 j 0 -p∆t, c 1 j 0 + p∆t], such that the L 2 norm of e(m 2 ) is minimal over the interval [τ 2 j 0 , τ 2 j 0 +1 ] (resp. [τ 2 j 0 -1 , τ 2 j 0 +2 ]).
These two strategies are designed to test the influence of the neighborhood on the computation of the optimal mean envelope.

Loop part

Once c 2 has been computed the interval indexed by j 0 is marked and we pass on other intervals following the same strategy, never considering the interval indexed by j 0 again. After #θ steps, all the intervals have been marked and we obtain a certain set (c #θ , x(c #θ )). Then, we unmark all the intervals and we start again the procedure from the initialization step with (c, x(c)) = (c #θ , x(c #θ )). We run the algorithm until convergence.

Numerical results

We give an illustration of the two strategies on the signal x of Figure 3.(A) assuming θ = t (2) . The results are shown on Figure 4, where we depict the mode m f (corresponding to the computed mode at the last iteration) as well as its upper, lower and mean envelope, for both strategies, when p = 15. The results show very little differences between the obtained modes. We show on Figure 5.(A) and (B), the decrease of the L 2 norm of the mean envelope of m j computed with both strategies for p = 5, 10, 15 with respect to the iteration. Again, the results show very little differences between the two strategies. On Figure 4.(C) and (D), we check that with the proposed strategies, the error between x 1 and m j decreases in L 2 norm, again for p = 5, 10, 15. We notice that the convergence of m j to x 1 is improved with the second strategy (three intervals strategy) when p is larger which is not the case with the first strategy. This suggests that to consider a larger interval for the minimization of the L 2 norm of e(m j ) leads to smoother upper and lower envelopes for m j . Note also that with the second strategy when p is increased the convergence to x 1 is improved, which is in accordance which what we expected.

Conclusion

In this paper we first discussed the approximation of the extrema of the first mode in the EMD method by the extrema of even order derivatives. This study enabled us to build a signal mean envelope approximation, which we then used as initial guess for a new optimization procedure to improve the approximation. This procedure replaces advantageously the SP of the original EMD for AM/FM signals as numerical results show. Future work should involve the improvement of the computational cost which is beyond the scope of present article.

Figure 1 :

 1 Figure 1: Some of the significant mode obtained for the decomposition of the signal of Figure 3.(A)

Figure 3 :

 3 Figure 3: (A): signal x made of two modulated signals x 1 and x 2 .(B): signal x 1 and its values at t, t(2) and t(4) .

Figure 4 :

 4 Figure 4: The signal m f obtained from the signal x of Figure 3.(A) as well as its upper and lower envelopes, for one interval strategy (A) and for three intervals strategy (B).

Figure 5 :

 5 Figure 5: L 2 norm of m j with respect to j, for one interval strategy (A) and for three intervals strategy (B). (C) and (D): L 2 error of x 1 -m j with respect to j, for one interval strategy (C) and for three intervals strategy (D).

Acknowledgments

The authors are greatly indebted to the organizors of the Colloque Franco-Roumain, Bucarest 2012 and also to the anonymous referee. His supportive criticism was a valuable help.