Brigitte Bigi
email: brigitte.bigi@lpl-aix.fr

The SPPAS participation to Evalita 2011

1 Introduction 522)5 -SPeech Phonetization Alignment and Syllabication, is a new tool to automatically produce annotations which includes utterance, word, syllabic and phonemic segmentations from a recorded speech sound and its transcription.

The whole procedure is a succession of 4 automatic steps. 522)5 is currently implemented for French, English, Italian and Chinese and there is a very simple procedure to add other languages. An important point for software which is intended to be widely distributed is its licensing conditions. 522)5 uses only resources, tools and scripts which can be distributed under the terms of the GPL license. 522)5 tools and resources are currently available at the URL: http://www.lpl-aix.fr/∼bigi/sppas/. 522)5 is based on a dictionary look-ups approach for the phonetization and the use of Julius for grammar based speech alignments. A grammar contains sets of predened combinations of words and contains one or more representations of the distinct phones that make up each word.

EVALITA is an initiative devoted to the evaluation of Natural Language Processing and Speech tools for Italian . In Evalita 2011 the Forced Alignment on Spontaneous Speech task was added. Training data is about 15 map task dialogues recorded by couples of speakers exhibiting a wide variety of Italian variants. During the Forced Alignment test, participants were asked to produce the alignment from a word transcription level of a 89 utterances data set. This working note is related to the 522)5 participation to the Evalita 2011 campaign for the Forced Alignment task. Systems were required to align audio sequences of spoken dialogues to the provided relative transcriptions. We participated to both subtasks: phone segmentation and word segmentation. Two modalities were allowed. We participated to the rst one only, named closed where only distributed data are allowed for training and tuning the system.

The Forced Alignment ' task included both phonetization and alignment tasks. Phonetization is the process of representing sounds by phonetic signs.

Alignment is the process of aligning speech with these sounds. 522)5 is described in section 2. Section 3 is related to the resources we created for the Open-Source Large Vocabulary CSR Engine Julius, rev. 4.1.5, Nagoya Institute of Technology, http://julius.sourceforge.jp/en/ http://www.evalita.it/ SPPAS participation to Evalita 2011: a dictionary and an acoustic model. The evaluation and discussion is presented in section 4. Final results report a correct phoneme alignment rate of 88.4%, and a correct word alignment rate of 96.7%.

SPPAS description

Phonetization

Clearly, there are dierent ways to pronounce the same utterance. Dierent speakers have dierent accents and tend to speak at dierent rates. The SPPAS phonetization process is based on a dictionary solution which consists of storing a maximum of phonological knowledge in a lexicon. Phonetic variants are proposed to the aligner during the alignment stage. The phonetization is then the equivalent of a sequence of dictionary-look-ups. Then, an important step is to build the pronunciation dictionary, where each word in the vocabulary is expanded into its constituent phones. Actually, some words can correspond to several entries in the dictionary with various pronounciations. To deal with such a case, SPPAS determines their correct pronounciation during the alignment step because the pronunciation generally can be observed in the speech. The dictionary contains a set of possible pronouciations of words, including accents as perchè pronounced as /b e r k e/, and reduction phenomena as /p e k/.

Alignment

The alignment problem consists in a time-matching between a given speech utterance along with a phonetic representation of the utterance. The goal is to generate an alignment between the speech signal and its phonetic representation.

SPPAS is based on the Julius Speech Recognition Engine (SRE). Julius was designed for dictation applications, however the Julius distribution only includes Japanese Acoustic Models. But since it uses Acoustic Models trained using the HTK toolkit, it can also use Acoustic Models trained in other languages. To perform alignment a nite state grammar that describes sentence patterns to be recognized and an acoustic model are needed. A grammar essentially denes constraints on what the SRE can expect as input. It is a list of words that the SRE listens for. Each word has a set of associated list of phonemes, extracted the dictionary. When given a speech input, Julius searches for the most likely word sequence under constraint of the given grammar. The alignment task is a 2-steps process: the rst one choose the phonetization and the second one perform the segmentation. Speech alignment requires also an acoustic model in order to align speech. An acoustic model is a le that contains statistical representations of each of the distinct sounds of one language. Each phoneme is represented by one of these statistical representations. Acoustic models were trained with HTK (Hidden Markov Toolkit) by taking the training corpus of speech, previously segmented in utterrances and phonetized. Compared to ideal conditions (i.e. like readed words recorded in an anechoic room) we think this corpus added 3 main diculties: 1/ noisy conditions (the interlocutor recorded in the speaker signal, people walking...) 2/ spontaneous speech (reduction phenomena, hesitations, ...) and 3/ regional accents. During the development stage it was then dicult to analyse which of these phenomena caused errors, and then how to improve algorithms. Consequently, we did not specically adapted our system.

3.2

Phonetic resources for phonetization 522)5 made use of the phoneme set proposed in the dialogues phonetization, except for the # symbol which we renamed gb. We also added a specic phoneme fp to represent lled pauses for words like <eh>, <ah>, etc.

The Italian dictionary was downloaded from the Festival synthetizer tool.The phoneset was automatically modied to match with our Evalita phoneset. This dictionary was enriched by word pronounciations observed in the Evalita train corpus (excluding the extracted development corpus). We corrected manually a large set of these both phonetizations. Finally, the dictionary is made of about 390k words and 5k variants.

Acoustic Model Training

522)5 is based on a common practice and uses context-independent Hidden

Markov models (HMMs) for speech segmentation. The phoneme statistical representation is based on a 5-states model with a left-to-right topology with selfloops and no transitions which skip over states, as represented in Figure 1 with its initial probabilities. We used context-dependent models such as triphones.

The HMM states are modeled by Gaussian mixture densities whose parameters are estimated using an expectation maximization procedure. The outcome of this training procedure is dependent on the availability of accurately annotated 1 0.7 0.6 0.4 0.6 0.4 0.3 Step 3 created tied-state triphones from monophones and from some language specicities dened by the way of a conguration le. This le summarizes Italian phonemic informations as for example the list of vowels, liquids, fricatives, nasals or stop. We created manually this resource (tree.hed).

Evaluation

The most common and direct form of evaluation is comparing the automatic segmentation to a manual segmentation. This evaluation is performed using the proposed phonetizations and alignments (which are supposed to be the right ones) on the development corpus we extracted. The quality of the generated annotation depends largely on the robustness of the HMM recognizer, and on the dictionary. The evaluation we propose is made of 3 stages. First of all we evaluated the alignment only, from the given phonetization. Next, we evaluated the phonetization only. Finally, we evaluated the whole process made of phonetization plus alignment. These evaluations was performed using tools we developped or using Sclite ! . Accuracy was calculated as a function of words or phonemes.

Acoustic model evaluation:

In this experiment, the automatic alignment was estimated on the basis of the manual phonetization. Table 1 proposes detailed alignment performances depending on the delta range between the manual and the automatic alignment, by using the time-location of the middle of each phoneme. The system produced 88.4% of correct alignments with a delta range of 60ms and only 3.6% outliers. .14 86.05 88.43 90.42 91.58 92.39 93.26 96.39 Evaluation was also performed with Sclite using the time-alignment option.

It reports a correct rate of 89.8%, with 7.6% substitutions, 2.6% deletions and 2.6% insertions.

Figure 2 is a boxplot of the most frequent phones. It represents the delta between the phone durations of the automatic alignments and the phone durations of the manual alignments. It shows that in general the automatic system produced vowels with a smaller duration. It also shows that pauses, lled pauses and garbage have the greatest ranges. This is probably due to the recording conditions (noises are frequents) and to some aspects of spontaneous speech. We propose an evaluation of the phonetization only, which represents the availability of our system to propose the good phoneme sequence. This evaluation was estimated with Sclite but without any time-alignment constraints. It reports a correct rate of 89.5%, with 8.1% substitutions, 2.3% deletions and 6.9% insertions. Most frequent errors are due to the garbage manual annotation: in this case the system anyway proposes a phonetization. For example, the sentence A set of these instances can be added to the dictionary but it will not cover all the possible observed realisations. We think this is the main limitation of the dictionary-based phonetization approach.

Conclusion

This working note presented the LPL tool developped to perform the forcedalignment task during the Evalita 2011 campaign, on Italian map-task dialogues.

It described the development of resources and free tools, consisting of acoustic models, phonetic dictionaries, and programs to deal with these data. Figure 3 is a Screenshot of SPPAS alignments. It is important to mention that SPPAS can deal with various languages (French, Chinese) and it was not specically devoted to Italian.

  corpus: train/dev segmentation The train corpus is made of about 15 map-task dialogues recorded by couples of speakers exhibiting a wide variety of Italian variants. Dialogues length ranges from 7/8 minutes to 15/20 minutes. It contains 8063 utterances with word segmentation and phonetic segmentation. The development corpus was automatically extracted from these data and was not used to train the acoustic model. Extracted les are whose corresponding to the following criteria: last 2 utterances of each speaker in each dialogue and all utterances containing from 100 to 106 phonemes. This development corpus was made of 200 utterances; its duration is 12 minutes 04 seconds. It contains 2373 words, 6282 phonemes, including 689 __ and 246 # which represents 14.88% (it is about the same rate in the train corpus).

Fig. 1

 1 Fig. 1. 5-states HMM with initial probabilities

Fig. 2 .

 2 Fig. 2. Phones delta durations for the most frequent phones

  bravissimo a questo is phonetized as b r A V I s I M O A k we s t o but the manual phonetization is b r # s # k we s t o which generates 5 insertion errors and 2 substitutions. The same is for the phonetization d I R E T A M E n t e as the manual one is d # D # n t e that generates 4 insertion errors and 3 substitutions. Many other errors are due to phenomena frequently observed in spontaneous speech. One of the characteristics of Spontaneous Speech is an important gap between a word's phonological form and its phonetic realisations. Specic realisation due to elision or reduction processes (for example perchè pronounced as /b e k/, il videotelefono as /jo d e l e f/) are frequent in spontaneous data. It also presents other types of phenomena such as non-standard elisions, substitutions or addition of phonemes which intervene in the automatic segmentation.

Fig. 3 .

 3 Fig. 3. 522)5 output example

Table 1

 1

	. Alignment-only 522)5 performances
	Delta 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.20 Nb Corr 2513 3929 4712 5160 5406 5555 5680 5753 5804 5859 6055 % Corr 40.02 62.55 75.09 82


Brigitte Bigi
email: brigitte.bigi@lpl-aix.fr

Daniel Hirst
email: daniel.hirst@lpl-aix.fr

SPeech Phonetization Alignment and Syllabification (SPPAS): a tool for the automatic analysis of speech prosody

Keywords: phonetic, annotation, speech prosody, intonation

SPASS, SPeech Phonetization Alignment and Syllabification, is a tool to automatically produce annotations which include utterance, word, syllable and phoneme segmentations from a recorded speech sound and its transcription. SPPAS is currently implemented for French, English, Italian and Chinese and there is a very simple procedure to add other languages. The tool is developed for Unix based platforms (Linux, MaxOS and Cygwin on Windows) and is specifically designed to be used directly by linguists in conjunction with other tools for the automatic analysis of speech prosody. The tools will all be distributed under a GPL license.

Introduction

The last ten years or so has seen an explosion in the quantity of linguistic data which has become available as evidence for the nature of linguistic representations of speech. Whereas it was common some years ago to formulate prosodic models on the basis of rather limited data, today it is becoming more and more expected for linguists to take into account large quantities of empirical data, often including several hours of recorded speech.

Up until a few years ago it was difficult, if not impossible, to manipulate such large quantities of data outside of specialized laboratories using expensive main-frame computers. Software for the analysis of acoustic data was often difficult to install and use, nearly always platform specific and, above all, very expensive.

Today the situation has changed radically and the availability of cheaper and cheaper data storage has made it possible to envisage data analysis involving several hundreds of gigabytes of data. In the same period, a number of software tools for the analysis of audio and/or video recordings of speech have become available such as Anvil [START_REF] Kipp | Anvil, DFKI, German Research Center for Artificial Intelligence[END_REF], Elan [START_REF]Language archiving technology portal[END_REF], Praat [START_REF] Boersma | Praat: Doing phonetics by computer[END_REF], Transcriber [START_REF]TranscriberAG. A tool for segmenting, labeling and transcribing speech[END_REF] and WaveSurfer [START_REF] Wavesurfer | [END_REF], to name just some of the most popular tools, all of which are both free and multi-platform. For an extensive list of speech analysis software see [START_REF] Llisterri | Speech analysis and transcription software[END_REF].

The biggest obstacle today facing linguists is not the storage of data, nor its analysis, but its annotation.

Tools for prosody analysis

The description of the prosodic structure of speech requires:

"the explicit characterization of the length, pitch and loudness of the individual sounds which make up an utterance". [START_REF] Hirst | ProZed: A speech prosody analysis-bysynthesis tool for linguists[END_REF] Linguists need tools for the automatic analysis of at least these three prosodic features of speech sounds. In this paper we concentrate on the tool for the automatic analysis of segmental duration. We return in the final section to the question of interfacing the tool we describe here with the analysis of other prosodic parameters.

The analysis of the prosodic structure of speech nearly always requires the alignment of the speech recording with a phonetic transcription of the speech, usually in some version of the International Phonetic Alphabet. This task is extremely labour-intensive -it may require several hours for even an experienced phonetician to transcribe and align a single minute of speech manually. It is consequently obvious that transcribing and aligning several hours of speech by hand is not generally something which can be envisaged.

A number of tools are currently available which can be used to automate the task, including the HTK Toolkit [START_REF] Young | The HTK Hidden Markov Model Toolkit: Design and Philosophy[END_REF], Festival [START_REF]The Centre for Speech Technology Research[END_REF], Julius [START_REF]Open-Source Large Vocabulary CSR Engine Julius[END_REF], the P2FA [START_REF] Yuan | Speaker identification on the scotus corpus[END_REF], EasyAlign [START_REF] Goldman | EasyAlign: a friendly automatic phonetic alignment tool under Praat[END_REF],

For most of these, our experience is that the tools require a level of expertise in computer science which is beyond the reach of most linguists without the help of an engineer. P2FA, for example, assumes that HTK is installed on the user's computer and it is run via a text-based command-line interface. The EasyAlign Praat plugin [START_REF] Goldman | EasyAlign: a friendly automatic phonetic alignment tool under Praat[END_REF] is currently the most "linguist-friendly" but it currently runs only on Windows and relies on components which cannot all be distributed with a GPL license. EasyAlign is currently available for French and English although the author claims that adding a new language is relatively easy and announces forthcoming versions for German, Dutch and Hebrew.

We are developing a GPL licensed set of tools for Unix based platforms, including Linux, Mac-OSX and Windows (using Cygwin). The tool we describe here implements the automatic phonetization, alignment and syllabification of speech recordings and it is specifically designed to be used directly by linguists in conjunction with other tools for the automatic analysis of speech prosody.

SPPAS description

Overview

SPPAS -SPeech Phonetization Alignment and Syllabification, is a tool to automatically produce annotations which include utterance, word, syllable and phoneme segmentations from a speech recording and its transcription. SPPAS is currently implemented for French, En-glish, Italian and Chinese and there is a very simple procedure to add other languages.

The whole process is a succession of 4 automatic steps. The resulting alignments are a set of TextGrid files, the native file format of the Praat software [START_REF] Boersma | Praat: Doing phonetics by computer[END_REF] which has become the most popular tool for phoneticians today. SPPAS generates separate TextGrid files for (i) utterance segmentation, (ii) word segmentation, (iii) syllable segmentation and (iv) phoneme segmentation. The output is illustrated as a Screenshot in Figure 1, with TextGrid files merged in Praat. For the Chinese example, only 3 tiers are illustrated with the tokens tier corresponding to both syllables and morphemes since the corpus was transcribed in Pinyin.

A important point for software which is intended to be widely distributed is its licensing conditions. SPPAS uses only resources, tools and scripts which can be distributed under the terms of the GPL license. SPPAS tools and resources are currently available at the URL: http://www.lpl-aix.fr/∼bigi/sppas/.

Recently, the software was successfully applied during the Evalita 2011 campaign, on Italian map-task dialogues. EVALITA is an initiative devoted to the evaluation of Natural Language Processing and Speech tools for Italian1 . Systems were required to align audio sequences of spoken dialogues to the provided relative transcriptions. Final results report a correct phoneme alignment rate of 88% (both phonetization and alignment errors), and a correct word alignment rate of 97.6%.

There is a simple procedure to add new languages in SPPAS: it requires only putting a dictionary and an acoustic model into the appropriate directory.

Inter-Pausal Unit Segmentation

Inter-pausal unit segmentationn is a completely open research problem. It consists in aligning sentences of a document with the corresponding sound. Our current implementation of SPPAS includes an algorithm for this step which we hope to improve in future versions of the software.

The algorithm currently implemented identifies silent pauses in the signal and attempts to align them with the inter-pausal units proposed in the transcription, under the assumption that each such unit is separated by a silent pause. For a given minimum duration for pauses and for inter-pausal units a dichotomic search adjusts the silence threshold (in dB) and identifies the number of units thus defined. If the number of units found is greater or less than the desired number, the search is renewed adjusting the minimum duration of the silences and units accordingly. The search halts when the three parameters are correctly adjusted: minimal duration of pauses, minimal duration of units and silence threshold. Silent pauses can be indicated in the transcription either by the symbol '#' or by a newline, or by both together.

A recorded speech file with the .wav extension must correspond to each .txt file. The correspondences are established by means of the file names.

Phonetization

Clearly, there are different ways to pronounce the same utterance. Different speakers have different accents and tend to speak at different rates. Phonetization is the process of representing sounds by phonetic symbols. There are two general ways to construct a phonetization process: rule based systems (with rules based on inference approaches or proposed by expert linguists) and dictionary based solutions which consist of storing a maximum of phonological knowledge in a lexicon. SPPAS implements both approaches for French, but currently only the second for other languages since the rule-based approach is of course highly language dependent.

The program for the phonetization of the orthographic transcription produces a phonetic transcription based on a phonetic dictionary. In the case of phonetic variants, no rules are applied: all possibilities are taken into consideration. The phonetization contains all possible grapheme-to-phoneme conversions of each word of the transcription. If a word cannot be found in the dictionary, the phonetization is "UNK". In this case, the utterance containing the missing word will not be aligned: the missing word will have to be added to the dictionary and SPPAS will need to be re-started.

To perform the phonetization, an important step is to build the pronunciation dictionary, where each word in the vocabulary is expanded into its constituent phones. The phonetization is the equivalent of a sequence of dictionary-look-ups. It supposes that all words of the speech transcription are mentioned in the pronunciation dictionary. Actually, some words can correspond to several entries in the dictionary with various pronunciations.

To deal with such a case, SPPAS decides on the correct pronunciation during the alignment step because the pronunciation can generally be inferred from the speech signal.

For example, the following input sentence: "I never get to sleep on the airplane" will produce the following phonetization: "ay n.eh.v.er g.eh.t|g.ih.t t.uw|t.ix|t.ax s.l.iy.p aa.n|ao.n dh.ax|dh.ah|dh.iy eh.r.p.l.ey.n" using the Arpabet transcription alphabet.

In this phonetization, spaces separate words, dots separate phonemes and pipes (|) separate phonetic variants. The user can either let the system choose the correct phonetization automatically or can force the transcription manually by selecting the appropriate pronunciation of the sentence.

Alignment

Phonetic alignment consists in a time-matching between a speech utterance and a phonetic representation of the utterance. SPPAS is based on the Julius Speech Recognition Engine (SRE) [START_REF]Open-Source Large Vocabulary CSR Engine Julius[END_REF]. For each utterance, the orthographic and phonetic transcriptions are used. Julius performs an alignment to identify the temporal boundaries of phones and words. Julius was originally designed for dictation applications, and its distribution includes only Japanese acoustic models. Since it uses acoustic models trained using the HTK toolkit [START_REF] Young | The HTK Hidden Markov Model Toolkit: Design and Philosophy[END_REF], however, it can also use acoustic models trained for other languages.

To perform alignment, a finite state grammar describing sentence patterns to be recognized and an acoustic model are needed. A grammar essentially defines con- straints on what can be expected as input. It is a list of words that the SRE listens for. Each word has a list of associated phonemes, taken from the dictionary. For a speech input, Julius searches for the most likely word sequence under the constraints of the given grammar.

Speech alignment also requires an acoustic model in order to align speech. An acoustic model is a file that contains statistical representations of each of the distinct sounds of one language. Each phoneme is represented by one of these statistical representations. SP-PAS is based on the use of HTK-ASCII acoustic models. Acoustic models were trained with HTK by taking the training corpus of speech, previously segmented into utterances and phonetized. HTK (Hidden Markov Toolkit) is a portable toolkit for building and manipulating the statistical Hidden Markov models (HMMs) used to represent sound in Speech Recognition. HTK's licence requires registering before it can be downloaded. The software is open source but there are limitations on the distribution of the HTK Toolkit itself. For example, an HTK tool should not be embedded in another tool. There is, however, no limitation on the distribution of the models created with the toolkit, which can consequently be distributed under GPL license.

In the example presented in the previous section, the system chose the following phonetization: "ay n.eh.v.er g.eh.t t.ax s.l.iy.p aa.n dh.ax eh.r.p.l.ey.n".

Syllabification

The syllabification of phonemes is performed with a rulebased system previously described for French in [START_REF] Bigi | Automatic detection of syllable boundaries in spontaneous speech[END_REF]. A new set of rules was developed to deal with Italian and English. In the previous example, the system proposed the following syllabification (spaces separate syllables): "ay n.eh v.er g.eh.t t.ax s.l.iy p aa.n dh.ax eh.r p.l.ey.n".

Resources

Phonetic resources

An important step is building pronunciation dictionaries, where each word in the vocabulary is expanded into its constituent phones.

The Italian dictionary was downloaded from the Festival synthesizer tool [START_REF] Black | The Festival Speech Synthesis System[END_REF]. This dictionary was enriched by word pronunciations observed in the Evalita training corpus. We manually corrected a large set of both these phonetizations. The final version of the dictionary contains about 390k words and 5k variants.

The French dictionary was constructed by merging several available dictionaries. Some word pronunciations were also added using the LIA_Phon tool. It contains about 350k words and 300k variants.

The English dictionary that SPPAS uses is taken from the VoxForge project [START_REF]Voxforge[END_REF], under the terms of the GPL license.

The Chinese dictionary was hand-made. It now contains about 350 tokens and is still in progress. While this number may seem small compared to that of the other languages it should be noted that the inventory of syllables in Standard Chinese is generally held to consist of only about 410 phonemically distinct syllables, not counting tones. The Chinese alignment was carried out on a corpus transcribed with the Pinyin system. The present version does not carry out conversion from Chinese characters to Pinyin but it would be very simple to incorporate such a resource.

Acoustic Model Training

SPPAS is based on common practice and uses contextindependent HMMs for speech segmentation. SPPAS can use various types of acoustic models. This section describes how we created models which are included in SPPAS. The Italian, French and Chinese models were trained following these steps. An English model can be downloaded from the VoxForge web site.

The phoneme statistical representation is based on a 5-state model with a left-to-right topology with self-loops and no transitions skipping over states Ideally, the phones would have unique articulatory and acoustic correlates. But the acoustic properties of a given phone can depend on the phonetic environment. This phenomenon of coarticulation motivated the adoption of context-dependent models such as triphones.

Our training procedure was adapted from the Vox-Forge tutorial. Typically, the HMM states are modeled by Gaussian mixture densities whose parameters are estimated using an expectation maximization (EM) procedure. The outcome of this training procedure is dependent on the availability of accurately annotated data and on good initialization. As more audio speech data is collected, better acoustic models can be created. Acoustic models were trained from 16 bit, 16000 Hz wav files. The Mel-frequency cepstrum coefficients (MFCC), along with their first and second derivatives, were extracted from the speech in the standard way.

The Then, it fixes the "sp" model from the "sil" model by extracting only 3 of the initial 5 model states. This model is then re-estimated using the MFCC files and the phonetization.

Step 3 creates tied-state triphones from monophones and from some language specificities defined by means of a configuration file. This file summarizes phonemic informations such as for example the list of vowels, liquids, fricatives, nasals or stop. We created this resource manually for Italian, French and Chinese.

Adding a new language to SPPAS

SPPAS can deal with a new language L by simply adding the language resources to the appropriate directories:

• the dictionary to: SPPAS/dict/L.dict

• the acoustic model to: SPPAS/models-L

The only step in the procedure which is probably beyond the means of a linguist without external aid is the creation of a new acoustic model when it does not already exist for the language being analyzed. This only needs to carried out once for each language, though, and we plan to provide detailed specifications of the information needed to train an acoustic model on an appropriate set of recordings and dictionaries or transcriptions. Acoustic models obtained by such a collaborative process will be made freely available to the scientific community.

Interfacing with other tools

There are today a number of automatic systems for the automatic modeling and coding of pitch including Momel/INTSINT [START_REF] Hirst | Automatic modelling of fundamental frequency using a quadratic spline function[END_REF][START_REF] Hirst | The analysis by synthesis of speech melody: from data to models[END_REF], which was developed in our lab.

SPPAS and Momel have a number of characteristics in common. Both can be used entirely automatically, with of course the risk of error that that implies, but also the possibility of applying the programs to very large quantities of data. They can also both be run in a supervised mode which allows the user to intervene and to manually correct each intermediate step. Both systems have, moreover, been conceived as language-independent modules -for SPASS only the resources are language dependent, not the tools or the algorithms.

Our current undertaking is to provide these two programs with a common interface so that they can be used together. At present there is, to our knowledge, no generally available algorithm modeling the loudness of speech sounds in a useful way but when and if such algorithms become available, we will of course be pleased to integrate them into our speech analysis environment.

Conclusions

SPPAS is a tool to perform automatic phonetization, alignment and syllabification. Current development is in progress to improve the portability and the accessibility (a GUI is still under development). SPPAS is specifically designed with the aim of providing a tool not for computer-scientists but rather for phoneticians because no such tool is currently available under a GPL licence.

Acknowledgements.

Our thanks to Masahiko Komatsu for allowing us to use the Chinese Multext corpus [START_REF] Komatsu | Chinese MULTEXT: Recordings for a Prosodic Corpus[END_REF] and to Na Zhi for her help in developing the resources for annotating Chinese.

Figure 1 :

 1 Figure 1: SPPAS output example for the utterance 因为我们没有另外的房间. Pinyin: Yīnwèi wǒmen méiyǒu lìngwài de fángjiān (Because we do not have another room)

  acoustic model training procedure comprises 3 main steps. Step 1 is data preparation. It establishes the list of phonemes, plus silence and short pauses. It converts the input data (phonetization) into the HTKspecific data format. It codes the (audio) data (= "parameterizing the raw speech waveforms into sequences of feature vectors") from wav format to MFCC. Step 2 is the generation of monophones. It creates a Flat Start Monophone model by defining a prototype model and copying this for each phoneme. Next, this flat model is re-estimated using the MFCC files to create a new model.

http://www.evalita.it/


Brigitte Bigi
email: brigitte.bigi@lpl-aix.fr

Pauline Péri

Roxane Bertrand

Orthographic Transcription: which Enrichment is required for phonetization?

Keywords: transcription, speech, phonetization

This paper addresses the problem of the enrichment of transcriptions in the perspective of an automatic phonetization. Phonetization is the process of representing sounds with phonetic signs. There are two general ways to construct a phonetization process: rule based systems (with rules based on inference approaches or proposed by expert linguists) and dictionary based solutions which consist in storing a maximum of phonological knowledge in a lexicon. In both cases, phonetization is based on a manual transcription. Such a transcription is established on the basis of conventions that can differ depending on their working out context. This present study focuses on three different enrichments of such a transcription. Evaluations compare phonetizations obtained from automatic systems to a reference phonetized manually. The test corpus is made of three types of speech in French: conversational speech, read speech and political debate. A specific algorithm for the rule-based system is proposed to deal with enrichments. The final system obtained a phonetization of about 95.2% correct (from 3.7% to 5.6% error rates depending on the corpus).

Introduction

The study presented in this paper is part of the OTIM project (also called TOMA -Tools for Multimodal Information Processing) 1 described in [START_REF] Blache | Creating and exploiting multimodal annotated corpora: the ToMA project[END_REF]. The project focuses on the different requirements and needs in the perspective of multimodal annotations. A broadcoverage approach, aiming at annotating a large set of linguistic domains is proposed. The multimodal annotation is faced with the necessity of encoding different information types, from different domains, with different levels of granularity. OTIM aims to develop such a multimodal annotation scheme and tools for face to face interaction. This implies technical and methodological levels to produce high quality multimodal annotations [START_REF] Blache | Multimodal annotation of conversational data[END_REF]. In this field, transcription of the speech signal is the first annotation. Depending on the focus of a study, a transcription can be annotated following various conventions. Even more so, in a multimodal perspective, the transcription has to satisfy the needs and constraints of each domain. The difficulty comes from the fact that each domain investigated a different perspective and had different objectives; researchers interested in morpho-syntax level or in phonetic or prosodic level have not the same needs. Phonetic level is one of the domains annotated in the OTIM project. Phonetization is the process of representing sounds with phonetic signs. There are two general ways to construct a phonetization process: dictionary based solutions which consist in storing a maximum of phonological knowledge in a lexicon and rule based systems with rules based on inference approaches or proposed by expert linguists. In both cases, phonetization is based on the manual transcription. When a speech corpus is transcribed into a written text, the transcriber is immediately confronted with the following question: how to reflect the reality of oral speech in a corpus? Conventions are then designed to pro-1 http://www.lpl-aix.fr/∼otim/ vide a set of rules for writing speech corpora. These conventions establish which phenomena have to be annotated and also how to annotate them.

Numerous studies have been carried otu in prepared speech, as for example for broadcast news (ESTER2, 2008). However, conversational speech refers to an activity more informal, in which participants have constantly to manage and negociate turn-taking, topic (among other things) "on line" without any preparation. As a consequence, numerous phenomena appear such as hesitations, repeats, feedback, backchannels, etc. Other phonetic phenomena such as nonstandard elision, reduction phenomena [START_REF] Meunier | Vowel reduction in conversational speech in french: The role of lexical factors[END_REF], truncated words, and more generally, nonstandard pronunciations are also very frequent. All these phenomena can impact on the phonetization. This paper focuses on phenomena that are mentioned in the transcription and the consequence of this annotation on the quality of the phonetization. The aim was to compare some phonetization approaches based on various transcription enrichments and to answer the question: which speech phenomena are needed to be transcribed to obtain a good phonetization? Section 2. reports three different transcription enrichments, and a description of the corpus used in this study. Sections 3. presents a dictionary-based approach for phonetization (language independent approach), and a rule-based system dedicated to the phonetization of French. This latter system was initially developed to deal with a standard transcription. Section 4. reports a tree-based algorithm that adapts to transcription enrichments. Finally, experiments are reported in Section 5. Evaluation were carried out by comparing the automatic phonetization systems to a manual reference. The hand-made test corpus represents about 7 minutes of speech and is divided into three types of speech: conversational data, read speech and a political debate.

Enriched Orthographic Transcription

Transcription conventions

The transcription process follows specific conventions. The result is what is called an enriched orthographic construction. In this study, three enrichments were selected. The first one represents the text as a standard orthographic written text. For example, if the speech signal includes specific productions like reductions, the transcription must contain the expended written text: for example je suis pronounced as [S4i] instead of the standard pronunciation [Zs4i], and the same for il y a that can be pronounced [iA]. Other specific speech phenomena are also ignored. In the second transcription, transcribers provided an enriched orthographic transcription, which included, for example, manually annotating non-standard events such as: truncated words, laughter, etc. Compared to the previous one, this enriched orthographic transcription includes the following specific speech phenomena:

• short pauses, annotated '+'

• various noises, annotated '*'

• laughter, annotated '@'

• filled pauses, annotated 'euh'

• truncated words, annotated with a '-'

• repeats

Moreover, the transcription was not systematically expanded to the written text: the speech sound [iA] was transcribed as y a. But specific reductions like [S4i] were transcribed as a standard orthographic written text je suis. The third transcription used in this study represented both transcriptions: the orthographic written text (as the previous convention) and, if any, the specific production using an orthographic written text the nearest as possible of what the transcriber could hear. Thereby, from this manual transcription, two derived transcriptions can be generated automatically: the "real orthographic" transcription (the list of orthographic tokens) and a specific transcription from which the obtained phonetic tokens are used by the phonetization system. This latter spelling is called "faked spelling" in this paper. If a token was not modified manually by the transcriber, it is then supposed to be pronounced in a standard way. This is the approach proposed in the OTIM project. These two versions of transcription, synchronized and aligned on the signal, are used either by the morpho syntactic and discourse level, or by the phonetic and prosodic level. Specific productions have a direct consequence on the phonetization procedure:

• Elision is the omission of one or more sounds (such as a vowel, a consonant, or a whole syllable). Nonstandards elisions are explicit in this transcription, manually annotated by parenthesis of the omitted sounds. For example:

j'ai on a j'ai p-(en)fin j'ai trouvé l(e) meilleur moyen c'(é)tait d(e) loger chez des amis 'I've we've I've -well I found the best way was to live in friends' apartment'

Consequently, the phonetizer will not produce phonemes for elision in the words enfin, le, etc. Another word frequently produced with elision is parce que phonetized as /pAsk/ or even /psk/ instead of /pAösk/.

• Transcribers also mentioned particular phonetic realizations by using brackets, such as the pronunciation of specific words, pronounced schwa, etc. For example:

[elle, è] dormait 'to do stats'

• Optional liaisons were also manually mentioned in this enriched transcription.

Test corpus description

To our knowledge, there is no publicly available corpus phonetically transcribed for French that could be used for this study. We thus constructed such a corpus. This annotation was performed by a phonetician, well skilled in the perception and transcription of speech sounds. Phonetic transcription is therefore a very time consuming task.

In parallel, the corpus was transcribed using the three transcription enrichments described previously in this paper.

The test corpus was based on parts of three different French corpora downloaded from the SLDR -Speech & Language Data Repository: http://www.sldr.org About two minutes of each corpora (about 7 minutes altogether) were manually segmented and transcribed.

The first one was extracted from the corpus created during the "Amennpro" project, a Franco-British partnership program that attempts to develop methods of automated evaluation of rhythm in non native speech. Only French speech as spoken by French native speakers were selected. This audio corpora was called AixOx [START_REF] Herment | A multi-layered learners corpus: automatic annotation[END_REF]. This corpus is related to "read speech", as speakers were asked to read paragraphs made of about 3 to 6 sentences. The second part of the test corpus was extracted from CID -Corpus of Interactional Data [START_REF] Bertrand | Le CID -Corpus of Interactional Data[END_REF]. CID is an audio-video recording of 8 hours of spontaneous French dialogues, 1 hour of recording per session. Each dialogue involved two participants of the same gender. One of the following two topics of conversation was suggested to participants: conflicts in their professional environment or funny situations in which participants may have found themselves. Finally, the test corpus contained an extract of a political debate; this corpus was named Grenelle [START_REF] Bigi | Multimodal annotations and categorization for political debates[END_REF] and CID contained a larger number of elisions and special pronunciations. The AixOx corpus that represented read speech was made up a larger set of special pronunciation as it was not expected in this type of speech. This was mainly due to a regional accent of one speaker in the test set. Two examples of each corpus are presented in Tables 3,4 and 5 depending on the transcription annotation (an English translation of these examples is proposed in Table 2). Table 3 is related to the standard orthographic transcription. In case of read speech, this transcription corresponds to the script that participants had to read.

Dictionary-based phonetization

Clearly, there are different ways to pronounce the same utterance. Different speakers have different accents and tend to speak at different rates. A system based on a dictionary solution consists in storing a maximum of phonological knowledge in a lexicon. Phonetic variants are proposed to an aligner to choose the phoneme string. By using this approach, the hypothesis is that the answer to the phonetization question is in the signal. This approach can take as input a standard orthographic transcription and some enrichments only if the acoustic model includes them. Experiments reported in this paper were carried out using SPPAS -SPeech Phonetization Alignment and Syllabification [START_REF] Bigi | SPeech Phonetization Alignment and Syllabification (SPPAS): a tool for the automatic analysis of speech prosody[END_REF]. SPPAS is a tool to produce automatically annotations which includes utterance, word, syllabic and phonemic segmentations from a recorded speech sound and its transcription. The whole procedure is a succession of 4 automatic steps. Resulting alignments are a set of TextGrid files. TextGrid is the native file format of the Praat software which became the most common tool for phoneticians [START_REF] Boersma | Transcription détaillée et enrichie[END_REF]. It is currently implemented for French, English, Italian and Chinese and there is a very simple procedure to add other languages. An important point for software which is intended to be widely distributed is its licensing conditions. SPPAS uses only resources, tools and scripts which can be distributed under the terms of the GPL license. SPPAS tools and resources are freely available at the URL: http://www.lpl-aix.fr/∼bigi/sppas/ To perform the phonetization, an important step is to build the pronunciation dictionary, where each word in the vocabulary is expanded into its constituent phones. The phonetization is the equivalent of a sequence of dictionary lookups. This approach supposes that all words of the speech transcription are mentioned in the pronunciation dictionary otherwise a pronunciation is constructed from inputs of the dictionary using a longuest-matching algorithm. Actually, some words can correspond to several entries in the dictionary with various pronunciations. Thus, the dictionary contains a set of possible pronunciations of each words, including accents, reduction phenomena and liaisons like "je suis":

• /Zs4i/ is the standard pronunciation,

• /Zs4iz/ is the standard pronunciation plus a liaison,

• /Z@s4i/ is the South of France pronunciation,

• /Z@s4iz/ is the South of France pronunciation plus a liaison,

• /S4i/ is a frequent specific realization.

The French dictionary included in SPPAS contains 350k entries and 300k variants. SPPAS determines the pronunciation during the alignment (also called phonetic segmentation) step because the pronunciation generally can be observed in the speech. Phonetic segmentation is the process of aligning speech with its corresponding transcription at the phone level. The alignment problem consists in a time-matching process between a given speech utterance and a phonetic representation of the utterance. The goal is to generate an alignment between the speech signal and its phonetic representation. SPPAS is based on the Julius Speech Recognition Engine (SRE). To perform alignment, a finite state grammar that describes sentence patterns to be recognized and an acoustic model are needed. A grammar essentially defines constraints on what the SRE can expect as input. This is a list of words; and each word has a set of associated list of phonemes, extracted from the dictionary. When given a speech input, Julius searches for the most likely word sequence under constraint of the given grammar. Speech Alignment also requires an Acoustic Model in order to align speech. An acoustic model is a file that contains statistical representations of each of the distinct sounds of one language. Each phoneme is represented by one of these statistical representations. These are called Hidden Markov models (HMMs).

The French acoustic model included in SPPAS (version 1.4) was trained using HTK [START_REF] Young | The HTK Hidden Markov Model Toolkit: Design and philosophy[END_REF]) from 7h30 of conversational speech extracted from CID and 30 minutes of read speech extracted from AixOx (previously segmented in utterances and automatically phonetized).

Rule-based phonetization

Basic phonetization system

The phonetization system used in this study was LIA_Phon [START_REF] Bechet | LIA_PHON -un système complet de phonétisation de textes[END_REF] which is distributed under the term of the GPL license. LIA_Phon contains a set of scripts that transform a raw text into its phonetic form. There are 3 main steps in this process:

1. Formatting the text, 2. POS tagging + accentuation, 3. Grapheme-to-Phoneme transcription.

The POS-tagger aims at the pronunciation disambiguation, for example:

• est 'is' (verb) is pronounced /e/
• est 'east' (noun) is pronounced /Est/ POS tagging was performed using the LIA_TAGG. LIA_TAGG contains a set of scripts in order to clean, format, tagg and bracket French or English texts. It is based on a set of 103 tags using short-cuts like: NMS for the tuple (name, masculine, singular), ADV for adverbs, V3S for the tuple (verb, 3rd person, singular), etc. LIA_Phon and LIA_TAGG tools and resources are freely available at the URL:

http://pageperso.lif.univ-mrs.fr/∼frederic.bechet/ LIA_Phon was conceived to take as input a standard orthographic transcription. The pronunciation was supposed to correspond to a standard French. To deal with the two enrichments of transcriptions (Section 2.), the faked spelling has been sent to the phonetizer. But faked entries were recognized as unknown words and the tagger still had to assign a tag. This necessarily implies tag errors. Thus this could cause phonetization errors, not only on the concerned entry but also on the n previous or following entries due to the use of n-gram models in these tools.

In the following example, the use of the LIA_Phon with a faked spelling produced one phonetization error on the word "dit" which was pronounced [d]. The automatic phonetization was [de], because the faked-word was "d" and it was recognized as a noun and then it was spelled.

Example with a real spelling is: oui ben oui puisque de toute façon il m'a dit il a trouvé un appart et tout là-haut donc c'est que euh 2 2 yeah so anyway because he said he found an apartment and all thereby thus it's that hum Example with a faked spelling is: oui bè oui pusque tfaçon i m'a d il a trouvé un appart et tout là-haut donc c'est qu euh

Tree-based phonetization system

Initially, the phonetization system dealt only with standard orthographic transcriptions. The system could be used with some enrichments like repeats or truncated words because it included a French-specific algorithm for the phonetization of unknown words. The phonetization process was based on the use of a POS-tagger and these phenomena could cause errors.

A tree-based approach is proposed. It consists in sending the real orthographic transcription to the tagger to obtain good tags. Then, the tuple containing the faked spelling plus the tags were sent to the phonetizer. Figure 1 illustrates 2 examples of the use of this algorithm. Gray circles represent nodes of the tree. Nodes can be of types: root, token, laugh, pronunciation, elision, liaison, trunc or pause. Elision and pronunciations could have two children: leftchild was the real orthographic written text and right-child corresponded to the faked spelling. First, the tree was explored to phonetize automatically (and independently) each leaf of type: pause, laugh, trunc and liaison. Then, the tree was explored by using only the left part of each node, and by ignoring pauses, laugh, trunc and liaisons. This sentence was sent to the LIA_TAGG to obtain the POS-tags (orangecolor in the examples). These POS-tags were then copied to right leaves for trunc and pronunciations nodes. The tree was then explored to get the right part of each node and the associated POS-tag (also by ignoring pauses, laugh, trunc and liaisons). Lastly, the tree was explored to obtain the phonetization. This algorithm was implemented in ESPPAS -Enriched-SPPAS, a plugin to SPPAS also available under the terms of the GPL license (for unix-based systems only).

Results

The most common and direct form of evaluation is comparing the automatic phonetization to a manual one. Evaluations were performed using Sclite. Sclite is provided by the National Institute of Standards and Technology (Sclite, 2009). Accuracy is calculated as a function of phonemes, by estimating the sum of the following errors:

• substitution (Sub), example : Ẽ / Ũ

• deletion (Del), example : p @ t i / p t i

• insertion (Ins), example : Z / Z @ Evaluations considered a reduced set of phonemes by combining the following pairs: o/O, e/E, '/i. These 3 cases was related to about 2.7% of substitution errors, independently on the corpus or the transcription.

For the measurement of accuracy rates, the manual phoneme transcription of the test files was compared to:

• the output that SPPAS (version 1.4) system produced (dictionary-based approach),

• the output that LIA_Phon system produced (rule-based approach),

• the output that Enriched-SPPAS system produced (rule-based approach, with a tree-based algorithm). This is the case independently of the corpus, but significantly for CID using LIA_Phon. A detailed analysis shows that major part of error are related to insertions, specially for CID that contained a large set of specific pronunciations due to the type of speech. Deletion errors are also observed because this transcription did not include specific speech phenomena that was not automatically phonetized but that the manual reference included. 8 presents results by transcribing with the first enrichment. It included a set of speech phenomena (silent pauses, filled pauses, repeats, etc.) but not specific realizations. This enrichment allowed automatic systems to produce a significantly better phonetization. The LIA_Phon improved its scores of about 3.0% for CID and AixOx and 1.7% for Grenelle. SPPAS is the better system to deal with conversational speech. The use of the LIA_Phon in a treebased approach produced the same scores as the use of the LIA_Phon directly. The enrichments proposed in the transcription was particularly interesting to reduce deletion errors. However, a large set of deletion errors still occurred in the AixOx phonetization due to the regional accent of one speaker in the test corpus. This speaker added schwas that are not commons in standard French and he pronounced standard elisions. Despite the improvements this enrichment can provide, both automatic phonetization systems produced a large set of errors: compared to the previous one, there was no consequence on substitutions or insertions.

System

System

Algorithm 9 presents results using the second enrichment. The basic idea of this enrichment was to suppose that the pronunciation was standard, except if the manual transcription mentioned something else. The transcriber disambiguated pronunciations. Thus, this enrichment was particularly adapted to automatic rule-based systems which, by default, proposed a standard pronunciation. Even by using this enrichment, the dictionary-based approach did not suppose any kind of pronunciations (except for enriched-entries) and proposed phonetic variants like for the previous enrichments. This is the reason of 1/ the small improvements compared to the previous enrichment and 2/ the lower performances of SPPAS in this experiment compared to the other systems. However, SPPAS used an acoustic model trained on 8h of speech and better results could be potentially expected by using an acoustic model trained from a larger corpus. Unlike rule-based systems, it is also important to note that the algorithms of a dictionary-based approach, as implemented in SPPAS, are completely language-independent (only resources are language-dependent). Performances of the LIA_Phon were significantly improved by using this enrichment, specially for CID where the number of errors was divided by 2. This enrichment allowed the system to reduce insertions errors: divided by 3 for CID and divided by 2.5 for Grenelle. This enrichment also allowed to reduce deletions errors for AixOx (particularly for the speaker with a regional accent). Despite POStags errors (see Section 4.), this phonetization was quitey good, and there was just a little room for improvements. Performances of the tree-based algorithm (also based on the use of LIA_Phon) improved performances of about 7% relative gain for AixOx and Grenelle and about 14% relative gain for CID. Errors that were corrected were mainly concerning insertions. This was essentially due to POS-tag errors that induced the phonetizer to spell words instead of phonetizing them.

Conclusion

This paper examined three transcription enrichments and showed how it impacted on the performances of automatic phonetization. A dictionary-based system (SPPAS) and a rule-based system (LIA_Phon) were compared. Evaluations were carried out by using a French test corpus manually phonetized by an expert, related to three different types of speech: conversational speech, read speech, political debate. Results indicated clearly that the richer transcription the better phonetization. By using a standard transcription, important differences were observed depending on the corpus type (from 8.0% errors to 17.3% error rates using the rule-based system). The latter enrichment supposed that the pronunciation is standard, except if the manual transcription mentioned something else: the transcriber disambiguated pronunciations. Thus, this enrichment was particularly adapted to the automatic rule-based system. Indeed, the manual enrichment allowed the rule-based system to obtain a quite good phonetization. Furthermore, this paper proposed an algorithm that improved this phonetization by using a tree-based approach. The final system obtained a phonetization of about 95.2% correct (from 3.7% to 5.6% error rates depending on the corpus). Finally, the phonetization of Conversational Speech is as good as other types of corpora. Although if the enrichment is more time consuming, it constitutes therefore an effective alternative to phonetize this type of corpus. Such a transcription enrichment is necessary due to the fact that conversational data are still largely unknown.

AixOx -ex1

I opened the front door to let the cat out

AixOx -ex2 send an ambulance to sixteen chadwick close as soon as possible

Grenelle -ex1 to replenish the bee population annually Grenelle -ex2 beekeepers and particularly we do not know very well what is the cause of bee mortality but there are still perhaps systemic attacks

CID -ex1

thus he took the recipe and all well he said okay

CID -ex2

oh but that's just it, it was to sell you blablabla the guy pissed him off and then he bought him the whatsit and then the guy left so i said shit, the guy wanted to...

  ouvert la porte d'entrée pour laisser sortir le chat AixOx -ex2 envoyer d'urgence une ambulance devant le numéro seize de l'impasse Claire Voie Grenelle -ex1 à reconstituer leur cheptel d'abeilles tous les ans Grenelle -ex2 les apiculteurs et notamment on ne sait pas très bien quelle est la cause de mortalité des abeilles mais enfin il y a quand même peut-être des attaques systémiques CID -ex1 donc il prend la recette et tout bon il dit bon okay CID -ex2 ah mais justement c'était pour vous vendre bla bla bla bla le mec il te l'a emboucané en plus il lui a acheté le truc et le mec il est parti je dis putain le mec il voulait Table 3: Standard Orthographic Transcription AixOx -ex1 j'ai ouvert la porte d'entrée pour laisser chortsortir le chat AixOx -ex2 envoyer d'urgence une ambulance devant le numéro seize de l'impasse Claire Voie Grenelle -ex1 à reconstituer + leur cheptel d'abeilles tous les ans Grenelle -ex2 euh les apiculteurs + et notamment bon ne sait pas très bien + quelle est la cause de mortalité des abeilles mais enfin y a quand même peut-être des attaques systémiques CID -ex1 donc + iil prend la èrecette et tout bon il véil dit bon okay CID -ex2 ah mais justement c'était pour vous vendre bla bla bla blle mec il te l'a emboucané en plus il lui a acheté le truc et le mec il est parti je dis putain le mec il voulait Table 4: Orthographic Transcription with the first enrichment AixOx -ex1 j'ai ouvert la porte d'entrée pour laisser chortsortir le chat AixOx -ex2 envoyer d'urgence une [ambulance,ambulanceu] devant [le,leu] numéro [seize,seizeu] de l' [impasse,impasseu] [Claire Voie,claireuvoi] Grenelle -ex1 à [reconstituer,reuconstituer] + leur cheptel d'abeilles tous les ans Grenelle -ex2 euh les apiculteurs + et notamment bon n(e) sait pas très bien + quelle est la cause de mortalité des abeilles m(ais) enfin y a quand même peut-êt(r)e des attaques systémiques CID -ex1 donc + i-i(l) prend la èrecette et tout bon i(l) vé-i(l) dit bon [okay, k] CID -ex2 ah mais justement c'était pour vous vendre bla bla bla blle mec i(l) te l'a emboucané en plus i(l) lu(i) a [acheté,acheuté] le truc et le mec il est parti j(e) dis put(ain) le mec i(l) voulait

Table 1 :

 1 . Grenelle concerns a political debate on environmental issue recorded at the French National Assembly on the 4th of May 2010. While AixOx and CID have been recorded in a sound attenuated room, Grenelle has been recorded in a naturalistic environment.

		CID	AixOx Grenelle
	Duration	143s	137s	134s
	Number of speakers	12	4	1
	Number of phonemes	1876	1744	1781
	Number of tokens	1269	1059	550
	Silent pauses	10	23	28
	Filled pauses	21	0	5
	Noises (breathes,...)	0	8	0
	Laughter	4	0	0
	Truncated words	6	2	1
	Optional liaisons	4	2	5
	Elisions (non stds)	60	21	34
	Special Pron.	58	37	23

Test corpus description

Table 1 reports a detailed description of the test corpus. It is important to mention that the three corpora were equally represented (at least for the present study): about the same duration and the same number of phonemes. This corpus represented the expected phenomena like silent pauses or truncated words. Moreover it contained the expected rate of these phenomena depending on the speech types. Particularly, conversational data (CID) compared to both other speech types included a very high rate of hesitations. CID also contained laughter but not in the other corpora;

  Table6illustrates tag errors that are produced by the use of a faked orthograph directly in the LIA_TAGG. For this sentence (extracted from CID corpus), about 32% of entries obtained a wrong tag.

	Real	POS-Tag	Faked	POS-Tag	Tag
	Ortho.		Ortho.		Error
	oui	ADV	oui	ADV
	ben	ADV	bè	AFS	X
	oui	ADV	oui	ADV
	puisque COSUB	pusque AFS	X
	de	PREPADE			X
	toute	DETFS			X
	façon	NFS	tfasson NFS
	il	PPER3MS i	NMS	X
	m'	PPOBJMS m'	PPOBJMS
	a	VA3S	a	VA3S
	dit	VPPMS	d	NMS	X
	il	PPER3MS il	PPER3MS
	a	VA3S	a	V3S
	trouvé	VPPMS	trouvé VPPMS
	un	DETMS	un	DETMS
	appart	NMS	appart	NMS
	et	COCO	et	COCO
	tout	ADV	tout	ADV
	là-haut	ADV	là-haut ADV
	donc	COCO	donc	COCO
	c'	PPER3MS c'	PPER3MS
	est	VE3S	est	VE3S
	que	COSUB	qu	AMS	X
	euh	ADV	euh	ADV
	Table 6: LIA_TAGG outputs depending on the spelling
	given as input
	A suitable adaptation of such a tool to deal with enriched
	orthographic transcriptions is proposed in this paper.

Table 7 :

 7

		Sub Del Ins	Err
	AixOx
	Standard Trs Lia_Phon	1.4 5.0	3.0	9.5
	Standard Trs SPPAS 1.4 3.6 4.5	2.8 10.8
	Grenelle
	Standard Trs Lia_Phon	1.1 2.8	4.1	8.0
	Standard Trs SPPAS 1.4 2.3 2.8	3.8	8.8
	CID
	Standard Trs Lia_Phon	2.8 4.5 10.0 17.3
	Standard Trs SPPAS 1.4 3.6 4.9	6.0 14.5

Phonetization errors (in %), obtained from a standard transcription Results using the standard orthographic transcription are presented in Table

7

. For both systems, the automatic phonetization is very different from what it is expected.

Table 8 :

 8 Phonetization errors (in %), obtained with the first enriched transcription

	Sub Del Ins	Err

Table

Table 9 :

 9 Phonetization errors (in %), obtained with the second enriched transcription Table

	Sub Del Ins Err

Table 5 :

 5 Orthographic Transcription with the second enrichment

Acknowledgements

This work was supported by ANR OTIM project Ref. Nr. ANR-08-BLAN-0239.


Brigitte Bigi
email: brigitte.bigi@lpl-aix.fr

SPPAS: a tool for the phonetic segmentation of speech

Keywords: phonetic, annotation, segmentation

SPPAS is a tool to produce automatic annotations which include utterance, word, syllabic and phonemic segmentations from a recorded speech sound and its transcription. SPPAS is distributed under the terms of the GNU Public License. It was successfully applied during the Evalita 2011 campaign, on Italian map-task dialogues. It can also deal with French, English and Chinese and there is an easy way to add other languages. The paper describes the development of resources and free tools, consisting of acoustic models, phonetic dictionaries, and libraries and programs to deal with these data. All of them are publicly available.

Introduction

The last ten years or so have witnessed an explosion in the quantity of linguistic data which have become available as evidence for the nature of linguistic representations of speech. Whereas it was common some years ago to formulate phonetic models on the basis of rather limited data, today it is becoming more and more expected for linguists to take into account large quantities of empirical data, often including several hours of recorded speech. Up until a few years ago it was difficult, if not impossible, to manipulate such large quantities of data outside specialized laboratories using expensive main-frame computers. Software for the analysis of acoustic data was often difficult to install and use, nearly always platform specific and, above all, very expensive. Today, the situation has radically changed and the availability of cheaper and cheaper data storage has made it possible to envisage data analysis involving several hundreds gigabytes data. In the same period, a number of software tools for the analysis of audio and/or video recordings of speech have become available such as Anvil (Kipp, 2001), Elan (Sloetjes et al., 2010), Praat (Boersma and Weenink, 2009), Transcriber (Tran-scriberAG, 2011) and WaveSurfer (WaveSurfer, 2012), to name just some of the most popular tools, all of which are both free and multi-platform. For an extensive list of speech analysis software see (Llisterri, 2011). The biggest obstacle linguists are faced with today is not the storage of data, nor its analysis, but its annotation. The analysis of the phonetic entities of speech nearly always requires the alignment of the speech recording with a phonetic transcription of the speech. This task is extremely labour-intensive -it may require several hours for even an experienced phonetician to transcribe and align a single minute of speech manually. It is consequently obvious that transcribing and aligning several hours of speech by hand is not generally something which can be envisaged. A number of tool boxes are currently available which can be used to automate the task, including the HTK Toolkit (Young and Young, 1994), Sphinx (Carnegie Mellon University, 2011), or Julius (Nagoya Institute of Technology, 2010). For most of these, our experience is that the tools require a level of expertise in computer science which is beyond the reach of most linguists without the help of an engineer. Some wrapper tools propose to simplify the use of HTK, as for example P2FA (Yuan and Liberman, 2008) or EasyAlign (Goldman, 2011). The EasyAlign Praat plugin is currently the most "linguist-friendly" but it currently runs only on Windows. Moreover, due to the HTK license limitations, both tools assume that HTK is installed on the user's computer. SPPAS is a tool to produce automatic annotations which include utterance, word, syllabic and phonemic segmentations from a recorded speech sound and its transcription. The whole procedure is a succession of 4 automatic steps and resulting alignments are a set of TextGrid files. TextGrid is the native file format of the Praat software which became one of the most common tool for phoneticians (Boersma and Weenink, 2009). SPPAS is implemented with python and was tested under Linux, Mac-OSX and Windows R . SPPAS is currently designed for French, English, Italian and Chinese and there is an easy way to add other languages. Recently, it was successfully applied to the forced alignment task during the Evalita 2011 campaign, on Italian map-task dialogues. Evalita is an initiative devoted to the evaluation of Natural Language Processing and Speech tools for Italian1 . Systems were required to align audio sequences of spoken dialogues to the provided relative transcriptions. This tool and all resources are distributed under the terms of the GPL license at the URL: http://www.lpl-aix.fr/∼bigi/sppas/ SPPAS is also designed to be used directly by linguists. Current developments consist in integrating Momel (Hirst and Espesser, 1993) and Intsint (Hirst and Di Cristo, 1998). Section 2. of this paper presents an overview of what SP-PAS (version 1.4) can do. Section 3. describes resources included in SPPAS. Finally, Section 4. gives details about the SPPAS architecture.

SPPAS overview

Main description

SPPAS can be executed using the GUI -Graphical User Interface, or by using tools in a console-mode. Figures 6 and7 show the GUI of version 1.3 and version 1.4 respectively. To execute SPPAS, the user just need to double-click on the SPPAS main program. A wav file or a directory containing a set of wav files and the corresponding transcriptions have to be selected. Then, the user selects the appropriate language and modules to execute. The Preferences button allows to configure these modules (to fix options). Figure 1 summarizes the workflow, with all possible inputs/outputs. Inputs are represented in yellow boxes and black arrows. Outputs are represented with the green boxes and the green arrows. SPPAS can be used in a completely automatic mode. It can also be used semi-automatically, by respecting file name conventions and transcription conventions.

Figure 1: SPPAS workflow

History of the versions

The first version of SPPAS was released on March 9, 2011. This version was made only of tcsh and gawk scripts and it could be used only in a console mode. In this first version, no resources were included. Some resources were added in version 1.1 (in June, 2011). The next release was distributed on July 23, 2011. This version included resources to deal with English, French, Italian (read speech, monophones) and Chinese. MacOS support was added in version 1.3, and the first GUI was included in this version. Italian acoustic models and the dictionary was changed and the French acoustic models was improved. Version 1.4 will to be released on May, 2012. This version will be entirely implemented with python and far easier to install than previous versions. The present paper describes this version.

Inter-Pausal Unit Segmentation

Inter-pausal unit segmentation is an open research problem. It consists in aligning macro-units of a document with the corresponding sound, as illustrated in Figure 2. The implementation of SPPAS -version 1.4, includes an algorithm for this step which we hope to improve in future versions of the software. In SPPAS, silent pauses can be indicated in the transcription either by the symbol '#' or by a newline, or both. A recorded speech file with the .wav extension must correspond to each .txt file. The correspondences are established by means of the file names. The algorithm currently implemented identifies silent pauses in the signal and attempts to align them with the inter-pausal units proposed in the transcription, under the assumption that each such unit is separated by a silent pause. For a given minimum duration for pauses and for inter-pausal units a dichotomic search adjusts the silence threshold (in dB) and identifies the number of units thus defined. If the number of units found is greater or smaller than the desired number, the search is renewed adjusting the minimum duration of the silences and units accordingly. The search halts when the three parameters are correctly adjusted: minimal duration of pauses, minimal duration of units and silence threshold. This ipu-segmentation was used to align read speech in two projects. The first one consisted in a set of recordings of 40 English words. SPPAS succeeded to align all files. The second one consisted of a set of recordings of paragraphs made of 3 to 6 sentences. These are part of the AixOx corpus (Herment et al., 2012). In this case, SPPAS failed to align about 10% of the files and 40% of the sentence boundaries had to be manually corrected. Errors were only due to the fact that silences were not at the end of sentences as it was supposed to be in the script speakers had to read. To perform this ipu-segmentation, and despite these errors, the use of SPPAS saved time.

Phonetization

Phonetization is the process of representing sounds with phonetic signs. There are two general ways to construct a phonetization process: dictionary based solutions which consist in storing a maximum of phonological knowledge in a lexicon and rule based systems with rules based on inference approaches or proposed by expert linguists. In both cases, phonetization is based on a manual transcription. Clearly, there are different ways to pronounce the same utterance. Different speakers have different accents and tend to speak at different rates. When a speech corpus is transcribed into a written text, the transcriber is immediately confronted with the following question: how to reflect the orality of the corpus? Conventions are then designed to provide rules for writing speech corpora. These conventions establish phenomena to annotate and also how to annotate them.

A system based on a dictionary solution consists in storing a maximum of phonological knowledge in a lexicon. In this sense, this approach is language-independent unlike rulebased systems. The SPPAS module for the phonetization of the orthographic transcription produces a phonetic transcription based on a phonetic dictionary. The phonetization is the equivalent of a sequence of dictionary look-ups. It is assumed that all words of the speech transcription are mentioned in the pronunciation dictionary. Otherwise, SPPAS implements a language-independent algorithm to phonetize unknown words. At this stage, it consists in exploring the unknown word from left to right and to find the longuest strings in the dictionary. Since this algorithm uses the dictionary, the quality of such a phonetization will depend on this resource. Actually, some words can correspond to several entries in the dictionary with various pronunciations. Unlike rulebased systems, in dictionary-based solutions the pronunciation is not supposed to be "standard". Phonetic variants are proposed for the aligner to choose the phoneme string. The hypothesis is that the answer to the phonetization question is in the signal. For example, the French sentence je suis can be pronounced:

• /Zs4i/ is the standard pronunciation,

• /Zs4iz/ is the standard pronunciation plus a liaison,

• /Z@s4i/ is the South of France pronunciation,

• /Z@s4iz/ is the South of France pronunciation plus a liaison,

• /S4i/ is a frequent specific realization.

The corresponding dictionary entries for both words are:

je [je] Z je(2) [je] Z@ je(3) [je] S suis [suis] s4i suis(2) [suis] s4iz suis(3) [suis] sui suis(4) [suis] 4i suis(5) [suis] 4iz
SPPAS can take as input a tokenized standard orthographic transcription and some enrichments only if the acoustic model includes them. For example, the French transcriptions can contain laugh (represented by the symbol '@' in the transcription). An example of SPPAS phonetization output is presented in Figure 3 on a French sentence. By using SPPAS in a semiautomatic way, this phonetization can be manually modified. The SPPAS conventions are:

• spaces separate tokens,

• dots separate phonemes,

• pipes separate phonetic variants.

Thus, the phrase "je suis" is phonetized as:

Z|Z.@|S s.4.i|s.4.i.z|s.u.i|4.i|4.i.z

Alignment

Phonetic segmentation is the process of aligning speech with its corresponding transcription at the phone level. The alignment problem consists in a time-matching between a given speech unit along with a phonetic representation of the unit. The goal is to generate an alignment between the speech signal and its phonetic representation. SPPAS is based on the Julius Speech Recognition Engine (SRE) (Nagoya Institute of Technology, 2010) for three reasons: 1/ it is easy to install which is important for users; 2/ it is also easy to use then easy to integrate in SPPAS; and 3/ its performances correspond to the state-of-the-art of HMMbased systems and are quite good. Julius was designed for dictation applications, however the Julius distribution only includes Japanese Acoustic Models. But since it uses Acoustic Models trained using the HTK toolkit (Young and Young, 1994), it can also use Acoustic Models trained for other languages. Initially, Julian was a special version of Julius that performed grammar based speech recognition.

The release 4 merged Julius and Julian.

To perform alignment, a finite state grammar that describes sentence patterns to be recognized and an acoustic model are needed. A grammar essentially defines constraints on what the SRE can expect as input. It is a list of words that the SRE listens to. Each word has a set of associated list of phonemes, extracted from the dictionary. When given a speech input, Julius searches for the most likely word sequence under constraint of the given grammar. For example, the sentence "we are reading" will produce the following list of words with associated pronunciations (extracted from the CMU pronunciation dictionary):

0 [w_0] w iy 0 [w_0] w ih 1 [w_1] aa r 1 [w_1] er 2 [w_2] r eh dx ix ng 2 [w_2] r iy dx ix ng
The Julius corresponding grammar file is defined as:

0 2 1 0 1 1 1 2 0 0 2 0 3 0 0 3 -1 -1 1 0
The grammar indicates that each word of the dictionary has to be aligned, from word 0 to word 1, from word 1 to word 2, and so on. The Julius alignment task is a 2-step process: the first step chooses the phonetization and the second step performs the segmentation. Speech Alignment also requires an Acoustic Model in order to align speech. An acoustic model is a file that contains statistical representations of each of the distinct sounds of one language. Each phoneme is represented by one of these statistical representations. SPPAS is based on the use of HTK-ASCII acoustic models. Then, acoustic models were trained with HTK by taking the training corpus of speech, previously segmented in to utterances and phonetized. HTK (Hidden Markov Toolkit) is a portable toolkit for building and manipulating the statistical models used to represent sound in Speech Recognition. These are called Hidden Markov models (HMMs). HTK's licence requires registering before it can be downloaded. The software is open source but there are limitations on the distribution of the HTK Toolkit itself. For example, an HTK tool can not be embedded in an other tool. However, there is no limitation on the distribution of the models created with the toolkit. This model can be distributed under GPL license. Figure 4 illustrates a SPPAS alignment output on a French sentence. The alignment of words is deduced from their alignment into phonemes.

Figure 4: Example of alignment (French)

Syllabification

The syllabification of phonemes is performed with a rulebased system previously described for French in (Bigi et al., 2010). This RBS phoneme-to-syllable segmentation system is based on 2 main principles:

1. a syllable contains a vowel, and only one.

2. a pause is a syllable boundary.

These two principles focus the problem of the task of finding a syllabic boundary between two vowels. As in state-ofthe-art systems, phonemes were grouped into classes and rules established to deal with these classes. This system was applied for the labelling of CID (Bertrand et al., 2008). Evaluations were carried out by using a subset of 7 minutes. Two experts labelled this corpus manually. The syllable agreement rate was 97.8% between experts. This agreement was 95.8% and 94.9% between automatic system and each expert.

A new set of rules was developed to deal with Italian. At present there is, to our knowledge, no generally available corpus manually labelled to evaluate such a syllabification. We will of course be pleased to perform this evaluation as soon as we can obtain one.

SPPAS Resources

Phonetic resources

An important step is building pronunciation dictionaries, where each word in the vocabulary is expanded into its constituent phones.

The dictionary file format is the same as that used in the HTK toolkit (Young and Young, 1994) and in the VoxForge project (VoxForge, 2006(VoxForge, 2011)), as for example:

A [A] a A(2) [A] wa A(3) [A] ja ACQUA [ACQUA] a k wa ACQUA(2) [ACQUA] k wa
The dictionary contains a set of possible pronunciations for words, including accents as perchè pronounced as /b e r k e/, and reduction phenomena as /p e k/ or /k wa/ for the word acqua.

The Italian dictionary was downloaded from the Festival synthetizer tool (The Centre for Speech Technology Research, 2011). This dictionary was enriched by word pronunciations observed in the Evalita train corpus. A large set of both phonetizations was manually corrected. Finally, the dictionary contains about 390k words and 5k variants.

The French dictionary was constructed by merging several free dictionaries. Some word pronunciations were also added using the LIA_Phon tool. It contains about 350k words and 300k variants. The English dictionary SPPAS deals with was downloaded from the VoxForge project (VoxForge, 2006(VoxForge, 2011)).

The Chinese dictionary was hand-made. It now contains about 350 entries and is still in progress.

Acoustic Model (AcM) Training

SPPAS is based on a common practice and uses contextindependent HMMs for speech segmentation. SPPAS can use various types of acoustic models. This section describes how we created models which are included in SPPAS. The Italian, French and Chinese models were trained following these steps. An English model can be downloaded from the VoxForge web site. Step 3 creates tied-state triphones from monophones and from some language specificities defined by the way of a configuration file. This file summarizes phonemic informations as for example the list of vowels, liquids, fricatives, nasals or stops. This resource was created manually for Italian, French and Chinese and are available ondemand.

SPPAS Architecture

SPPAS 1.4 packages

SPPAS is implemented using the programming language Python. This choice was motivated by many reasons.

First of all, "All Python releases are Open Source (see http://www.opensource.org/ for the Open Source Definition). Historically, most, but not all, Python releases have also been GPL-compatible" (http://docs.python.org/license.html). For many operating systems, Python is a standard component. Thanks to being scripting language with module architecture, syntax SPPAS can deal with a new language L by simply adding the language resources to the appropriate directories:

• the dictionary to: SPPAS/dict/L.dict

• the acoustic model to: SPPAS/models/models-L

• the syllabification rules to: SPPAS/syll/syllConfig-L.txt

By using the GUI, SPPAS dynamically creates the list of available languages by exploring appropriate directories.

Steps using language-dependent resources (phonetization, alignment and syllabification) are activated or disabled depending on the directory contents.

SPPAS 1.4 modules

SPPAS library is placed in the "lib" directory. Shared modules are placed directly in this directory. It is the case for example of the Class "sppasLog" that handles the log file created by SPPAS during each run. Sub-directories are used to separate SPPAS modules. For each one of these modules, a set of classes is implemented:

wav This module contains a set of classes dealing with wav files: the main class "Wave", the class "WavePitch", the class "WaveSil".

trs This module contains a set of classes dealing with a "Transcription" represented as a set of " phon This module contains a set of classes dealing with the phonetization problem. The class "DictPhon" is the main class to perform phonetization; "PhonUnk" is related to the phonetization of unknown words; and "PhonTrs" uses DictPhon on a whole tier.

align This module contains a set of classes dealing with the alignment problem. It is a 3-steps process: 1/ the transcription and related wav files are split into units; 2/ each unit is aligned; then 3/ unit alignments are merged in a transcription tier and saved.

Step 2 is a little bit more complicated than it can look at first. We encountered 2 difficulties using Julius. Firstly, a unit is not aligned if a triphone is missing in the tiedlist. We implemented a "TiedList" class that adds the unobserved triphone into the tiedlist. Secondly, we observed that the alignment failed for about 3-5% of units. In these cases, Julius performs its 1st step properly (it chooses the phonetization, depending on the grammar) but its 2nd step (the segmentation) fails. To deal with these errors (and to produce a result instead of nothing), we implemented a function that uses the same duration for each phone of the unit. gui All classes related to the graphical user interface.

term All classes related to the terminal user interface.

These modules constitute the API -Application Programming Interface, of SPPAS.

SPPAS 1.4 tools

The main tool to work with SPPAS is sppas.py that executes the GUI or that can be used in command-line mode. However, this latter usage does not let the possibility to fix specific options for each module. A set of tools is then available: they propose main programs to deal with the SP-PAS API. For each one of these tools, a large set of options can be fixed. SPPAS 1.4 tools are:

wavsplit.py is the main tool to perform IPU-segmentation. This tool enables to find silences from a wav input file, depending on 4 parameters:

• the window width (used to estimate rms) in seconds,

• the minimal speech units duration in seconds,

• the minimal silence duration in seconds,

• the volume in dB.

The silence research can also be controlled by using one of these options: fix a number of units or set a transcription file. This tool can provide three different outputs:

• a directory containing units as a set of wav files and a set of text files with the corresponding transcription,

• a text file containing the list of start time and end time of each unit,

• a TextGrid file with silences/units segmentation.

phonetize.py is the main tool to perform phonetization. This tool takes as input a transcription in the form of a tier in a TextGrid file, and a dictionary. It produces a tier containing the phonetization that optionally can be added to the intput or written in a separate TextGrid file. For units containing unknown words, this tool can optionally use an internal algorithm to phonetize it or it can use the label "UNK".

alignment.py is the main tool to perform alignment. This tool takes as input a phonetization in the form of a tier in a TextGrid file, and an acoustic model. It produces 2 tiers containing the alignments (words and phonemes) that optionally can be added to the intput or written in 2 separate TextGrid files.

LPL-Syllabeur-2.2.jar is the main tool to perform syllabification. This tool takes as input a phoneme alignment in the form of a tier in a TextGrid file, and a configuration file (with a set of rules). It produces a tier containing syllables time-aligned whichnis written in a separate TextGrid file.

momel.py is the SPPAS implementation of the Momel tool. This tool allows to find pitch targets from a wav input file. The Intsint tool can be optionally activated. This tool produces 2 point tiers containing the momel targets and the Intsint labels that can be optionally added to the intput or written in a separate TextGrid file.

SPPAS 1.4 also contains a set of utility tools to deal with wav files, as for example:

• wavcut.py is used to cut a wav file by using a start time and a duration.

• wavstats.py is used to obtain some statistics about a wav input file.

• wav2intensity.py is used to create an intensity tier from a wav input file.

Evaluations

A large set of evaluations were carried on the phonetization of French (Bigi et al., 2012). Evaluations on Italian were also carried out during the Evalita evaluation campaign: SPPAS participated to the "Forced Alignment on Spontaneous Speech". Both phonetization and alignment (phonemes and words) were evaluated (Bigi, 2012).

Conclusion

SPPAS is a tool to perform automatic phonetic segmentations. SPPAS is not only dedicated to computer-scientists.

It is rather dedicated to phoneticians because such a tool

does not yet exist under GPL license. The only step in the procedure which is probably beyond the means of a linguist without external aid is the creation of a new acoustic model when it does not yet exist for the language being analysed. This only needs to carried out once for each language, though, and we plan to provide detailed specifications of the information needed to train an acoustic model on an appropriate set of recordings and dictionaries or transcriptions. Acoustic models obtained by such a collaborative process will be made freely available to the scientific community. Version 1.4 deals with TextGrid files. Future development will consist in improving portability to other transcription tools as Transcriber or Elan. Other automatic annotation modules are still in progress: Momel and Intsint are currently implemented. A multilingual tokenizer will also be integrated complying with the architecture proposed in (Bigi, 2011). Finally, we plan to add some languages and to improve the Chinese resources.

Figure 2 :

 2 Figure 2: Example of IPU-segmentation

Figure 3 :

 3 Figure 3: Example of Phonetization (French)

Figure 5

 5 Figure 5: 5-states HMM with initial probabilities

momel

  This module contains the SPPAS implementation of the Momel (Hirst and Espesser, 1993) and Intsint (Hirst and Di Cristo, 1998) tools. Momel (Modelling melody) is an algorithm for the analysis and synthesis of intonation patterns. INTSINT is an acronym for INternational Transcription System for INTonation. INTSINT codes the intonation of an utterance by means of an alphabet of 8 discrete symbols constituting a surface phonological representation of the intonation: T (Top), H (Higher), U (Upstepped), S (Same), M (mid), D (Downstepped), L (Lower), B (Bottom).

Figure 6 :

 6 Figure 6: SPPAS GUI, version 1.3 (based on pyGtk)

  simplicity and rich text processing tools, Python is objectoriented programming. SPPAS version 1.4 package content is detailed below:

	README	Several general information
	CHANGES	Versions tracking
	COPYRIGHT	GPL license
	dict	Pronunciation dictionaries
	FR.dict	French
	IT.dict	Italian
	EN.dict	English
	ZH.dict	Simplified Chinese
	models	Acoustic models
	models-FR	French: hmmdefs and tiedlist
	models-IT	Italian: hmmdefs and tiedlist
	models-EN	English: hmmdefs and tiedlist
	models-ZH	Simplified Chinese: hmmdefs
	syll	Syllabification configurations
	syllConfig-FR.txt	French
	syllConfig-IT.txt	Italian
	samples	A set of samples
	...	(wav and transcriptions)
	samples-FR	French set of examples
	samples-IT	Italian set of examples
	samples-EN	English set of examples
	samples-ZH	Simplified Chinese
	lib	SPPAS library
	param.py	sppasParam Class
	log.py	sppasLog Class
	wav
	trs
	phon
	align
	momel
	gui
	term
	tools	SPPAS tools
	wavsplit.py
	phonetize.py
	alignment.py
	LPL-Syllabeur-2.2.jar
	momel.py
	...
	sppas.command	Bash to execute SPPAS
		(Unix-based systems)
	sppas.py	Main SPPAS program

http://www.evalita.it/

Acknowledgements

Our thanks to Masahiko Komatsu for allowing us to use the Chinese Multext corpus and to Na Zhi for her help in developing the resources for annotating Chinese. This work was supported by ANR OTIM project Ref. Nr. ANR-08-BLAN-0239.

R. Bertrand, P. Blache, R. Espesser, G. Ferré, C. Meunier, B. Priego-Valverde, and S. Rauzy. 2008. Le CID -Corpus of Interactional Data. Traitement Automatique des Langues, 49(3):105-134. B. Bigi, C. Meunier, I. Nesterenko, and R. Bertrand. 2010. Automatic detection of syllable boundaries in spontaneous speech. In Language Resource and Evaluation Conference, pages 3285-3292, La Valetta (Malta). B. Bigi, P. Péri, and R. Bertrand. 2012. Orthographic Transcription: Which Enrichment is required for Phonetization? In The eighth international conference on Language Resources and Evaluation, Istanbul (Turkey). B. Bigi. 2011. A multilingual text normalization approach. In 2nd Less-Resourced Languages workshop, 5th Language & Technology Conference, Poznàn (Poland). B. Bigi. 2012. The SPPAS participation to Evalita 2011. In Working Notes of EVALITA 2011, ISSN: 2240-5186, Roma (Italy). P. Boersma and D. Weenink. 2009. Praat: doing phonetics by computer, http://www.praat.org. Carnegie Mellon University. 2011. CMUSphinx: Open Source Toolkit For Speech Recognition. http://cmusphinx.sourceforge.net. J.-Ph. Goldman. 2011. EasyAlign: an automatic phonetic alignment tool under Praat. In InterSpeech, Florence (Italy). S. Herment, A. Loukina, A. Tortel, D. Hirst, and B. Bigi. 2012. A multi-layered learners corpus: automatic annotation. In 4th International conference on corpus linguistics Language, corpora and applications: diversity and change, Jaen (Spain). D.-J. Hirst and A. Di Cristo. 1998. Intonation Systems. A survey of Twenty Languages. D.-J. Hirst and R. Espesser. 1993. Automatic modelling of fundamental frequency using a quadratic spline function. Travaux de l'Institut de Phonétique d'Aix, 15:75-85. M. Kipp. 2001. Anvil -a generic annotation tool for multimodal dialogue. In 7th European Conference on Speech Communication and Technology, pages 1367-1370, Scandinavia. J. Llisterri. 2011. Speech analysis and transcription software. Nagoya Institute of Technology. 2010. Open-source large vocabulary csr engine julius, rev. 4.1.5. H. Sloetjes, A. Russel, and A. Klassmann. 2010. Elan: a free and open source multimedia annotation tool. The Centre for Speech Technology Research. 2011. The festival speech synthesis system. TranscriberAG. 2011. A tool for segmenting, labeling and transcribing speech. [computer software] paris: Dga. http://transag.sourceforge.net/. VoxForge. 2006-2011. http://www.voxforge.org. WaveSurfer. 2012. http://www.speech.kth.se/wavesurfer/. S.J. Young and Sj Young. 1994. The htk hidden markov model toolkit: Design and philosophy. Entropic Cambridge Research Laboratory, Ltd, 2:2-44. J. Yuan and M. Liberman. 2008. Speaker identification on the SCOTUS corpus. In Acoustics.