Benjamin Lecouteux

Georges Linarès

Stanislas Oger

Integrating imperfect transcripts into speech recognition systems for building high-quality corpora

Keywords: Speech processing, acoustic model training, text-to-speech alignment

The training of state-of-the-art automatic speech recognition (ASR) systems requires huge relevant training corpora. The cost of such databases is high and remains a major limitation for the development of speech-enabled applications in particular contexts (e.g. low-density languages, or specialized domains). On the other hand, a large amount of data can be found in news prompts, movie subtitles or scripts, etc. The use of such data as training corpus could provide a low-cost solution to the acoustic model estimation problem. Unfortunately, prior transcripts are seldom exact with respect to the content of the speech signal, and suffer from a lack of temporal information. This paper tackles the issue of prompt-based speech corpora improvement, by addressing the problems mentioned above. We propose a method allowing to locate accurate transcript segments in speech signals and automatically correct errors or lack of transcript surrounding these segments. This method relies on a new decoding strategy where the search algorithm is driven by the imperfect transcription of the input utterances. The experiments are conducted on the French language, by using the ESTER database and a set of records (and associated prompts) from RTBF (Radio Télévision Belge Francophone). The results demonstrate the effectiveness of the proposed approach, in terms of both error correction and text-to-speech alignment.

Introduction

Recent evaluation campaigns have demonstrated the significant advances of speech recognition technologies, especially in handling adverse acoustic conditions and various speaking styles (spontaneous and/or interactive speech, unconstrained dialogues, etc., [START_REF] Fiscus | Multimodal Technologies for Perception of Humans: International Evaluation Workshops CLEAR'07 and RT[END_REF]). These improvements were obtained by the integration of new sophisticated techniques, such as system combination, accurate speaker adaptation, discriminative training, etc. Most state-of-theart recognition systems rely on the Hidden Markov Model-based framework for acoustic and n-gram language modeling: the size and the relevance of training corpora remains a key issue in estimating such models. Moreover, recent techniques tend to increase the sensitivity of the acoustic models to their training corpus, especially discriminative training methods that involve decoding large and accurate training sets [START_REF] Woodland | Large scale discriminative training of HMM for speech recognition[END_REF]. Therefore, a particular attention is paid to the quality and relevance of these resources; the acoustic models are frequently estimated on hundreds of hours of annotated speech. Unfortunately, manual annotation is very expensive; this constitutes a technical limit to the development of speech technology on specialized domains or low-density languages, where the economical interest is low.

On the other hand, some recordings are composed of speech produced following a priori written text. This is the case for movie scripts, political speeches, news prompts, etc. Transcripts may also be produced a posteriori, when speech data is archived, and/or indexed. The use of these data as training corpus could provide a low-cost way of extracting training corpora.

Building prompt-based training corpora faces two main difficulties. First, the speakers do not follow systematically their written source and the resulting utterance may be only an approximate transcription. This limits the interest of such transcripts for acoustic model training. Second, prompts usually suffer from a lack of temporal information, especially on large shows where news, talks and live segments (such as interviews) or reported speech alternate. In this case, prompted segments have to be found in a relatively large amount of speech data, without the time marks that are required by the training process.

This paper addresses these different issues of using imperfect transcripts, composed of prompts or close-captions, to build high-quality corpora. We propose a generic approach where the recognition engine is driven by imperfect transcripts. This method first spots transcript islands in the speech signal and then drives the ASR system to take advantage of these accurate partial transcriptions. We call transcript islands small segments of transcripts that accurately match the pronounced speech, that are available prior to transcription, and for which precise timing is not known. The word island is used in order to emphasize the fact that these transcripts are generally scarce compared to the large quantity of unavailable or inaccurate transcripts.

The paper is structured as follows: Section 2 presents an overview of the techniques which are reported in the scientific literature on this field. In Section 3, we describe our approach for alignment and correction of imperfect transcripts. We propose Driven Decoding Algorithm (DDA) that integrates user-provided close-captions in the recognition engine. In Section 4, we tackle the issue of partially-prompted shows. We propose a transcript island spotting method that detects signal segments that match an a priori available partial transcripts. This method combined with the DDA framework allows us to extract clean speech corpora from partially-prompted speech databases, results are presented in section 5. Finally, Section 6 concludes the paper and proposes some perspectives.

Related work

Close-captions and the fidelity of prompts

The possibility of using prompts as corpus has been considered for a long time in the speech processing community. In [START_REF] Cardinal | Segmentation of recordings based on partial transcriptions[END_REF], the major issues in using news prompts are studied; the authors note that the prompts are not always strictly read, sentences are forgotten or inserted and other sentences are pronounced with some word variations, and close-captions have to respect additional constraints due to the size of the screen where the caption is displayed. [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] evaluate a close-captions Word Error Rate (WER) of 10% to 20%. It is important to note that in the two cases (prompts or closed captions), the grammaticality of the transcripts remains correct, as they are produced beforehand by journalists or by transcriptionists.

The differences between the expected utterance and the actual spoken content cause several problems to acoustic model training; first, the alignment algorithm may be dramatically affected by errors; second, it is crucial to be confident in the transcript for estimating the acoustic models. In the next sections we present an overview of speech-to-text alignment methods on both correct and imperfect transcripts.

Speech-to-text alignment

Speech-to-text alignment is a well-studied issue in the speech processing field. The classical approach consists in modeling the word utterance by a graph of Hidden Markov Models (HMMs) corresponding to the transcription pronunciation graph. The alignment is performed by using the well-known Viterbi algorithm, which is effective on reasonably correct transcripts. Alternative approaches must be used in order to address the issue of the (highly) imperfect transcripts.

In [START_REF] Moreno | A recursive algorithm for the forced alignment of very long audio segments[END_REF], P.J. Moreno et al. evaluate a method aiming to align long audio documents with their exact transcripts in the context of automatic indexing of multimedia documents. This method relies on a search of synchronization points between the exact transcript and an ASR-provided transcription. The first step consists in estimating a language model on the exact transcript. Synchronized areas are isolated from the extracted segments with a match between the a priori and automatic transcripts. Documents are then segmented according to these small confidence islands; on each segment, a specific language model is estimated. The algorithm is run recursively on unaligned segments, until the convergence point is reached. This method is restricted to exact transcripts, but the reported experimental results demonstrated its effectiveness, 99% of the words are correctly aligned on a broadcast news task. Moreover, the effectiveness of this approach depends on the ASR system accuracy -which has to be good enough, and relies on the availability of correct transcripts.

Another strategy consists in generating the alignment that minimizes the edit distance T edit between available transcripts and the outputs of an ASR system [START_REF] Cardinal | Segmentation of recordings based on partial transcriptions[END_REF]. In [START_REF] Mohri | Edit-distance of weighted automata[END_REF], the synchronization of the recognition hypothesis T ASR and the corresponding closed caption T cc is performed by a transducer-based optimization process, where T aligns is the result of the alignment between T ASR and T cc , and ⊕ is the composition operator:

T aligns = T ASR ⊕ T edit ⊕ T cc (1)
The best path is computed by performing a best path search, by using the classical Dijkstra algorithm [START_REF] Cormen | Introduction to algorithms[END_REF]. This operation is performed for each potential closed caption, the alignment hypotheses being validated by a simple decision rule. This method is evaluated for acoustic model training, by collecting the detected segments in a training corpus. Successive decoding passes must be performed and the alignment is computationally expensive.

In [START_REF] Witbrock | Improving acoustic models by watching television[END_REF], Witbrock et al. use temporal information and align the closed captions to audio signals by using a classical dynamic time warping (DTW) algorithm [START_REF] Wagner | The string-to-string correction problem[END_REF][START_REF] Keogh | Derivative dynamic time warping[END_REF]. In [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] Placeway and Lafferty propose an integrated method, based on a translation model that maps the captions to appropriate word sequences. The translation model is a Markov chain where transitions represent deletions, insertions or substitutions. This method was implemented in the Sphinx III decoder [START_REF] Placeway | The 1996 hub-4 sphinx-3 system[END_REF], where caption segments are first timestamped in a first pass.

All these methods are ASR-based and may be dramatically influenced by high word error rates. In [START_REF] Witbrock | Using words and phonetic strings for efficient information retrieval from imperfectly transcribed spoken documents[END_REF], the authors propose an approach based on a dictionary that, instead of words, contains phoneme sequences. This method allows one to deal with out-of-vocabulary words. In [START_REF] Haubold | Alignment of speech to highly imperfect text transcriptions[END_REF] Haubold et al. present a sub-lexical approach for the alignment of speech to highly imperfect transcripts.

Transcript island recognition and synchronization

The task of finding transcript islands is related to, but different from classical speech-to-text alignment. It consists in finding, in a set of transcripts, the part that matches mostly the current speech segment.

For example, in the case of prompt databases, large quantities of texts are available without timestamps. A transcript island recognition process allows one to select matching parts related to the content of speech segments.

In [START_REF] Cardinal | Segmentation of recordings based on partial transcriptions[END_REF], the authors propose a method that selects segments from a large database, competing segments being chosen according to a simple two-string matching score. Some algorithms focus on spotting short transcript islands. In [START_REF] Smith | Identification of common molecular subsequences[END_REF], Smith and Waterman propose to perform local sequence alignments by searching for matching areas, similarly to the techniques used for comparing two nucleotides or protein sequences. This algorithm compares segments of all possible lengths and offsets.

The problem of evaluating the similarity between a partial recognition hypothesis and a prior transcript segment may be viewed as an information retrieval issue. In text retrieval, the problem is to find documents that meet the user information needs, expressed as a query. Most of the approaches to this issue rely on the vector space model, where documents -and queries, are represented in a space D where the dimensions are the words that compose the documents. Words are extracted from the documents after stripping off stop words and stemming them [START_REF] Hull | Stemming algorithms: A case study for detailed evaluation[END_REF]. The TF × IDF (Term Frequency × Inverse Document Frequency) measure is frequently used to get an estimation of the information carried by a word [START_REF] Salton | Term-weighting approaches in automatic text retrieval[END_REF][START_REF] Salton | Automatic Text Processing[END_REF]. The similarities between queries and documents are computed, for example by using the cosine metric, and documents are ranked accordingly. This operation is fast and tractable in large documents. In [START_REF] Ibrahimov | Clustering of imperfect transcripts using a novel similarity measure[END_REF], the authors propose a retrieval method based on clustering. It consists in splitting the text into small parts, with a confidence measure assigned to each part. The intersection between a cluster and query is used as an index of document relevance.

Unsupervised training of ASR systems

In [START_REF] Kemp | Unsupervised training of a speech recognizer: Recent experiments[END_REF][START_REF] Wessel | Unsupervised training of acoustic models for large vocabulary continuous speech recognition[END_REF], the authors propose a study of unsupervised procedures for the training of acoustic models. Confidence measures are used to restrict unsupervised training to the words that are probably correct. Using only a few minutes of manually transcribed acoustic training data and an iterative process, initial models are improved. Moreover, in [START_REF] Kemp | Unsupervised training of a speech recognizer: Recent experiments[END_REF] the procedure is improved by training more than one recognizer and combining the recognition results with a ROVER. However, in [START_REF] Wessel | Unsupervised training of acoustic models for large vocabulary continuous speech recognition[END_REF] the final WER on the testing sets increased by 14% and 18% relative in comparison with a system trained on the full manual transcriptions. Generally, standard unsupervised training is not efficient for discriminative training which is highly sensitive to the transcription quality. However, in [START_REF] Yu | Unsupervised training and directed manual transcription for lvcsr[END_REF], the authors show that using a small amount of directed manual transcriptions (i.e. the proposed strategy is to incorporate some supervised data for the poorly recognized genre) is able to improve unsupervised discriminative training.

Approximated transcripts and ASR systems

As previously discussed, the need for large speech databases for acoustic model training is mainly due to the acoustic model learning paradigm: extensive HMM-based modeling requires the estimation of a large number of free parameters (classically, more than 10 million), requiring a large amount of training data (classically, hundreds of hours of annotated speech). Considering the low quality of closed caption-based corpora, most of the proposed approaches consist in using alternative strategies for model training, where both quality and quantity of training data are considered as macroparameters of the estimation algorithm. In the next subsections, we present related work exploiting approximated transcripts.

Biased language models

The linguistic variability can be reduced, thanks to the contribution of an exact or imperfect transcript. The reduction of the overall linguistic space helps improving the ASR performances. This can be achieved by estimating a language model on the transcript itself. However, such a language model would probably be too specific when the speaker deviates from the original transcript. Therefore, this model is interpolated with a generic language model, following the well-known linear combination of language models (LMs) [START_REF] Jelinek | Self-organized language modeling for speech recognition[END_REF]. The approach is limited by the transcript quality and the interpolation weight should be dynamic. Moreover, experiments using only interpolation carried out in [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] show the limits of this approach: when the interpolation weight is small, the resulting language model does not strongly improve the transcription, and with a large weight, the models match excessively the specialized data.

In [START_REF] Clarkson | Language model adaptation using mixtures and an exponentially decaying cache[END_REF][START_REF] Iyer | Modeling long distance dependence in language: topic mixtures versus dynamic cache models[END_REF][START_REF] Martins | Dynamic language modeling for a daily broadcast news transcription system[END_REF] the authors present two techniques for language model adaptation: the first uses a mixture-based model where mixing weights are computed using a first pass decoding, and the second is based on a cache containing words hypothesized in the past. In the two cases, a specific information can be integrated into the language model. The cache model introduced by [START_REF] Kuhn | A cache-based natural language model for speech recognition[END_REF] proposes to increase the probability of the words that have occurred recently. The assumption behind this model is that if a word is used in a context, then this word is likely to be used again. The trigger model, as explored by [START_REF] Ney | On structuring probabilistic dependencies in stochastic language modeling[END_REF] is a generalization of cache models: the long-distance dependency problem is addressed by interpolating n-grams with trigger pairs selected according to their mutual information.

However, cache models lack robustness, because boosted words are depending on the current hypothesis; an error can easily spread and too many hypotheses can be made with all trigger pairs: they are not easy to tune.

Prompts as training corpus for ASR

Some papers propose to perform ASR on the speech signal and to extract, from the output, the areas that match the prior transcript. In [START_REF] Witbrock | Improving acoustic models by watching television[END_REF], the authors apply this principle to teletext / closed caption materials, for a reestimation of the acoustic models, based on television input.

In [START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF], Lamel et al. propose a slightly supervised method for acoustic model training by using low-quality transcribed databases. This approach consists of three steps:

• automatically transcribe the training database.

• search for ASR output segments that match a part of the prior transcript.

• use the matching segments for acoustic model re-estimation.

The ASR output is used to find matching close captions. These segments are then used to re-estimate the acoustic models.

This idea is extended in [START_REF] Chen | Lightly supervised acoustic model training using consensus networks[END_REF], where closed captions are used in order to refine acoustic models. The authors use a confusion network (CN) to find similar parts between closed captions and audio signal: the ASR system generates a CN for each audio segment. Then, closed captions are aligned with the CN in order to extract matching parts. A threshold is used to cut off false assumptions. This method extracts more aligned data than the slightly supervised method [START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF], by using the entire ASR hypothesis.

In [START_REF] Chan | Improving broadcast news transcription by lightly supervised discriminative training[END_REF], this idea is applied jointly to Maximum Mutual Information Estimation (MMIE) or Minimum Phone Error (MPE) discriminative training.

In [START_REF] Nguyen | Light supervision in acoustic model training[END_REF], the authors propose a light supervision method to acquire acoustic training data from speech associated to corresponding prompts. They estimate an interpolated language model using imperfect transcripts. The ASR output is aligned to the approximate transcripts and only matching words are selected for acoustic training. They yield 13% relative error rate reduction with 702 hours added to the baseline (141 hours of training data).

In [START_REF] Paulik | Document driven machine translation enhanced asr[END_REF][START_REF] Paulik | Lightly supervised acoustic model training on epps recordings[END_REF], the authors deals with multiple audio streams and translation systems in order to improve the ASR transcripts. In the case of the European Parliament, debates are translated into multiple languages. The multiple knowledge sources (Final Text Editions (FTE), audio signal) are used to supervise the primary system:

• FTE into the language model training data.

• Automatic 1000-best translation hypotheses for each language are also used for language model training data.

• Acoustic models are trained according to a general FTE supervision.

In [START_REF] Finke | Flexible transcription alignment[END_REF], the authors propose a transcription alignment method using improved pronunciation models and HMM based transcripts, in order to reduce typical errors in the transcriptions. HMMs introduce more flexibility in the transcripts: instead of aligning the plain transcription, a HMM is generated for each utterance allowing multiple pronunciations, multi-words and optional silences or noises. The Flexible Training Alignment applied to the Switchboard and CallHome corpora reduces significantly the WER. These results demonstrate how important the transcript quality is.

Another approach presented in [START_REF] Petrik | Reconstructing medical dictations from automatically recognized and non-literal transcripts with phonetic similarity matching[END_REF][START_REF] Petrik | Automatic phonetics-driven reconstruction of medical dictations on multiple levels of segmentation[END_REF] corrects a posteriori ASR output using words in associated medical reports. Authors introduce a flexible automatic phonetic transcription to solve the issues of formatted entity prompts and alternative pronunciations. The methods based on phonetic similarity matching and text alignment on multiple levels of segmentation allow one to improve the accuracy of the transcriptions.

Another approach proposed by Placeway et al. [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] is to estimate a language model with prompts or closed captions. The estimated model is interpolated with a generic language model in the ASR system. This technique improves the results, but the information brought by subtitles is mostly drowned in the data quantity. Placeway et al. include a closed caption model into a beam search: the words that match the imperfect transcript are favored.

All related work allows exploiting imperfect transcripts in order to improve an ASR system. [START_REF] Petrik | Reconstructing medical dictations from automatically recognized and non-literal transcripts with phonetic similarity matching[END_REF][START_REF] Petrik | Automatic phonetics-driven reconstruction of medical dictations on multiple levels of segmentation[END_REF][START_REF] Finke | Flexible transcription alignment[END_REF] are dedicated to good quality transcripts and do not influence the ASR system itself. The methods presented in [START_REF] Witbrock | Improving acoustic models by watching television[END_REF][START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF][START_REF] Chen | Lightly supervised acoustic model training using consensus networks[END_REF][START_REF] Chen | Dynamic language modeling for broadcast news[END_REF][START_REF] Paulik | Document driven machine translation enhanced asr[END_REF][START_REF] Paulik | Lightly supervised acoustic model training on epps recordings[END_REF] are more classical approaches where additional data are used in order to improve/bias language models and acoustic models. Finally, the framework proposed by [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] directly introduces approximated transcripts into the ASR engine.

In our experiments, we mainly use two methods as baseline: the first one is the method proposed by [START_REF] Placeway | Cheating with imperfect transcripts[END_REF], allowing to include prompts into a synchronous Viterbi decoding framework. The second baseline method will be the slightly supervised approach [START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF] which is widely used in the speech recognition community.

Error correction by Driven Decoding Algorithm

Background

Our goal is to exploit imperfect transcripts within an asynchronous decoder based on the A * algorithm, in the framework of a broadcast news system. We expect an integrated approach that does not require multiple passes.

We first present the characteristics of the Laboratoire Informatique d'Avignon (LIA) decoder. We also show how the information from imperfect transcripts can be integrated in the decoding process.

Then we evaluate a naive method that consists in combining a generic language model and a language model estimated on the imperfect transcripts.

Finally, we propose a new approach that relies on driving the ASR system search algorithm with the imperfect transcripts.

The LIA French broadcast news system

The LIA Broadcast News system relies on the Speeral decoder [START_REF] Linarès | The lia speech recognition system: from 10xrt to 1xrt[END_REF] and the Alize-based segmenter [START_REF] Bonastre | Alize, a free toolkit for speaker recognition[END_REF]: segments are automatically segmented by speaker (in order to adapt the acoustic models by using Maximum Likelihood Linear Regression method) and their size is limited to 30 seconds.

Here, we use the system involved in the ESTER evaluation campaign [START_REF] Galliano | The ester phase 2 based evaluation campaign for the rich transcription of french broadcast news[END_REF]. Cross-word context-dependent acoustic models with 230k Gaussians are used. Tying is achieved by decision trees, by using acoustic context related questions. We train the acoustic models on ESTER materials (about 80 hours of annotated speech). We use a classical PLP parameterization; feature vectors are composed of 12 coefficients and energy with the first and second derivative of these 13 coefficients. A cepstral normalization is performed in a 500ms sliding window. Two sets of speaker-independent acoustic models are used: a wide band model and a narrow band model, both gender-dependent.

The language models are classical trigrams estimated on about 330M of words from the French newspaper Le Monde 1987-2003 and the broadcast news training data (manual transcripts) provided during the ESTER campaign (960K words) with a vocabulary of 65K words. Language model includes 16.7M of bigrams and about 20M of trigrams. The system runs two passes. The first one provides intermediate transcripts which are used for Maximum Likelihood Linear Regression (MLLR) adaptation. The second transcription pass uses these MLLR models. The first pass takes about 3xRT and the second one about 5xRT on a standard desktop computer.

The LIA English broadcast news system -baseline

The English system is an adaptation of the previously presented French system.

The phone set used for the acoustic model is the same as the one used in the Carnegie Mellon University Pronouncing Dictionary1 . The acoustic model has the same structure as the French model: cross-word contextdependent acoustic models with 230k Gaussians and tying by decision trees using acoustic context related questions. The acoustic models are trained on the HUB4 English corpus [START_REF] Stern | Specifications of the 1996 hub-4 broadcast news evaluation[END_REF] (about 70 hours of annotated speech). We use the same PLP parametrization as used for the French system. The baseline LM for transcribing broadcast news is a 65k word classical 3-gram, estimated on 2.7G words from the Gigaword, North American News and HUB4 corpora, by using the modified Kneser-Ney smoothing technique. The weights of the corpora in the language model are chosen such as the perplexity is minimized on a small part of the development corpus of HUB4.

For transcribing surgery-related data from the AVISON corpus we used a combined 65k word LM, by interpolating general 3-grams learned on the HUB4 English corpus, with 3-grams estimated on all the reference transcriptions available in the AVISON training corpus. We rely, here as well, on the modified Kneser-Ney smoothing technique. The LM mix factors are chosen so that the perplexity of the resulting LM is minimized on a development corpus, which contains 5 hours of imperfect transcripts. The development set has been excluded from the training corpus. The acoustic models are the same as the HUB4 system.

We used the Carnegie Mellon University Pronouncing Dictionary for obtaining the phone representation of the lexicon words. The words that were not in this dictionary was automatically phonetized with Festival [START_REF] Taylor | The architecture of the festival speech synthesis system[END_REF].

Confidence test

We use the following confidence test in our experiments [START_REF] Chollet | Evaluation of asr systems, algorithms and databases[END_REF]:

wer f -u α 2 wer f (1 -wer f ) k < wer p < wer f + u α 2 wer f (1 -wer f ) k ( 2
)
Where k is the number of words in test, wer f the WER on test. α defines a confidence interval: if α = 95%, then wer p is defined with a confidence of ±0.05%. u α 2 is defined by the Student value: u 0.425 = 1.96. We also propose the Matched Pairs Sentence Segment Word Error Test (mapsswe) score provided by the sc stats NIST tool for WER results.

Anatomy of the Speeral decoder

The Speeral decoder [START_REF] Linarès | The lia speech recognition system: from 10xrt to 1xrt[END_REF] is derived from the A * search algorithm. A * is an algorithm dedicated to the search of the best path in a graph. It has been used in several speech recognition engines, generally for word-graph decoding. In Speeral, the search algorithm operates on phoneme lattice, which is estimated by using cross-word and context-dependent HMM.

The exploration of the lattice is supervised by an estimation function F (h n ), which evaluates the probability of the hypothesis h n crossing the node n:

F (h n ) = g(h n ) + p(h n ) (3)
Where p(h n ) is the probe that estimates the probability of the best hypothesis from the current node n to the ending node. In Speeral, the probe p combines the acoustic probability and a linguistic look-ahead score (LMLA -Language Model Look-Ahead) [START_REF] Massonié | Scalable language model look-ahead for lvcsr[END_REF]. The acoustic term is determined via an acoustic decoding process, carried out as a Viterbi algorithm operating backwards (i.e., from the end of the signal, to its beginning), on the phone lattice. The LMLA used in Speeral enables the comparison of competing hypotheses before reaching a word boundary. The probability of a partial word corresponds to the best probability in the list of words sharing the same prefix:

P (W * |h) = max i P (W i |h) (4)
where W * is the best possible continuation word and h the word history (partially present in g(h n )). The LMLA approximation does not affect the results and speed-up the search process. This technique will be useful for Driven Decoding Algorithm to anticipate explored words. g(h n ) is the probability of the current hypothesis that results from the partial exploration of the search graph (from the starting point to the current node n):

g(h n ) = max P (W ) β δ |W | P (X|W ) (5)
Where P (W ) is the linguistic probability of the current word sequence, P (X|W ) is the acoustic probability according the word sequence W , β is the language model fudge factor (14 for French and 10 for English), δ is the linguistic penalty (-12 for French and -14 for English) and |W | the number of words in the W sequence.

The best paths are then explored in a depth-first manner. This depthfirst search refines the evaluation of the current hypothesis. Low-probability paths are stacked, thus allowing for a backtracking on these paths. In such situations, the search is desynchronized from the audio stream.

The Driven Decoding Algorithm

The Driven Decoding Algorithm aims to align and correct imperfect transcripts by using a speech recognition engine [START_REF] Lecouteux | Imperfect transcript driven speech recognition[END_REF][START_REF] Lecouteux | Text island spotting in large speech databases[END_REF][START_REF] Lecouteux | Using prompts to produce quality corpus for training automatic speech recognition systems[END_REF]. This algorithm improves the system performance by taking advantage of the availability of the approximate transcripts. This DDA integrates a DTW-based alignment within the A * algorithm. A first on-demand synchronization of the current explored hypothesis is performed with the prior transcript. Then, a transcript-to-hypothesis matching score is evaluated and used for fudging the n-gram probabilities.

ASR synchronization to the imperfect transcript

We assume that the presented synchronization is performed after a presegmentation stage. In the next experiments, we propose to use manual and automatic pre-segmentation stages.

The Speeral speech recognition system generates hypotheses as the phonelattice is being explored. The best hypotheses at time t are extended according to the current hypothesis probability and the probe results.

In order to locate an anchoring point in the auxiliary transcript T of size m, each evaluated word from the current hypothesis H is aligned to T by using a Dynamic Time Warping (DTW [START_REF] Berndt | Using dynamic time warping to find patterns in time series[END_REF]) algorithm.

In practice, a partial hypothesis H of size n is built by collecting the current word (i.e. the currently explored node, using LMLA) and its history from the path found during the search process. The sequence alignment is achieved by constructing a n-by-m matrix where the (i th , j th ) element of the matrix contains the distance between the two words T i and H j . We use a basic distance function:

Considering a pre-segmentation stage, the auxiliary transcript size is close from the maximal hypothesis size.

d(T i , H j ) = 0 if T i = H j d(T i , H j ) = 8 in the insertion cases d(T i , H j ) = 6 in the deletion cases d(T i , H j ) = 12 in the substitution cases (6)
The deletion, insertion and substitution costs are computed via the estimated probability of each event in the approximated transcripts. The cumulative distance γ(i, j) between H j and T i is computed as:

γ(i, j) = d(T i , H j ) + min{γ(i -1, j -1), γ(i -1, j), γ(i, j -1)} ( 7
)
The alignment is performed at each newly encountered word in the phone lattice, during the search: a cache of the previous alignment is used in order to quickly increment or decrement the cumulative distance γ(i, j).

Considering the low complexity of the word alignment with the cache, this on-demand synchronization process requires low computational resources. Moreover, the additional alignment cost may be balanced by the speed-up provided by well-transcribed sections (about 20% faster).

In Figure 1 we illustrate the dynamic synchronization of the search algorithm driven by the alignment on an imperfect transcript: for each explored node, the DTW algorithm allows one to align the imperfect transcript with the current generated hypothesis.

The synchronization robustness depends on the DTW alignment. If the partial hypothesis matches several repeated parts (this behavior is frequently observed at the segment beginning), the part with the best score is selected. However, after five words, the DTW path usually becomes stable.

The best hypothesis-to-reference matching provides a synchronization point that will be used to compute a matching score that is presented in the next subsection.

Weighting of the current hypothesis according to alignment

In order to use imperfect transcript information, the linguistic part of the F () function is rescored according to a transcript-to-hypothesis matching score α(w). This mechanism drives the search by dynamically fudging g(h n ), according to the alignment scores.

The matching score denoted α is based on the number of words in the short-term history that are correctly aligned with the transcript. α is greater when the trigram is aligned, and decreases with the misalignments of the history. The values of α were determined empirically by using a development corpus composed of a 15-minute show with a grid-search method:

α(W ) =            0.01 if w 1 ,
Where (w n , ..., w 1 ) are the words (left to right) of the currently explored hypothesis of the ASR system (w 1 is the word corresponding to the current node). (t n , ..., t 1 ) are the transcript words aligned to the current hypothesis. W is the whole word hypothesis. These α values minimize the WER by using a simulated imperfect transcript (20% WER). Then, linguistic probabilities are modified by using the following fudging rule:

P (W k ) = P (W k ) α(W k ) ( 9
)
where k is the size of the currently explored word hypothesis W , P (W k ) is the updated n-gram probability and P (W k ) is the initial probability of the n-gram. However, in practice, the α values have a limited impact (in a ±0.5 range), while they decrease with the misalignments of the history. The α values presented in equation 8 are used in all our experiments.

With the DDA, equation 5 becomes:

g(h n ) = max k=|W | k=1 P (W k ) (β+α(W k )) δ |W | P (X|W ) (10)
where α(W k ) is dynamically evaluated according to the imperfect transcript alignment with the currently explored hypothesis.

The set of experiments based on DDA in section 3.8 uses the mechanisms described above.

DDA with a synchronous Viterbi decoding framework

The proposed DDA can be easily transposed into a Viterbi decoding framework. The only requirement is to know the best history alignment of the currently explored word in the beam search, in order to fudge the linguistic score. This implementation requires another cost function inside the DTW algorithm and to compare multiple hypotheses at any time. The method proposed by [START_REF] Placeway | Cheating with imperfect transcripts[END_REF] aligns closed captions in a similar way. The main difference in the Placeway approach is the rescoring trigram rule based on a conditional distribution P (w i |w 1 ...w i-1 ; c 1 ...c m ) where w i is the next word in a transcript whose closed-caption is c 1 ...c m . This conditional distribution is inspired from [START_REF] Brown | Automatic speech recognition in machine aided translation[END_REF], where the authors combine speech recognition and machine translation models. In our approach, the rescoring (or fudging) of linguistic rule is based on the history alignment, strongly biasing the ASR search. Moreover, in our framework, the best path is evaluated for each newly explored node: the ASR system is more robust to find matching segments.

Experiments 3.8.1. Experimental context

The experiments are carried out in the framework of the French ES-TER evaluation campaign [START_REF] Galliano | The ester phase 2 based evaluation campaign for the rich transcription of french broadcast news[END_REF]. The ESTER corpus contains French radio broadcast news, including ad-hoc interviews, non-native speakers, on-the-fly translations, ... The results are reported on a test set of 4 hours from three broadcasters (France Inter 1 & 2, France Info and RFI), extracted from the official ESTER development set. The 3 : 1 balance between deletions and insertions was a choice during the ESTER evaluation campaign allowing the best WER. The language model perplexity is 132 for the whole test set. Only the first pass is assessed.

Baseline results are presented in

Artificial transcripts

Here, the imperfect transcripts are generated by manually adding errors to the initial transcripts, while ensuring a correct journalistic form in order to respect the traditional style of a radio broadcast. In all cases, the meaning of sentences is preserved:

• We change some wording Finally, 10% WER are introduced in three show transcripts, and 20% WER are introduced in the last show transcript. Table 2 presents the deletion, insertion and substitute rates in our artificial transcripts.

Here are some examples of imperfect transcripts (ref erence and modified transcripts): • mod: donc on respectera ce qu'on a dit ... il n'y aura However, we do not take normalization issues into account: some dates, numbers or named entities can have wrong forms.

Segmentation

In the experiments with artificial transcripts, we first run the decoder without the DDA in order to align the transcripts and the decoder output: alignment to the reference is performed by using the sclite NIST tool. Then, reference transcripts are segmented in advance with large margins (about 10 words before and after): for each audio segment, the corresponding imperfect transcript is known. These marginal words are added in order to increase robustness and to involve slight overlaps between reference segments. Moreover, timestamps are removed from the transcripts.

Segment duration is based on automatic speaker segmentation with a 30 seconds threshold (generally, segment length is from 5 to 30 seconds). The number of segments per hour is about 200. Table 3

Language model interpolation

The linguistic variability can be reduced thanks to the contribution of an exact or imperfect transcript. Reducing the overall linguistic space helps improving the ASR performances. This can be achieved by estimating a language model on the transcript itself. However, such a language model would probably be too specific when the speaker deviates from the original transcript. Therefore, this model is interpolated with a generic language model, following the well-known linear combination scheme of language models (LMs).

Preliminary experiments are carried out in order to identify the potential benefits of the proposed methods. First, a language model is estimated with the accurate transcript. This model is combined with the generic 65K language model. These experiments measure the real effect of the proposed techniques on the decoder recognition performance: Out-Of-Vocabulary (OOV) words are removed. In Table 4 we show the results of the interpolation of a language model trained on the exact transcript with the generic language model. For comparison, baseline decoding is performed by using the generic broadcast news language model. We obtain a WER of 22.7%. Then, a language model is generated by using the imperfect transcript (10% WER) as well. The experiments using this language model combined with the generic model are also presented in Table 4.

Outline

These experiments show that a decoding process based on a language model estimated on imperfect transcripts significantly improves WER. However, the WER remains higher than 10%, that corresponds to the initital WER of the imperfect transcript. The best results are obtained by combining the generic language model and the model estimated on the imperfect transcript.

Moreover, in the case of models trained on the exact transcript, the WER remains higher than 5%. Combined with a generic language model, the WER increases: transcript information is lost. Hence, we need to use transcript information more efficiently.

Experiments with model interpolation and Driven Decoding Algorithm

In this section we present experiments using the decoding strategy based on DDA. The experiments combining the interpolation of the language models with an alignment on the exact transcript are presented in Table 5. We obtain a minimum WER of 3.5%. This error rate can be considered as minimal for a concurrent hypothesis re-estimation method, without modifying the content of the hypothesis stack.

In Table 6 we show the experiments where the exact transcript is replaced with the imperfect transcript (10% WER).

Outline

WER FrInter 1: LM-TrErr + alTrErr 9.9% FrInter 1: LM-G + alTrErr 7.7% FrInter 1: LM-G 70% + LM-TrEr 30% + alTrErr 7.2% FrInter 1: LM-G 50% + LM-TrEr 50% + alTrErr 7.4% FrInter 1: LM-G 30% + LM-TrEr 70% + alTrErr 8.6% Although this approach removes some of the limitations observed in the model combination, potential sources of error remain. In particular, heuristics are used in the decoder to reduce the search space and to speed up the decoding process. In ordinary conditions, the pruning introduces only a few errors; however when the acoustic context is of low quality, the best hypothesis can be excluded from the stack of available hypotheses. This occurs more frequently in real-time configurations of the system, when the pruning is stricter: a strategy highlighting the synchronized hypothesis rather than the others is not able to find the good hypothesis in the search space.

The best result is obtained by combining the generic language model (with a 70% weight) with the model estimated on the imperfect transcript (with a 30% weight) and by carrying out a Driven Decoding with the latter.

The DDA reduces the WER to 7.2%. This makes possible to add temporal information that is poorly taken into account by the language model. The use of DDA associated to the interpolation of the models results in a new gain.

In order to validate these results, we tested the best configuration on the four hours of test data. The results are reported in Table 7.

Shows

Baseline In order to compare the results on the same show, we introduced a WER from 0% to 55% (0, 10, 20, 30, 41 and 54) into the RFI transcript. The results are plotted in Figure 2. We observe that the performance gain seems to be relatively independent of the quality of the initial transcript: the method is quite robust. These experiments show that imperfect information can be favorably used during the decoding process.

Discussion on the Driven Decoding Algorithm

A well-known advantage of the A* algorithm lies in the possibility to incorporate various information sources into the recognition process. However, it is an asynchronous algorithm and its application to alignment tasks can be difficult. We proposed an on-demand synchronization that allows the combination of asynchronous recognition and transcript-to-signal alignment. The system takes advantage of the approximate transcript as long as it allows for a gain and switches to free-recognition mode when the acoustic observations do not match the suggested transcript.

This method provides a significant gain in terms of WER even if the quality of the provided transcripts is low. The relative WER improvement obtained ranges between 28% and 40%. Moreover, we observe that the modified algorithm improves decoding speed slightly (by about 20%), in spite of the additional computational cost due to search synchronization. This gain in terms of execution time is due to the earlier exploration of the best paths on well-transcribed sections.

The work presented in this section relies on pre-segmented transcripts and the knowledge which imperfect transcript belongs to the segment. In real conditions, prompts or approximated transcripts do not contain accurate timestamps. Moreover, the data available is often incomplete and large. The lack of significant parts of transcript causes failures in the search for anchoring points. Therefore, the algorithm is not really relevant for the spotting of segments in large text corpora, without pre-segmentation. Sections 4 and 5 explore a dynamic extraction of segments in order to find parts matching the decoded speech. These matching parts will be called transcript islands. This dynamic extraction will eliminate the necessary first pass for pre-segmentation.

Fast hypothesis-to-transcript island matching: an integrated spotting task

The goal of our method is to take advantage of imperfect transcripts when they are available, while no timing information is available for the localiza-tion of transcript islands. Moreover this method allows one to use the full prompts without pre-segmentation. Therefore, the main issue is to integrate, in the ASR system, an identification module that would be able to decide, at each node of the search graph, when the recognizer is crossing one of the available transcript islands. As the search graph is developed dynamically, this integration can be achieved by an on-the-fly spotting process.

The principle of the proposed method is close to approaches used in the field of information retrieval. Typically, search engines try to find the most relevant documents by comparing the query (i.e. the current explored hypothesis) to the indexed collection of stored documents. Most of the algorithms consist in building a set of ranked document lists. Here, we follow a similar scheme, while focusing on the efficiency of the algorithm.

The lexicon is represented by a lexical space where each dimension is associated to a word. All documents, including the hypothesis itself and the transcript, are represented in this lexical space by word-frequency vectors. The coefficients of these vectors represent the frequencies of the words in the document.

As the current hypothesis h is developed at a time t, a set of word clusters C i is built and updated. These clusters result from the intersection of h and the transcript I. Start and end words of a cluster C i delimit a transcript island I i in the prompt. For each new word added to the hypothesis h, transcript islands are considered as candidates for guiding the search. This competition is arbitrated by a matching score S i , which is computed as follows:

S i (I i ) = |I i | |h| k w k ∈C i itf (w k ) ( 11
)
where | I i | and | h | are the cardinalities of the island I i and the current hypothesis h, respectively. C i is a cluster resulting from the intersection between h and the approximated transcript as: C i = h ∩ I i . w k are the words of the current hypothesis and itf (w k ) represents the inverse measure of the word frequency in the whole transcript:

itf (w) = 1 tf (w) ( 12
)
where tf (w) is the frequency of w in the document. This matching score represents a level of similarity of the hypothesis to the considered transcript island. This measure takes into account a semantic weight of the word, which depends on its relative frequency in the whole document.

If any matching score is higher than an a priori fixed threshold, the algorithm considers that it is on a transcript island and the search algorithm is driven by the corresponding word utterance: the transcript island becomes the imperfect transcript T as in section 3.6.1, then the search algorithm is driven by equation 9.

Algorithm development

This algorithm was developed for the Speeral ASR system. Despite of the deep-first search algorithm, in most case, only the last words of the current explored hypothesis are varying. This aspect allows us to control the computational cost of the proposed algorithm via the T f req value. The threshold T f req is based on word frequency and allows one to use only words appearing more than T f req . In the proposed algorithm, N max is the maximum number of desired clusters (100 is a good value). δ is the tolerance (±10%) of the maximum number of clusters. tf (w) is the term frequency of the word w into the transcription. C i is a cluster composed of the word positions {p 0 , ..., p m }. {C} is the set of clusters. I i is a transcript island delimited by the start (i.e. min position) and the end (i.e. max position) of a cluster C i .

The algorithm is developed and detailed in the next page (Algorithm 1).

In practice, the set of cluster is not cleared for each new hypothesis. The process is incremental: a new hypothesis h differs often in the last words compared to the previous hypothesis h . During the algorithm execution, all operations (creating a cluster, merging a cluster, adding a position in a cluster) are stacked. When a new hypothesis is explored, operations are unstacked to obtain the hypothesis h = h ∩ h . Then, algorithm is applied only on changing words. However, if T f req is varying, the set of cluster is then cleared and operations destacked.

Each word w is associated with its positions p in prompt; These experiments aim to evaluate the performance of the proposed method for building a high-quality speech corpus by using the closed-captions associated to a speech signal. On the spotting task, the system performance is evaluated in terms of precision/recall rates, on ESTER and RTBF ("Radio-Télévision Belge Francophone") corpora. F-measures are also reported. We first test the spotting performed on the ESTER corpus using gold and the degraded transcripts. Finally, we evaluate our method on the RTBF corpus based on real closed-captioning. Precision, recall and F-measure are defined as:

1 Initialize N max ; 2 T f req ← 0; 3 δ ← 10 100 N max ; 4 tf (
precision = {relevant documents} ∩ {retrieved documents} {retrieved documents} (13) recall
= {relevant documents} ∩ {retrieved documents} {relevant documents} (14)
F-measure = 2 • precision • recall precision + recall (15)
In the next ESTER experiments, the reference segments are viewed as documents. If the spotting process associates the correct reference segment (i.e. the relevant document) to the current decoded speech, the document is marked as well-retrieved. In the next experiments no DDA is used in order to focus on the spotting task We simulate scattered data by randomly discarding 50% of the transcript segments. The removed transcripts have been chosen according to the reference speech segmentation. Finally, we obtain a reference with 500 to 800 annotated segments per hour (Table 8). The average duration of the remaining segments is about 6 seconds.

Spotting exact transcripts on the ESTER database

In the following experiments, DDA is not used. We evaluate the performance of our spotter. The experiments are performed on the four hours from the development set (i.e. France Inter 1 & 2, France Info and RFI). The spotting targets are exact transcripts. The results are reported in Table 8. We can see that, in these simulated conditions, spotting performance is good: more than 96.9% of the documents have been found, with a precision of about 94.4%. The results seem relatively independent of the performance of the ASR systems, which are varying, in this test, from 27.2% (the RFI show) to 22.7% (the France Inter show).

Radio station Precision

Spotting imperfect transcripts on ESTER database

As previously, 50% of transcript segments have been removed for spotting evaluation. The experiments are conducted on the four hours of the development set by using the imperfect transcripts (the same as those used in section 3.8.5).

These experiment aims at evaluating how errors in transcripts impact the spotting task performance. The results are reported in Table 9. Comparing Table 8 and Table 9 we observe that the obtained precision and recall rates are very close to the ones obtained on perfect transcripts: the spotting task is robust to the increase of the WER.

Radio

Spotting real prompts on the RTBF corpus

The RTBF corpus has been collected in the framework of the AIDAR project [START_REF] Tshibasu-Kabeya | Aidar : Une architecture pour l'indexation de documents audio numériques[END_REF]. It contains about 1000 hours of radio programs from the Radio Télévision Belge, in French and mostly recorded under clean conditions. Those programs mainly consist of news where topics and linguistic styles are rather close to those of the ESTER corpus. Among these 1,000 hours, the proportion of speech exceeds 300 hours (the remaining 700 hours are music, advertising or jingles) and prompts (provided in XML files) are available for about 60 hours of news. These prompts were used by the journalists. No further refinement on transcripts was done after recording, and no timestamps are available for precisely locating the speech segments corresponding to the provided transcripts. In order to evaluate our algorithm on this database, we have manually annotated the timestamps related to prompted paragraphs for 11 hours of the corpus, whereas all the 60 hours prompts are used. Audio without prompts is not used.

Precision Recall F-measure Par. number Conf. Int. RTBF shows 99.28 % 97.13 % 98.41 % 501 1.52

Table 10: Precision/recall, F-measure and confidence interval (for F-measure) of transcript island spotter. The experiment is performed on broadcast news shows from RTBF, by using the real journalist prompts. Par. number is the total number of paragraphs based on manual annotation.

In Table 10 we present experiments with our spotting technique on the 11 hours of annotated data. The results are better than those observed on the ESTER corpus: the average F-Measure is around 98%. 10 out of the 22 shows (of 30 minutes each) are fully timestamped at the paragraph level. These results are probably due to the fact that the prompt mapping is usually related to the global structure of the document, including speaker information, speech turns, non-speech segments, etc. Then, transcript islands match the natural segmentation of the document. Well-segmented transcripts are easier to spot. Moreover, paragraphs are significantly longer in the RTBF corpus: we have about 22 annotated paragraphs per show (30 minutes), for a total of 501; this limits the risk of missing an island that should be spotted.

The next section presents the combination between the transcript island algorithm and the Driven Decoding Algorithm.

Combining DDA and transcript island spotting

In previous sections, we presented two algorithms:

• The DDA method allows one to improve recognition rates by taking benefit of the available transcripts, even if they are not perfect.

• The transcript island spotting algorithm aims to find on-the-fly matching parts of large transcripts with the current hypothesis of the ASR system.

In this section, we evaluate the quality of the transcripts provided by the Speeral decoder guided by the prompts spotted on-the-fly into a large corpus: in the next experiments, prompt data are not pre-segmented.

The Figure 3 presents the spotter integration with the Driven Decoding Algorithm. The spotting system transcript islands extracts, from the large quantity of transcriptions and feeds them to the DDA. 5.1. Impact of DDA combined to transcript island spotting on the ESTER corpora The ASR system is using together the spotting algorithm and the DDA. Two tests on four hours have been performed and compared to the baseline system (consisting of a classical Speeral run, without any helpful transcript): we first evaluate the WER by using segments of exact transcripts; then, the same experiment is performed on imperfect transcripts. We used previous imperfect transcripts of 10% to 20% WER, and 50% of the text segments have been removed for evaluating the spotting performance.

Table 11 reports the results obtained by the DDA search algorithm driven by the perfect transcript, and the DDA driven by imperfect transcripts: the DDA performance variance between Table 11 and Table 4 is due to the 50% of removed segments.

The results show that the driven recognizer takes advantage of the spotted segments: the ASR system is able to extract on-the-fly transcript islands and to use them via the DDA. As expected, the correct prompts remain more effective than imperfect ones; nevertheless, approximate transcripts yield a WER gain of about 14% relative, while exact transcripts yield a WER gain close to 24% relative. This set of experiments shows that the spotting algorithm combined with DDA is able to produce a better transcript. This technique takes advantage of all available information (i.e. audio stream and transcript islands): we obtain a better automatic transcription and a better alignment, thanks to the decoder quality improvement. These two criteria allow one to increase the size of the correct ASR system transcripts in just one pass (without acoustic adaptation for a second pass). The search of text-segments is fast, on-thefly and synchronized to the decoding process. In addition, experiments with more than one pass show that the system converges towards the best potential solution during the first pass.

Impact of DDA combined to transcript island spotting on the RTBF corpora, in real condition

We used 200 hours of speech signal with associated prompts. Prompts are grouped by month and are imperfect transcription of the spoken contents. This data allows us to measure the efficiency of our approach. A language model is estimated on all prompts (about 2.4 million words) and interpolated with a generic language model proportionately to the amount of data. This language model is used both for the baseline and DDA. Acoustic models are those used in previous experiments. The baseline is based on a slightly supervised decoding approach. Baseline results are presented in Table 12 for one pass decoding associated with an a posteriori alignment: we obtain about 30 hours of exact annotated speech. Then, we use the transcript island algorithm combined with the Driven Decoding algorithm, which allows us to assess the amount of the aligned data.

The transcript island algorithm allows one to use only one pass to align on-the-fly approximated transcripts with the ASR system. The results show that with the transcript island Driven Decoding, 38% additional words are aligned to the prompts: we have 50 hours of exact annotated speech. We observe 25% of aligned prompts. The data loss can be explained by several aspects:

System

• An initial decoder with about 25% WER • Parts of prompts are not pronounced

• Titles and abstracts are inside the prompt data

• The initial WER of prompts

• The prompt normalization is imperfect: date formats, OOV word pronunciation, named entities, ... These results show a significant increase of the quantity of usable data. The DDA allows us to correct the ASR system on the fly: this increase of data quantity should result in a WER improvement. This experiment yields the expected results: a larger corpus with a potentially improved quality (imperfect transcripts are rarely in the ASR output).

The combination of the two proposed algorithms allows us to extract only aligned paragraphs and to improve the ASR output via the DDA (in Section 3.8, we show that the DDA always improves the baseline system and the imperfect transcripts). Consecutively, experiments on the broadcast news RTBF corpus show that the method produces a larger corpus than the baseline approach.

In order to evaluate the potential of the method for acoustic adaptation, we also performed experiments on a specialized-domain corpus, in the medical field (the AVISON corpus, described in the next section).

Experiments on the AVISON corpus: acoustic adaptation

The AVISON corpus contains around 48 hours of commented English surgical intervention films. The spoken material in this corpus contains speech in several registers: read speech documenting surgical issues, spontaneous descriptions of surgical interventions, or spoken dialogues between surgeons and students. We have only imperfect transcripts for these audio documents. The AVISON corpus also contains a collection of textual documents related to surgery (scientific articles, surgery proceedings, protocol descriptions, etc.), which can be used for language model training purposes. We build a test corpus by manually transcribing four hours of this corpus.

On these four hours, comparing the initial imperfect transcript WER is 11.7% (sub = 1.56 % / del = 3.14 % / ins = 6.98 % ). The language model perplexity is 127 on the test corpus. We present some examples of errors (transcription and ref erence):

• tra: the thyroid space here the carotid artery is here *********** it is For building a transcription system for this kind of data, we used the transcription system, Speeral, that was described in the previous Section 3.2.

The system used for transcribing this domain-specific corpus is described in Section 3.3. A baseline WER of 41.6%, with unsupervised speaker adaptation, was obtained with this system on the AVISON test corpus: a first pass is used to adapt acoustic models with the MLLR technique (no confidence threshold is used for the Baseline adaptation). Moreover, 5% of the test words are OOV.

In order to improve the baseline acoustic model, we used techniques for automatically obtaining aligned speech and references from the spoken documents, where imperfect transcripts are available. These aligned data are then used for adapting the acoustic models to the speech conditions of the database. We tried two approaches, the first one consists in decoding the speech documents, and then aligning the result of the decoding with the imperfect transcripts. The alignment algorithm used is DTW. The second one consists in driving the decoding of the speech documents with the imperfect transcripts and thus producing aligned data. In the two cases, the LM used for decoding (first and second pass) is interpolated with the imperfect transcripts themselves. The lexicon used contains all the words that are present in the imperfect transcripts.

The amount of aligned data obtained with the two approaches is reported in Table 13. The Table also contains the ASR system accuracy, in terms of WER, on the AVISON test corpus. These performance figures have been obtained by using the acoustic models adapted with the aligned data.

System

Aligned The first point to note is that the DDA increases the quantity of aligned data by 18%, compared to a classic a posteriori alignment: from the around 10 hours obtained without the DDA, we obtain around 12 hours by using the DDA. This increase of the amount of training data yields a gain of 0.8% WER absolute, which represents a significant relative gain of 2.5%. The mapsswe test finds a significant difference at the level of p = 0.01. Table 14 presents more details about encountered errors in the baseline, DDA-based adaptation and slightly supervised adaptation.

Outline

Corr Sub Del Ins WER Baseline P1 65. [START_REF] Cardinal | Segmentation of recordings based on partial transcriptions[END_REF]

Acoustics models trained from scratch on the AVISON corpus

In this section, we present experiments where the ASR system is trained from scratch by using the generated corpora. As previous, our baseline is based on the lightly supervised approach. We have decoded the 48 hours of AVISON corpus using lightly supervised training approach (Baseline) and the DDA approach (DDA-full). Then, we have trained acoustic models on these two sets. The results obtained on the test corpus with these adapted models are reported in Table 15 The results show a 4% relative WER improvement compared to the baseline. This aspect highlights that the DDA training database is significantly better (the mapsswe test finds a significant difference at the level of p = 0.001.) for training compared to the slightly supervised training. However, these experiments do not take information about the improvement causes: the larger amount of data available and/or the better quality of the transcripts.

In a second way, we randomly remove segments in the DDA generated data in order to obtain the same amount of training data that the slightly supervised approach. Then, we have trained the acoustic models. The results are presented in the last line of Table 15. The results are similar to the Baseline: lightly supervised training approach and DDA approach generates data of equivalent quality.

Conclusion

We have presented a method that aims to improve imperfect transcripts by using ASR technology. The proposal consists mainly in a Driven Decoding Algorithm that combines asynchronous recognition and transcript-to-signal alignment.

The system takes advantage of the approximate transcripts as long as they match the speech contents, and switches to free-recognition mode when the acoustic observations do not match the suggested transcript. Performance is evaluated step-by-step, by comparing systematically the results obtained to the potential gains estimated by oracle measures.

The results demonstrated the efficiency of this technique, although the quality of the provided transcripts is low: the relative WER improvement is between 28% and 40%, compared to the initial prior transcript. Moreover, we observe that the modified algorithm improves slightly the decoding speed, in spite of the additional computational cost due to the search synchronization.

Partial transcriptions, corresponding to the situation where prompted and unprompted speech segments alternate, are handled by a spotting mechanism integrated in the DDA. This algorithm is inspired from information retrieval techniques. Its role consists in dynamically detecting transcript islands when the recognizer encounters them. Our experiments have shown that the proposed technique yields very good results by using real prompts provided with the RTBF database. Moreover, this method seems to be quite robust to imperfect transcripts.

Finally, we have proposed a method that allows one to synchronize an imperfect transcript on the fly and to drive the ASR output with them, allowing it to determine the missing timestamps and to generate transcripts closer to the audio stream than slightly supervised algorithm [START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF]: with the same source of corpora, DDA approach produces a larger amount of data.

Prospects of improvement are related to the use of the generated corpus as source and target of the system adaptation process: by performing a system adaptation to the local context of a speech segment, one can expect a significant improvement of ASR system accuracy on the targeted segment. The quality of the transcript may be improved by such a recursive selftraining of the system, that may be viewed as a local version of the slightly supervised algorithm [START_REF] Lamel | Lightly supervised and unsupervised acoustic models training[END_REF].

More generally, Driven Decoding offers a generic scheme for the integration of a text stream into the search algorithm. In previous papers, we studied how this paradigm may be used for ASR system combination [START_REF] Lecouteux | System combination by driven decoding[END_REF] or for query-driven term spotting [START_REF] Rouvier | On-the-fly term spotting by phonetic filtering and request-driven decoding[END_REF]. We now plan to improve the usage of highly uncertain information sources by integrating confidence scores and temporal dependencies in the DDA-based framework.

Figure 1 :

 1 Figure 1: Synchronization of the search beams with the imperfect transcript by the DTW algorithm during an asynchronous decoding. Linguistic probabilities are fudged according to the quality of the alignment.

•

  ref: vous écoutez france info il est midi • mod: vous êtes à l'écoute de france info il est midi • ref: le gouvernement n' évitera sans doute pas • mod: le gouvernement n' évitera probablement pas • ref: une entreprise américaine bechtel géant du btp • mod: l' entreprise bechtel géant du bâtiment • ref: un cadre de carrefour • mod: un employé de carrefour • ref: HEC et polytechnique • mod: polytechnique et HEC • ref: donc on tiendra (%HESITATION) ce qu' on a dit c' est à dire à l' heure actuelle... eh bien il n'y aura

Figure 2 :

 2 Figure 2: Impact of the WER variation on the RFI input transcript.

Figure 3 :

 3 Figure 3: Scheme of the spotter integration with DDA. The transcript island detector computes an hypothesis-to-island matching score via a clustering algorithm. According to it, the spotter sets the decoder in driven-decoding mode. w 1 , .., w n are the words provided by the approximated transcript, H 1 , .., H n are the words of the current explored hypothesis in the A * algorithm.

•

  ref: the ****** ***** **** *** carotid artery is here carotide artery is • tra: the common **** *** side of • ref: the common bile duct side of • tra: cholecystitis **** *** when the plane between • ref: cholecystitis when the when the plane between • tra: are now slowly reaching *** • ref: are **** slowly reaching now • tra: cystic duct very rapidly here you see the anatomy • ref: cystic duct **** ****** here you see *** anatomy

  w 2 , w 3 and more match the hypothesis 0.2 if w 1 = t 1 and w 2 = t 2 and w 3 = t 3 0.4 if w 1 = t 1 and w 2 = t 2 and w 3 = t 3 0.9 if w 1 = t 1 and w 2 = t 2 and w 3 = t 3 0.99 if w 1 is not found

Table 1 :

 1 Outline Corr Sub Del Ins WER S.Err Conf.

	Int

Table 1 :

 1 Details

on the baseline system: Correct Rate (Corr), substitution rate (Sub), deletion rate (Del), Insertion rate (Ins), Word Error Rate (WER), Sentence Error rate (S.Err) and confidence Interval (Conf. Int)

Table 2 :

 2 Details on errors added in the artificial transcripts: Correct rate (Corr), Substitution rate (Sub), Deletion rate (Del), Insertion rate (Ins), Word Error Rate (WER) and Sentence Error rate (S.Err)

	• All disfluencies are removed
	• Spontaneous speech is changed to formal speech
	• Some words are replaced by synonyms

Table 3 :

 3 Automatic segmentation for the development set: number of segments according to the duration bins

	reports details on

Table 4 :

 4 Interpolation of the generic language model (LM-G) with a model trained on the exact transcript (LM-TrEx) and with the model trained on the imperfect transcript (LM-TrErr -10% WER)

Table 5 :

 5 Interpolation of the generic language model (LM-G) with the model trained on the exact transcript (LM-TrEx), and DDA with the exact transcript (alTrEx)

Table 6 :

 6 Interpolation of the generic language model (LM-G) with the model trained on the imperfect transcript (LM-TrErr -10% WER), and DDA with imperfect transcript (alTrErr -10% WER)

Table 7 :

 7

WER obtained by the baseline system (Baseline), the original transcript (Transcript), the DDA

  w) is computed for each word w; 5 while The ASR system generates an hypothesis h = {w 1 , ..., w n } do ∈ h satisfying condition tf (w j ) > T f req do 8 foreach position p g of the word w j into the prompt do

	4.2. Island spotting experiments
	4.2.1. Performance metric for spotting task
	6
		{C} ← {};
	14
	15	end
	16	end
	17	end
		if p g / ∈ {C} then
	30
		The transcript island is used for DDA if |C i | |h| > 0.5 (i.e. half of
		words are similar);
	32	end
		Algorithm 1: Development of the text island spotter.

7 foreach w j 9 foreach cluster C i do 10 if (min C i {p 0 , .., p m }) -2 ≤ p g ≤ (max C i {p 0 , .., p m }) + 5 11 then p g is added to C i ; 12 if two clusters overlap then 13 they are merged; 18 a new cluster is created with p g ; 19 end 20 end 21 end 22 if |{C}| > N max + δ then 23 T f req ← T f req + 1 ; 24 end 25 if |{C}| < N max -δ then 26 T f req ← T f req -1; 27 end 28 S i (I i ) is computed for each transcript island using equation 11; 29

Transcript island with the best confidence score is selected.;

  Recall F-measure Doc. number Conf. Int.

	FrInter 1	90.9%	98.89%	94.8%	478	2.37
	FrInter 2	94.3%	97.91%	96.1%	452	2.20
	FrInfo	93.7%	92.9%	91.5%	468	2.87
	RFI	98.9%	97.8%	98.4%	812	1.11
	Mean	94.4%	96.9%	95.2%	2210	2.13

Table 8 :

 8 Precision/recall, F-measure and confidence interval (for F-measure) of transcript island spotter on the exact transcript ESTER database. Doc. number is the total number of text segments based on manual annotation.

Table 9 :

 9 Precision/recall, F-measure and confidence interval (for F-measure) of transcript island spotter on the imperfect transcript ESTER database: 10% in FrInter 1 & 2, RFI and 20% in FrInfo. Doc. number is the total number of text segments based on manual annotation.

Table 11 :

 11 WER of the systems involved in the experiments; DDA+IT consists in Driven-Decoding with imperfect transcripts; DDA+PT is the DDA search algorithm, driven by the correct word sequence. Moreover, 50% of the text segments have been removed.

Table 12 :

 12 Number of matching words between the prompts and the ASR output (baseline system on the first column, island Driven Decoding on the second column).

  data WER after Adapt. Conf. Int.

	First Pass	nothing	45.6%	0.50
	MLLR adaptation	48h	41.6 %	0.49
	Baseline [22]	9h48	32.0 %	0.46
	DDA	11h38	31.2 %	0.46

Table 13 :

 13 Quantity of aligned data with slightly supervised adaptation (Baseline) or with the DDA, and impact of the aligned data on acoustic model adaptation

  28.7 5.8 11.0 45.6 Baseline P2 71.1 24.4 4.4 12.8 41.6 Slightly sup. P1 76.9 19.4 3.6 10.1 33.1 Slightly sup. P2 78.8 17.9 3.1 10.8 32.0

	DDA P1	77.0 19.4 3.6 9.4	32.4
	DDA P2	79.0 17.6 3.2 10.3 31.2

Table 14 :

 14 Precision on encountered errors: Correct Rate (Corr), substitution rate (Sub), deletion rate (Del), insertion rate (Ins) and word error rate (WER). P1 and P2 are respectively the first and second pass of the ASR system

  .

	Outline	Corr Sub Del Ins WER Conf. Int.
	Baseline	71.7 24.5 3.9 11.7 40.0	0.49
	DDA-full	73.2 23.0 3.7 11.6 38.4	0.49
	DDA-random 71.5 24.6 3.9 11.9 40.5	0.49

Table 15 :

 15 Acoustic models trained from scratch: Correct Rate (Corr), substitution rate (Sub), deletion rate (Del), insertion rate (Ins), word error rate (WER) and confidence interval (Conf. Int.). Baseline is a lightly supervised training approach, DDA-full use the same data sources, DDA-random use randomly selected segments in order to obtain the same amount of data that the Baseline

http://www.speech.cs.cmu.edu/cgi-bin/cmudict


G Linares

B Lecouteux
email: benjamin.lecouteux@lia.univ-avignon.fr

D Matrouf
email: driss.matrouf@lia.univ-avignon.fr

P Nocera

PHONE DURATION MODELS FOR FAST BROADCAST NEWS TRANSCRIPTIONS

Phone duration modeling in HMM based LVCSR systems has been largely studied during last few years. In this paper, we address the problem of duration modeling in the particular context of fast decoding on LVCSR task. Discrete distributions are integrated in the LIA's broadcast news transcription system, and influence of duration modeling is studied using various pruning schemes. Experimental results show that duration modeling improve significantly the pruning efficiency. In a second time, we show that durations are intrinsecly acoustic-context dependent. Crossed experiments are conducted combining context independent acoustic models and context dependent duration models. We show how durations are affected by acoustic context. At last, we propose a rate dependent modeling of phone durations. This method outperforms significantly our rate independent duration model based system. Globally, integration of rate dependent models allows a absolute WER gain of 3.3%.

INTRODUCTION

HMM based systems are known to be deficient in modeling phone durations. The implicit duration distribution of Markov Models is determined by state to state transitions. Therefore, durations follow a geometric distribution which do not reflect the true duration distributions. This observation has been the start point of several works proposing and evaluating various modeling techniques. Most authors propose to plug explicit duration functions into HMMs, using probability density functions, such as gaussian, gamma functions ( [START_REF] Levinson | Continuously variable duration hidden markov models for automatic speech recognition[END_REF]), or discrete distributions ( [START_REF] Mitchell | Modeling duration in a hidden markov model with the exponential family[END_REF]). Other works propose approaches based on low level signal analysis ( [START_REF] Nedel | Duration normalization for improved recognition of spontaneous and read speech via missing feature methods[END_REF]), or specific HMM topologies which model durations in a more realistic way( [START_REF] Ney | Acoustic modelling of phoneme units for continuous speech recognition[END_REF]). Most of these studies have been achieved on limited tasks (digit or letter recogniton, small vocabulary,...), on LVCSR using large recognition systems, where the acoustic space is deeply explorated. In a context of fast decoding process -less than 5x Real Time (RT)-the search space is reduced to the few best hypothesis. We suggest that introduction of explicit duration models in a fast decoding engine may reduce the rank of best hypothesis in the decoding stack, leading to improvement of both decoding speed and recognition performances. We investigate this idea using the LIA's Broadcast News System. Experiments are achivied on the French radiophonic database developed for the evaluation campaign ESTER ( [START_REF] Galliano | Ester, une campagne d'évaluation des systèmes d'indexation d'émissions radiophoniques en franc ¸ais[END_REF]). First, we integrate discrete duration models to the recognition system. Two pruning schemes are defined, corresponding to 2xRT and 10xRT systems on a 3Ghz pentium 4 processor. Results with and without duration modeling are compared, and the influence of pruning on duration efficiency is evaluated. In a second part, we study the influence of acoustic context on duration modeling. Context dependent duration models and context independent ones are evaluated using various acoustic units. An other point largely studied concerns the effect of speech rates on phone durations ( [START_REF] Guiliani | A family of parallel hidden markov models[END_REF]). In the last section, we describe and evaluate a method for rate-dependent models estimation, and their integration to the full recognition process. Here, experiments are conducted using the two shows from the full ESTER corpus where our baseline system obtains the best and the worse recognition rates (France Inter 1998-17-12, 7h/8h show and RFI 2000-9-12, 9h30/10h30 show).

INTEGRATING PHONE DURATION MODELS IN LVCSR SYSTEM

Transcription System

The full transcription system is composed of 3 independent steps. First, speech signal is parametrized by 39 coefficient vectors: 12-mel cepstral coefficients plus energy and their first and second order derivative parameters.

The second step performs a segmentation into acoustic macroclasses. The aim of this step is to split each show signal into segments which will be suitable for the recognition engine. This segmentation is achieved using a hierarchical GMM classifier. A first classification process extracts speech segments from the full shows. A second one splits segments into clean-background and musical-background parts. At last, a gender dependent segmentation is achieved. At the end of this step, cepstral mean subtraction and variance normalization are performed on each extracted segment. This process splits a show into 6 classes which will be processed by speech recognizer using specific acoustic models (gender and background dependent). Third step consists in the decoding process. The LIA's speech recognition system is based on classical HMMs for acoustic modeling and n-gram language models. The next section describes the specific search algorithm of our recognition engine and the associated pruning strategy.

For ESTER decoding, we use a 20k word lexicon extracted from the train corpus. Language model results of a linear interpolation of an ESTER specific trigram model and a journalistic model trained on the Le Monde-1987-2002 database.

Speeral

Experiments are conducted using SPEERAL [START_REF] Massonié | Phoneme lattice based a* search algorithm for speech recognition[END_REF], the LVCSR system developed at the LIA. This system is able to use n-gram language models and classical 3-state left-to-right HMMs for acoustic modeling. Context dependent models can be used including cross word contexts. Search engine is an asynchronous stack decoder based on a modified A* algorithm. Rather than word by word, the A* used in Speeral is based on a phoneme lattice previously generated.

The search complexity is reduced using several pruning strategies. First, beam size and extensive backtracks are con-trolled by static thresholds. Moreover, a cache memory stores partially explorated paths, saving new exploration of already evaluated paths. At last, a fast probe function allows a good selection of well scored paths, satisfying A* specific constraint of probe function optimality. A full description of Speeral can be found in [START_REF] Massonié | Phoneme lattice based a* search algorithm for speech recognition[END_REF].

Using full context acoustic models and large trigram language models, light pruning schemes lead to a decoding time of about 60x Real Time. This ratio can be reduced to 2xRT using only right diphones, fast likelihood computation algorithm and a very strict pruning scheme. Of course, such a velocity improvement leads to a significant WER increase, about 60% relative.

Integrating duration models

As explained in the last section, the LIA's recognition engine explores a phone graph. For efficiency reasons, this graphis is not really built in a first pass, but it is locally built and deleted during search. The cost of each developed branch is evaluated addressing request to acoustic models about likelihood of an observation sequence knowing a given acoustic unit and a fixed duration. Therefore, integration of duration probabilities into this process can be achivied easily, combining strictly acoustic scores to duration likelihoods. On the other hand, there is no information about the word crossed on each node and usage of word-dependent duration units should by very complex. Considering complexity and computational cost of integration of such information to this search algorithm, we have chosen to use phone-level durations, in spite of few reported experiments which suggest better performances obtained using word specific durations. Discrete distributions are estimated collecting phone duration from the full train corpus, grouping all acoustic macroclasses from the first segmentation process. Therefore, phone lattices are built estimating the log-likelihood LogL(x|M i , T i ) of a frame sequence x of length |x|, knowing the acoustic model M i and the duration model T i :

LogL(x|M i , T i ) = LogL(x|M i ) + λ.LogL(|x||T i )
The first term represents the acoustic log-likelihood. It is estimated using a classical Viterbi decoding, without any temporal information. The computation of the second term (duration log-likelihood) consists in accessing directly to the histogram modeling duration of unit i. The duration fudge factor λ has been chosen empirically (λ = 2). We can see that additional computational cost of this method can be neglected, compared to the full decoding process cost.

PRUNING SCHEMES AND DURATION MODELS

The fist part of ESTER corpus includes shows from two French radios, France Inter and Radio France International (RFI). Shows from RFI are significantly badly decoded by the 60xRT system. These lower performances are due to larger acoustic and linguistic variability, due to larger interspeaker variability. We defined two pruning configuration of Speeral, allowing about 10xRT (F1 system) decoding and (F0 system) 2xRT decoding. These systems use diphone models WER obtained by these baseline systems are compared to systems using duration models (cf. Results shows first that using the slow system, duration models lead to an absolute WER reduction of 2.4% on RFI show, and remains equivalent on FranceInter show (-0.1%). This evaluation confirms that duration models are more efficient on adverse acoustic conditions. Nevertheless, duration allows a significant improvement of fast system performances: absolute gain of 3.5% and 1.6 % are obtained respectively on RFI and FranceInter shows. These results confirm that duration models reduce the deep of the best hypthesis in the search lattice, specialy in bad acoustic conditions.

CONTEXT-DEPENDENT VERSUS CONTEXT INDEPENDENT DURATION MODELS

Likelihood computation consumes a major part of CPU time in fast ASR systems. In spite of fast likelihood computation techniques, this complexity depends significativly of HMMs complexity. We tested the influence of duration models on low complexity models, and we compared them to recognition rates obtained using our context dependent models. 38 acoustic units are modeled by 3-state HMMs.

Each GMM is a mixture of 32 gaussian functions; therefore, these models provides very low complexity and allows real time ratio lower than 1xRT

2:WER of baseline system based on context independent units (F2-BASE), using context independent durations (F2-CI-DUR) , and context dependent durations (it F2-CD-DUR). These last configuration outperforms (F2-CD-DUR); this result illustrate context dependence of phone durations.

We can show that context independent models are not able to exploit temporal information contained by duration models in the F1 case: usage of phone duration affects system performances; WER don't change on RFI show but they increase of about 2% of FranceInter show. Nevertheless, light gains are still obtained using F0 system. This experiment suggest that duration are intrinscely context dependent. In order to confirm this assumption, we perform an other experiment where context dependent duration models are plugged on context independent acoustic models. Only duration modeling differs between the two model sets. Results (cf. Results shows that rate dependent models outperforms classical ones of 1.1% (absolute) both on RFI and Fran-ceInter shows, in spite of greater WER gain obtained on that last test signal using standard duration models. Cumulative WER absolute gains reach 2% and 4.5% on that two test shows.

CONCLUSION

We have studied phone duration modeling for fast decoding of broadcast news. Our experiments on a task of French broadcast news transcriptions show that usage of phone duration models can improve system performances. WER decreases significantly using restrictive pruning schemes, in spite of relatively low gains obtained using slower systems. Moreover, reported results using duration suggest that temporal information is more efficient when acoustic models fail. At last, speech rate dependent models improve significantly performances of duration based systems. Nevertheless, this last method is based on a speech estimation which is cputime consuming.

table 1

 1

			).
		Word Error Rate (%)
	System F-Inter	RFI
	F0	40.1	61.1
	F0-DUR	39.2	56.6
	F1	30.3	50.7
	F1-DUR	30.4	48.3

Table 1 :

 1 Baseline

	2xRT (noted F0) and 10xRT systems (noted F1
	versus Duration models based systems (F0-DUR and F1-DUR)

  . Results of these experiments are reported in the table 2.

		Word Error Rate (%)
	System	F-Inter	RFI
	F0-BASE	49.7	65.9
	F0-CI-DUR	48.7	63.6
	F1-BASE	41.6	59.4
	F1-DUR	43.6	59.4

Table

  table 3) shows that usage of acoustic context duration models leads to WER decreasing, in spite of identical acoustic model precision. Another interesting point is that gain obtained on RFI and FranceInter shows are similar, in spite of best efficiency reported on RFI, using context dependent models. The weakness of such low complexity models seems to be partially compensated by temporal information.

		Word Error Rate (%)
	System	F-Inter	RFI
	F2-BASE	49.7	65.9
	F2-CI-DUR	48.7	63.6
	F2-CD-DUR	45.9	62.1

Table 2 :

 2 these set are merged into 6 subband. Rate Subband overlap themselves in order to get enough learn data for model estimation. So, rate dependent models are learned on each subset. The figure1shows rate repartition and chosen subands. Speech rate repartion on ESTER corpus and rate subband. This curve shows the number of speech segment for for each speech rate ratio. Speech rate is expressed in number of frame per phone. The 6 subband correspond to 6 overlapping corpus subsets. Each of them is used for the learning of a rate dependent duration model.Before each recognition run, we estimate the rate performing a fast decoding. Usage of full recognizer instead of a simple phone recognizer is expensive in term of CPU time but allows more robust estimation of speech rate. This model selection process take about 1xRT. The second pass performs the decoding based on the duration model selected during the first pass. The table4shows results obtained using this technique.

	WER of baseline system based on context independent
	units (F2-BASE), using context independent durations (F2-CI-DUR)
	, and context dependent durations (it F2-CD-DUR). These last
	configuration outperforms (F2-CD-DUR); this result illustrate con-
	text dependence of phone durations.
	5. RATE DEPENDENT PHONE DURATION
	MODELS
	Generally, duration based systems do not take account of
	speech rate. Nevertheless, phone durations depends of the
	global rate of a speech segment. Moreover, deformations
	are phone dependent, and a simple linear normalization should
	not provides a good modeling of duration elasticity. [1] pro-
	pose to define 3 rate schemes (low, medium and hight rate).
	For each of them, rate-dependent models are learned; recog-
	nition process estimate the speech rate a select dynamically
	the best rate dependent model. We adopt this approach
	defining 6 rates schemes. Each of them correspond to a
	speech rate band. These bands are determined in two step;
	first, the train corpus is splitted into 50 subset following the
	rate factor of each speech segment. During second step,

Table 4 :

 4 WER using Rate Dependent duration models (RD) versus standard durations models and 2xRT baseline system (F0-BASE).

We work now on the integration of a fast speech rate estimator in the search using the A* probe function. Moreover, we work now on estimation of duration models depending on the high level context, such as speaker or environment.