M Benzeguiba

R De Mori

O Deroo

S Dupont
email: dupont@multitel.be

T Erbes

D Jouvet

L Fissore

P Laface

A Mertins

C Ris

R Rose

V Tyagi

C Wellekens

Automatic Speech Recognition and Speech Variability: a Review

Major progress is being recorded regularly on both the technology and exploitation of Automatic Speech Recognition (ASR) and spoken language systems. However, there are still technological barriers to flexible solutions and user satisfaction under some circumstances. This is related to several factors, such as the sensitivity to the environment (background noise), or the weak representation of grammatical and semantic knowledge.

Current research is also emphasizing deficiencies in dealing with variation naturally present in speech. For instance, the lack of robustness to foreign accents precludes the use by specific populations. Also, some applications, like directory assistance, particularly stress the core recognition technology due to the very high active vocabulary (application perplexity). There are actually many factors affecting the speech realization: regional, sociolinguistic, or related to the environment or the speaker herself. These create a wide range of variations that may not be modeled correctly (speaker, gender, speaking rate, vocal effort, regional accent, speaking style, non stationarity...), especially when resources for system training are scarce. This papers outlines current advances related to these topics.

Introduction

It is well known that the speech signal not only conveys the linguistic information (the message) but also a lot of information about the speaker himself: gender, age, social and regional origin, health and emotional state and, with a rather strong reliability, his identity. Beside intra-speaker variability (emotion, health, age), it is also commonly admitted that the speaker uniqueness results from a complex combination of physiological and cultural aspects [START_REF] Garvin | Speaker identification and message identification in speech recognition[END_REF][START_REF] Nolan | The phonetic bases of speaker recognition[END_REF].

Characterization of the effect of some of these specific variations, together with related techniques to improve ASR robustness is a major research topic. As a first obvious theme, the speech signal is non-stationary. The power spectral density of speech varies over time according to the source signal, which is the glottal signal for voiced sounds, in which case it affects the pitch, and the configuration of the speech articulators (tongue, jaw, lips...). This signal is modeled, through Hidden Markov Models (HMMs), as a sequence of stationary random regimes. At a first stage of processing, most ASR front-ends analyze short signal frames (typically covering 30 ms of speech) on which stationarity is assumed. Also, more subtle signal analysis techniques are being studied in the framework of ASR.

The effects of coarticulation have motivated studies on segment based, articulatory, and context dependent (CD) modeling techniques. Even in carefully articulated speech, the production of a particular phoneme results from a continuous gesture of the articulators, coming from the configuration of the pre-vious phonemes, and going to the configuration of the following phonemes (coarticulation effects may indeed stretch over more than one phoneme). In different and more relaxed speaking styles, stronger pronunciation effects may appear, and often lead to reduced articulation. Some of these being particular to a language (and mostly unconscious). Other are related to regional origin, and are referred to as accents (or dialects for the linguistic counterpart) or to social groups and are referred to as sociolects. Although some of these phenomena may be modeled appropriately by CD modeling techniques, their impact may be more simply characterized at the pronunciation model level. At this stage, phonological knowledge may be helpful, especially in the case of strong effects like foreign accent. Fully data-driven techniques have also been proposed.

Following coarticulation and pronunciation effects, speaker related spectral characteristics (and gender) have been identified as another major dimension of speech variability. Specific models of frequency warping (based on vocal tract length differences) have been proposed, as well as more general feature compensation and model adaptation techniques, relying on Maximum Likelihood or Maximum a Posteriori criteria. These model adaptation techniques provide a general formalism for re-estimation based on moderate amounts of speech data.

Besides these speaker specific properties outlined above, other extra-linguistic variabilities are admittedly affecting the signal and ASR systems. A person can change his voice to be louder, quieter, more tense or softer, or even a whisper; Also, some reflex effects exist, such as speaking louder when the environment is noisy, as reported in [START_REF] Lombard | Le signe de l'élévation de la voix[END_REF].

Speaking faster or slower, also has influence on the speech signal. This impacts both temporal and spectral characteristics of the signal, both affecting the acoustic models. Obviously, faster speaking rates may also result in more frequent and stronger pronunciation changes.

Speech also varies with age, due to both generational and physiological reasons. The two "extremes" of the range are generally put at a disadvantage due to the fact that research corpora, as well as corpora used for model estimation, are typically not designed to be representative of children and elderly speech. Some general adaptation techniques can however be applied to counteract this problem.

Emotions are also becoming a hot topic, as they can indeed have a negative effect on ASR; and also because added-value can emerge from applications that are able to identify the user emotional state (frustration due to poor usability for instance).

Finally, research on recognition of spontaneous conversations has allowed to highlight the strong detrimental impact of this speaking style; and current studies are trying to better characterize pronunciation variation phenomena inherent in spontaneous speech.

This paper reviews current advances related to these topics. It focuses on variations within the speech signal that make the ASR task difficult. These variations are intrinsic to the speech signal and affect the different levels of the ASR processing chain. For different causes of speech variation, the paper summarizes the current literature and highlights specific feature extraction or modeling weaknesses.

The paper is organized as follows. In a first section, variability factors are reviewed individually according to the major trends identified in the literature. The section gathers information on the effect of variations on the structure of speech as well as the ASR performance.

Methodologies that can help analyzing and diagnose the weaknesses of ASR technology can also be useful. These diagnosis methodologies are the object of section 3. A specific methodology consists in performing comparisons between man and machine recognition. This provides an absolute reference point and a methodology that can help pinpointing the level of interest. Man-machine comparison also strengthens interdisciplinary insights from fields such as audiology and speech technology.

In general, this review further motivates research on the acoustic, phonetic and pronunciation limitations of speech recognition by machines. It is for instance acknowledged that pronunciation variation is a major factor of reduced performance (in the case of accented and spontaneous speech). Section 4 reviews ongoing trends and possible breakthroughs in general feature extraction and modeling techniques that provides more resistance to speech production variability. The issues that are being addressed include the fact that temporal representations/models may not match the structure of speech, as well as the fact that some analysis and modeling assumptions can be detrimental. General techniques such as compensation, adaptation, multiple models, additional acoustic cues and more accurate models are surveyed.

Speech Variability Sources

Prior to reviewing the most important causes of intrinsic variation of speech, it is interesting to briefly look into the effects. Indeed, improving ASR systems regarding sources of variability will mostly be a matter of counteracting the effects. Consequently, it is likely that most of the variability-proof ASR techniques actually address several causes that produce similar modifications of the speech.

We can roughly consider three main classes of effects; first, the fine structure of the voice signal is affected, the color and the quality of the voice are modified by physiological or behavioral factors. The individual physical characteristics, the smoking habit, a disease, the environmental context that make you soften your voice or, on the contrary, tense it, ... are such factors. Second, the long-term modulation of the voice may be modified, intentionally -to transmit high level information such as emphasizing or questioning -or not -to convey emotions. This effect is an integral part of the human communication and is therefore very important. Third, the word pronunciation is altered. The acoustic realization in terms of the core spoken language components, the phonemes, may be deeply affected, going from variations due to coarticulation, to substitutions (accents) or suppressions (spontaneous speech).

As we will further observe in the following sections, some variability sources can hence have multiple effects, and several variability sources obviously produce effects that belong to the same category. For instance, foreign accents, speaking style, rate of speech, or children speech all cause pronunciation alterations with respect to the "standard form". The actual alter-ations that are produced are however dependent on the source of variability, and on the different factors that characterize it.

Although this is outside the scope of this paper, we should add a fourth class of effects that concerns the grammatical and semantic structure of the language. Sociological factors, partial knowledge of the language (non-nativeness, childhood, ...), may lead to important deviations from the canonical language structure.

Foreign and regional accents

While investigating the variability between speakers through statistical analysis methods, [START_REF] Huang | Analysis of speaker variability[END_REF] found that the first two principal components of variation correspond to the gender (and related to physiological properties) and accent respectively. Indeed, compared to native speech recognition, performance degrades when recognizing accented speech and nonnative speech [START_REF] Kubala | Comparative experiments on large vocabulary speech recognition[END_REF][START_REF] Lawson | Effect of foreign accent on speech recognition in the NATO N-4 corpus[END_REF]. In fact accented speech is associated to a shift within the feature space [START_REF] Vancompernolle | Recognizing speech of goats, wolves, sheep and[END_REF]. Good classification results between regional accents are reported in [START_REF] Draxler | Identification of regional variants of high german from digit sequences in german telephone speech[END_REF] for human listeners on German SpeechDat data, and in [START_REF] Lin | Phoneme-less hierarchical accent classification[END_REF] for automatic classification between American and British accents which demonstrates that regional variants correspond to significantly different data. For native accents, the shift is applied by large groups of speakers, is more or less important, more or less global, but overall acoustic confusability is not changed significantly. In contrast, for foreign accents, the shift is very variable, is influenced by the native language, and depends also on the level of proficiency of the speaker.

Non-native speech recognition is not properly handled by speech models estimated using native speech data. This issue remains no matter how much dialect data is included in the training [START_REF] Beattie | An integrated multidialect speech recognition system with optional speaker adaptation[END_REF]. This is due to the fact that non-native speakers can replace an unfamiliar phoneme in the target language, which is absent in their native language phoneme inventory, with the sound considered as the closest in their native language phoneme inventory [START_REF] Flege | Interaction between the native and second language phonetic subsystems[END_REF]. This behavior makes the nonnative alterations dependent on both the native language and the speaker. Some sounds may be replaced by other sounds, or inserted or omitted, and such insertion/omission behavior cannot be handled by the usual triphone-based modeling [START_REF] Jurafsky | What kind of pronunciation variation is hard for triphones to model?[END_REF].

Accent classification is also studied since many years [START_REF] Arslan | Language accent classification in american english[END_REF], based either on phone models [START_REF] Kumpf | Automatic accent classification of foreign accented australian english speech[END_REF][START_REF] Teixeira | Accent identification[END_REF] or specific acoustic features [START_REF] Fung | Fast accent identification and accented speech recognition[END_REF].

Speech recognition technology is also used in foreign language learning for rating the quality of the pronunciation [START_REF] Eskenazi | Detection of foreign speakers' pronunciation errors for second language training-preliminary results[END_REF][START_REF] Franco | Combination of machine scores for automatic grading of pronunciation quality[END_REF][START_REF] Neumeyer | Automatic text-independent pronunciation scoring of foreign language student speech[END_REF][START_REF] Townshend | Automatic text-independent pronunciation scoring of foreign language student speech[END_REF]. Experiments showed that the provided rating is correlated with human expert ratings [START_REF] Cucchiarini | Different aspects of expert pronunciation quality ratings and their relation to scores produced by speech recognition algorithms[END_REF][START_REF] Neumeyer | Automatic scoring of pronunciation quality[END_REF][START_REF] Witt | Phone-level pronunciation scoring and assessment for interactive language learning[END_REF] when sufficient amount of speech is available.

Proper and foreign name processing is another topic strongly related with foreign accent. Indeed, even if speakers are not experts in all foreign languages, neither are they linguistically naive, hence they may use different systems or sub-systems of rules to pronounce unknown names which they perceive to be non-native [START_REF] Fitt | The pronunciation of unfamiliar native and nonnative town names[END_REF]. Foreign names are hard to pronounce for speakers who are not familiar with the names and there are no standardized methods for pronouncing proper names [START_REF] Gao | Innovative approaches for large vocabulary name recognition[END_REF]. Native phoneme inventories are enlarged with some phonemes of foreign languages in usual pronunciations of foreign names, especially in some languages [START_REF] Eklund | Xenophones: an investigation of phone set expansion in swedish and implications for speech recognition and speech synthesis[END_REF]. Determining the ethnic origin of a word improves pronunciation models [START_REF] Llitjos | Knowledge of language origin improves pronunciation accuracy of proper names[END_REF] and is useful in predicting additional pronunciation variants [START_REF] Bartkova | Generating proper name pronunciation variants for automatic speech recognition[END_REF][START_REF] Maison | Pronunciation modeling for names of foreign origin[END_REF].

Speaker physiology

Beside the regional origin, another speaker-dependent property that is conveyed through the speech signal results from the shape of the vocal apparatus which determines the range within which the parameters of a particular speaker's voice may vary. From this point of view, a very detailed study of the speechspeaker dichotomy can be found in [START_REF] Mokhtari | An acoustic-phonetic and articulatory study of speech-speaker dichotomy[END_REF].

The impact of inter-speaker variability on the automatic speech recognition performance has been acknowledged for years. In [START_REF] Huang | On speaker-independent, speakerdependent, and speaker-adaptive speech recognition[END_REF][START_REF] Lee | A study on speaker adaptation of the parameters of continuous density Hidden Markov Models[END_REF][START_REF] Schwartz | The BBN BYBLOS continuous speech recognition system[END_REF], the authors mention error rates two to three times higher for speaker-independent ASR systems compared with speaker-dependent systems. Methods that aims at reducing this gap in performance are now part of state-of-theart commercial ASR systems.

Speech production can be modeled by the so-called sourcefilter model [START_REF] Fant | Acoustic theory of speech production[END_REF] where the "source" refers to the air stream generated by the lungs through the larynx and the "filter" refers to the vocal tract, which is composed of the different cavities situated between the glottis and the lips. Both of the components are inherently time-varying and assumed to be independent of each other.

The complex shape of the vocal organs determines the unique "timbre" of every speaker. The glottis at the larynx is the source for voiced phonemes and shapes the speech signal in a speaker characteristic way. Aside from the long-term F0 statistics [START_REF] Carey | Robust prosodic features for speaker identification[END_REF][START_REF] Iivonen | Development of a multiparametric speaker profile for speaker recognition[END_REF][START_REF] Markel | Long-term feature averaging for speaker recognition[END_REF] which are probably the most perceptually relevant parameters (the pitch), the shape of glottal pulse will affect the long-term overall shape of the power spectrum (spectral tilt) [START_REF] Nolan | The phonetic bases of speaker recognition[END_REF] and the tension of vocal folds will affect the voice quality. The vocal tract, can be modeled by a tube resonator [START_REF] Fant | Acoustic theory of speech production[END_REF][START_REF] Laver | Principles of phonetics[END_REF]. The resonant frequencies (the formants) are structuring the global shape of the instantaneous voice spectrum and are mostly defining the phonetic content and quality of the vowels.

Modeling of the glottal flow is a difficult problem and very few studies attempt to precisely decouple the source-tract components of the speech signal [START_REF] Blomberg | Adaptation to a speaker's voice in a speech recognition system based on synthetic phoneme references[END_REF][START_REF] Bozkurt | Zeros of z-transform representation with application to source-filter separation in speech[END_REF][START_REF] Plumpe | Modeling of the glottal flow derivative waveform with application to speaker identification[END_REF]. Standard feature extraction methods (PLP, MFCC) simply ignore the pitch component and roughly compensate for the spectral tilt by applying a pre-emphasis filter prior to spectral analysis or by applying band-pass filtering in the cepstral domain (the cepstral liftering) [START_REF] Juang | On the use of bandpass liftering in speech recognition[END_REF].

On the other hand, the effect of the vocal tract shape on the intrinsic variability of the speech signal between different speakers has been widely studied and many solutions to compensate for its impact on ASR performance have been proposed: "speaker independent" feature extraction, speaker normalization, speaker adaptation. The formant structure of vowel spectra has been the subject of early studies [START_REF] Peterson | Control methods used in a study of the vowels[END_REF][START_REF] Pols | Perceptual and physical space of vowel sounds[END_REF][START_REF] Potter | Toward the specification of speech[END_REF] that amongst other have established the standard view that the F1-F2 plane is the most descriptive, two-dimensional representation of the phonetic quality of spoken vowel sounds. On the other hand, similar studies underlined the speaker specificity of higher formants and spectral content above 2.5 kHz [START_REF] Pols | Perceptual and physical space of vowel sounds[END_REF][START_REF] Saito | Frequency spectrum deviation between speakers[END_REF]. Another important observation [START_REF] Ladefoged | Information conveyed by vowels[END_REF][START_REF] Nearey | Phonetic feature systems for vowels[END_REF][START_REF] Peterson | Control methods used in a study of the vowels[END_REF][START_REF] Potter | Toward the specification of speech[END_REF] suggested that relative positions of the formant frequencies are rather constant for a given sound spoken by different speakers and, as a corollary, that absolute formant positions are speaker-specific. These observations are corroborated by the acoustic theory applied to the tube resonator model of the vocal tract which states that positions of the resonant frequencies are inversely proportional to the length of the vocal tract [START_REF] Flanagan | Speech analysis and synthesis and perception[END_REF][START_REF] O'saughnessy | Speech communication -human and machine[END_REF]. This observation is at the root of different techniques that increase the robustness of ASR systems to inter-speaker variability (cf. 4.1.2 and 4.2.1).

Speaking style and spontaneous speech

In spontaneous casual speech, or under time pressure, reduction of pronunciations of certain phonemes, or syllables often happen. It has been suggested that this "slurring" affects more strongly sections that convey less information. In contrast, speech portions where confusability (given phonetic, syntactic and semantic cues) is higher tend to be articulated more carefully, or even hyperarticulated. Some references to such studies can be found in [START_REF] Bard | Taking the hit: leaving some lexical competition to be resolved post-lexically[END_REF][START_REF] Janse | Word perception in fast speech: artificially time-compressed vs. naturally produced fast speech[END_REF][START_REF] Lindblom | Explaining phonetic variation: a sketch of the H&H theory[END_REF][START_REF] Sotillo | Is hypo-articulation lexically constrained?[END_REF], and possible implications to ASR in [START_REF] Bell | Effects of disfluencies, predictability, and utterance position on word form variation in english conversation[END_REF].

This dependency of casual speech slurring on identified factors holds some promises for improving recognition of spontaneous speech, possibly by further extending the context dependency of phonemes to measures of such perplexity, with however very few research ongoing to our knowledge, except maybe in the use of phonetic transcription for multi-word compounds or user formulation [START_REF] Colibro | Learning pronunciation and formulation variants in continuous speech applications[END_REF] (cf. 4.3).

Research on spontaneous speech modeling is nevertheless very active. Several studies have been carried out on using the Switchboard spontaneous conversations corpus. An appealing methodology has been proposed in [START_REF] Weintraub | Effect of speaking style on LVCSR performance[END_REF], where a comparison of ASR accuracy on the original Switchboard test data and on a reread version of it is proposed. Using modeling methodologies that had been developed for read speech recognition, the error rate obtained on the original corpus was twice the error rate observed on the read data.

Techniques to increase accuracy towards spontaneous speech have mostly focused on pronunciation studies 1 . As a fundamental observation, the strong dependency of pronunciation phenomena with respect to the syllable structure has been highlighted in [START_REF] Adda-Decker | Investigating syllabic structures and their variation in spontaneous french[END_REF][START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF]. As a consequence, extensions of acoustic modeling dependency to the phoneme position in a syllable and to the syllable position in word and sentences have been proposed. This class of approaches is sometimes referred to as long-units [START_REF] Messina | Context dependent long units for speech recognition[END_REF].

Variations in spontaneous speech can also extend beyond the typical phonological alterations outlined previously. Disfluencies, such as false starts, repetitions, hesitations and filled pauses, need to be considered. The reader will find useful information in the following papers: [START_REF] Byrne | Automatic recognition of spontaneous speech for access to multilingual oral history archives[END_REF][START_REF] Furui | Introduction to the special issue on spontaneous speech processing[END_REF].

There are also regular workshops specifically addressing the research activities related to spontaneous speech modeling and recognition [START_REF]Disfluency in spontaneous speech (diss'05)[END_REF]. Regarding the topic of pronunciation variation, the reader should also refer to [241].

Rate of Speech

Rate of Speech (ROS) is considered as an important factor which makes the mapping process between the acoustic signal and the phonetic categories more complex.

Timing and acoustic realization of syllables are affected due in part to the limitations of the articulatory machinery, which may affect pronunciation through phoneme reductions (typical to fast spontaneous speech), time compression/expansion, changes in the temporal patterns, as well as smaller-scale acoustic-phonetic phenomena.

In [START_REF] Janse | Word perception in fast speech: artificially time-compressed vs. naturally produced fast speech[END_REF], production studies on normal and fast-rate speech are reported. They have roughly quantified the way people compress some syllables more than others. Note also that the study 1 besides language modeling which is out of the scope of this paper reports on a series of experiments investigating how speakers produce and listeners perceive fast speech. The main research question is how the perception of naturally produced fast speech compares to the perception of artificially timecompressed speech, in terms of intelligibility.

Several studies also reported that different phonemes are affected differently by ROS. For example, compared to consonants, the duration of vowels is significantly more reduced from slow to fast speech [START_REF] Kuwabara | Acoustic and perceptual properties of phonemes in continuous speech as a function of speaking rate[END_REF].

The relationship between speaking rate variation and different acoustic correlates are usually not well taken into account in the modeling of speech rate variation for automatic speech recognition, where it is typical that the higher the speaking rate is, the higher the error rate is. Usually, slow speaking rate does not affect performance; however, when people hyperarticulate, and make pauses among syllables, speech recognition performance can also degrade a lot.

In automatic speech recognition, the significant performance degradations [START_REF] Martinez | Characteristics of slow, average and fast speech and their effects in large vocabulary continuous speech recognition[END_REF][START_REF] Mirghafori | Fast speakers in large vocabulary continuous speech recognition: analysis & antidotes[END_REF][START_REF] Siegler | On the effect of speech rate in large vocabulary speech recognition system[END_REF] caused by speaking rate variations stimulated many studies for modeling the spectral effects of speaking rate variations. The schemes presented in the literature generally make use of ROS (Rate of Speech) estimators. Almost all existing ROS measures are based on the same principle which is how to compute the number of linguistic units (usually phonemes or syllables) in the utterance. So, usually, a speaking rate measure based on manually segmented phones or syllables is used as a reference to evaluate a new ROS measure. Current ROS measures can be divided into (1) lexically-based measures and (2) acoustically-based measures. The lexically-based measures estimate the ROS by counting the number of linguistic units per second using the inverse of mean duration [START_REF] Siegler | On the effect of speech rate in large vocabulary speech recognition system[END_REF], or mean of m [START_REF] Mirghafori | Fast speakers in large vocabulary continuous speech recognition: analysis & antidotes[END_REF]. To reduce the dependency on the phone type, a normalization scheme by the expected phone duration [START_REF] Martinez | Characteristics of slow, average and fast speech and their effects in large vocabulary continuous speech recognition[END_REF] or the use of phone duration percentile [START_REF] Siegler | Measuring and compensating for the effects of speech rate in large vocabulary continuous speech recognition[END_REF] are introduced. These kinds of measures are effective if the segmentation of the speech signal provided by a speech recognizer is reliable. In practice this is not the case since the recognizer is usually trained with normal speech. As an alternative technique, acoustically-based measures are proposed. These measures estimate the ROS directly from the speech signal without recourse to a preliminary segmentation of the utterance. In [START_REF] Morgan | Combining multiple estimators of speaking rate[END_REF], the authors proposed the mrate measure (short for multiple rate). It combines three independent ROS measures, i.e., [START_REF] Aalburg | Foreign-accented speakerindependent speech recognition[END_REF] the energy rate or enrate [START_REF] Morgan | Speech recognition using on-line estimation of speaking rate[END_REF], (2) a simple peak counting algorithm performed on the wideband energy envelope and (3) a sub-band based module that computes a trajectory that is the average product over all pairs of compressed sub-band energy trajectories. A modified version of the mrate is also proposed in [START_REF] Beauford | Compensating for variation in speaking rate[END_REF]. In [START_REF] Tuerk | Modeling speaking rate using a between frame distance metric[END_REF], the authors found that successive feature vectors are more dependent (correlated) for slow speech than for fast speech. An Euclidean distance is used to estimate this dependency and to discriminate between slow and fast speech. In [START_REF] Falthauser | On-line speaking rate estimation using gaussian mixture models[END_REF], speaking rate dependent GMMs are used to classify speech spurts into slow, medium and fast speech. The output likelihoods of these GMMs are used as input to a neural network whose targets are the actual phonemes. The authors made the assumption that ROS does not affect the temporal dependencies in speech, which might not be true.

It has been shown that speaking rate can also have a dramatic impact on the degree of variation in pronunciation [START_REF] Fosler-Lussier | Effects of speaking rate and word predictability on conversational pronunciations[END_REF][START_REF] Greenberg | The uninvited guest: information's role in guiding the production of spontaneous speech[END_REF], for the presence of deletions, insertions, and coarticulation effects.

In section 4, different technical approaches to reduce the impact of the speaking rate on the ASR performance are dis-cussed. They basically all rely on a good estimation of the ROS. Practically, since fast speech and slow speech have different effects (for example fast speech increases deletion as well as substitution errors and slow speech increases insertion errors [START_REF] Martinez | Characteristics of slow, average and fast speech and their effects in large vocabulary continuous speech recognition[END_REF][START_REF] Nanjo | Language model and speaking rate adaptation for spontaneous presentation speech recognition[END_REF]), several ROS estimation measures are combined in order to use appropriate compensation techniques.

Children Speech

Children automatic speech recognition is still a difficult problem for conventional Automatic Speech Recognition systems. Children speech represents an important and still poorly understood area in the field of computer speech recognition. The impact of children voices on the performance of standard ASR systems is illustrated in [START_REF] Elenius | Comparing speech recognition for adults and children[END_REF][START_REF] Hagen | Children speech recognition with application to interactive books and tutors[END_REF][START_REF] Traunmüller | Perception of speaker sex, age and vocal effort[END_REF]. The first one is mostly related to physical size. Children have shorter vocal tract and vocal folds compared to adults. This results in higher positions of formants and fundamental frequency. The high fundamental frequency is reflected in a large distance between the harmonics, resulting in poor spectral resolution of voiced sounds. The difference in vocal tract size results in a non-linear increase of the formant frequencies. In order to reduce these effects, previous studies have focused on the acoustic analysis of children speech [START_REF] Lee | Acoustics of children speech: developmental changes of temporal and spectral parameters[END_REF][START_REF] Potamianos | Analysis of children speech: duration, pitch and formants[END_REF]. This work demonstrates the challenges faced by Speech Recognition systems developed to automatically recognize children speech. For example, it has been shown that children below the age of 10 exhibit a wider range of vowel durations relative to older children and adults, larger spectral and suprasegmental variations, and wider variability in formant locations and fundamental frequencies in the speech signal. Several studies have attempted to address this problem by adapting the acoustic features of children speech to match that of acoustic models trained from adult speech [START_REF] Das | Improvements in children speech recognition performance[END_REF][START_REF] Giuliani | Investigating recognition of children speech[END_REF][START_REF] Potamianos | Robust recognition of children speech[END_REF][START_REF] Potamianos | Automatic speech recognition for children[END_REF]. Such Approaches included vocal tract length normalization (VTLN) [START_REF] Das | Improvements in children speech recognition performance[END_REF] as well as spectral normalization [START_REF] Lee | Speaker normalization using efficient frequency warping procedures[END_REF].

A second problem is that younger children may not have a correct pronunciation. Sometimes they have not yet learned how to articulate specific phonemes [START_REF] Schötz | A perceptual study of speaker age[END_REF]. Finally, a third source of difficulty is linked to the way children are using language. The vocabulary is smaller but may also contain words that don't appear in grown-up speech. The correct inflectional forms of certain words may not have been acquired fully, especially for those words that are exceptions to common rules. Spontaneous speech is also believed to be less grammatical than for adults. A number of different solutions to the second and third source of difficulty have been proposed, modification of the pronunciation dictionary, and the use of language models which are customized for children speech have all been tried. In [START_REF] Eskenazi | Pinpointing pronunciation errors in children speech: examining the role of the speech recognizer[END_REF], the number of tied-states of a speech recognizer was reduced to compensate for data sparsity. Recognition experiments using acoustic models trained from adult speech and tested against speech from children of various ages clearly show performance degradation with decreasing age. On average, the word error rates are two to five times worse for children speech than for adult speech. Various techniques for improving ASR performance on children speech are reported.

Although several techniques have been proposed to improve the accuracy of ASR systems on children voices, a large shortfall in performance for children relative to adults remains. [START_REF] Eskenazi | Kids: a database of children's speech[END_REF][START_REF] Wilpon | A study of speech recognition for children and the elderly[END_REF] report ASR performance to be around 100% higher, in average, for children speech than for adults. The difference increases with decreasing age. Many papers report a larger variation in recognition accuracy among children, possibly due to their larger variability in pronunciation. Most of these studies point to lack of children acoustic data and resources to estimate speech recognition parameters relative to the abundance of existing resources for adult speech recognition.

Emotional state

Similarly to the previously discussed speech intrinsic variations, emotional state is found to significantly influence the speech spectrum. It is recognized that a speaker mood change has a considerable impact on the features extracted from his speech, hence directly affecting the basis of all speech recognition systems [START_REF] Cowie | Describing the emotional states that are expressed in speech[END_REF][START_REF] Scherer | Vocal communication of emotion: A review of research paradigms[END_REF].

Studies on speaker emotions is a fairly recent, emerging field and most of today's literature that remotely deals with emotions in speech recognition is concentrated on attempting to classify a "stressed" or "frustrated" speech signal into its correct emotion category [START_REF] Ang | Prosody-based automatic detection of annoyance and frustration in human-computer[END_REF]. The purpose of these efforts is to further improve man-machine communication. Being interested in speech intrinsic variabilities, we will rather focus our attention on the recognition of speech produced in different emotional states. The stressed speech categories studied generally are a collection of all the previously described intrinsic variabilities: loud, soft, Lombard, fast, angry, scared, and noise. Nevertheless, note that emotion recognition might play a role, for instance in a framework where the system could select during operation the most appropriate model in an ensemble of more specific acoustic models (cfr Section 4.2.2).

As Hansen formulates in [START_REF] Hansen | Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition[END_REF], approaches for robust recognition can be summarized under three areas: (i) better training methods, (ii) improved front-end processing, and (iii) improved back-end processing or robust recognition measures. A majority of work undertaken up to now revolves around inspecting the specific differences in the speech signal under the different stress conditions. As an example, the phonetic features have been examined in the case of task stress or emotion [START_REF] Bou-Ghazale | Improving recognition and synthesis of stressed speech via feature perturbation in a source generator framework[END_REF][START_REF] Hansen | Evaluation of acoustic correlates of speech under stress for robust speech recognition[END_REF][START_REF] Hansen | Adaptive source generator compensation and enhancement for speech recognition in noisy stressful environments[END_REF][START_REF] Hansen | A source generator framework for analysis of acoustic correlates of speech under stress. part i: pitch, duration, and intensity effects[END_REF][START_REF] Murray | Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion[END_REF]. The robust ASR approaches are covered by chapter 4.

And more ...

Many more sources of variability affect the speech signal and this paper can probably not cover all of them. Let's cite pathologies affecting the larynx or the lungs, or even the discourse (dysphasia, stuttering, cerebral vascular accident, ...), long-term habits as smoking, singing, ..., speaking styles like whispering, shouting, ... physical activity causing breathlessness, fatigue, ... The impact of those factors on the ASR performance has been little studied and very few papers have been published that specifically address them.

ASR Diagnosis

ASR Performance Analysis and Diagnosis

When devising a novel technique for automatic speech recognition, the goal is to obtain a system whose ASR performance on a specific task will be superior to that of existing methods.

The mainstream aim is to formulate an objective measure for the comparison of a novel system to either similar ASR systems, or humans (cfr. Section 3.2). For this purpose, the general evaluation is the word error rate, measuring the global incorrect word recognition in the total recognition task. As an alternative, the error rate is also measured in smaller units such as phonemes or syllables. Further assessments put forward more detailed errors: insertion, deletion and substitution rates.

Besides, detailed studies are found to identify recognition results considering different linguistic or phonetic properties of the test cases. In such papers, the authors report their systems outcome in the various categories in which they divide the speech samples. The general categories found in the literature are acoustic-phonetic classes, for example: vocal/non-vocal, voiced/unvoiced, nasal/non-nasal [START_REF] Chollet | Evaluating the performance of speech recognizers at the acousticphonetic level[END_REF] [START_REF] Hunt | Speech recognition, syllabification and statistical phonetics[END_REF]. Further groupings separate the test cases according to the physical differences of the speakers, such as male/female, children/adult, or accent [START_REF] Huang | Analysis of speaker variability[END_REF]. Others, finally, study the linguistic variations in detail and devise more complex categories such as 'VCV' (Vowel-Consonant-Vowel) and 'CVC' (Consonant-Vowel-Consonant) and all such different variations [START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF]. Alternatively, other papers report confidence scores to measure the performance of their recognizers [START_REF] Williams | Knowing What You Don't Know: Roles for Confidence Measures in Automatic Speech Recognition[END_REF][START_REF] Zhou | Causal analysis of speech recognition failure in adverse environments[END_REF].

It is however more challenging to find reports on the actual diagnosis of the individual recognizers rather than on the abstract semantics of the recognition sets. In [START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF], the authors perform a diagnostic evaluation of several ASR systems on a common database. They provide error patterns for both phoneme-and word-recognition and then present a decisiontree analysis of the errors providing further insight of the factors that cause the systematic recognition errors. Steeneken et al present their diagnosis method in [START_REF] Steeneken | Objective and diagnostic assessment of (isolated) word recognizers[END_REF] where they establish recognition assessment by manipulating speech, examining the effect of speech input level, noise and frequency shift, on the output of the recognizers. In another approach, Eide et al display recognition errors as a function of word type and length [START_REF] Eide | Understanding and improving speech recognition performance through the use of diagnostic tools[END_REF]. They also provide a method of diagnostic trees to scrutinize the contributions and interactions of error factors in recognition tasks. Alongside, the ANOVA (Analysis of Variance) method [START_REF] Kajarekar | Analysis of sources of variability in speech[END_REF][START_REF] Kajarekar | Analysis of speaker and channel variability in speech[END_REF][START_REF] Sun | Analysis of acoustic-phonetic variations in fluent speech using Timit[END_REF] allows a quantification of the multiple sources of error acting in the overall variability of the speech signals. It offers the possibility to calculate the relative significance of each source of variability as they affect the recognition. On the other hand, Doddington [START_REF] Doddington | Word alignment issues in asr scoring[END_REF] introduces time alignment statistics to reveal systematic ASR scoring errors.

The second, subsequent, difficulty is in discovering research that attempts to actually predict the recognition errors rather than simply giving a detailed analysis of the flaws in the ASR systems. This aspect would give us useful insight by providing generalization to unseen test data. Finally, [START_REF] Fosler-Lussier | A framework for predicting speech recognition errors[END_REF] provides a framework for predicting recognition errors in unseen situations through a collection of lexically confusable words established during training. This work follows former studies on error prediction [START_REF] Deng | Estimating speech recognition error rate without acoustic test data[END_REF][START_REF] Hirschberg | Prosodic and other cues to speech recognition failures[END_REF][START_REF] Printz | Theory and practice of acoustic confusability[END_REF] and assignment of error liability [START_REF] Chase | Error-Responsive Feedback Mechanisms for Speech Recognizers[END_REF] and is adjacent to the research on confusion networks [START_REF] Goel | Segmental minimum Bayes-risk decoding for automatic speech recognition[END_REF][START_REF] Hetherington | New words: Effect on recognition performance and incorporation issues[END_REF][START_REF] Mangu | Finding consensus in speech recognition: Word-error minimization and other applications of confusion networks[END_REF][START_REF]SCLITE scoring software[END_REF][START_REF] Schaaf | Confidence measures for spontaneous speech recognition[END_REF].

Man-machine comparison

A few years ago, a publication [START_REF] Lippmann | Speech recognition by machines and humans[END_REF] gathered results from both human and machine speech recognition, with the goal of stimulating the discussion on research directions and contributing to the understanding of what has still to be done to reach close-tohuman performance. In the reported results, and although problems related to noise can be highlighted, one of the most striking observation concerns the fact that the human listener far outperforms (in relative terms) the machine in tasks characterized by a quiet environment and where no long term grammatical con-straints can be used to help disambiguate the speech. This is the case for instance in digits, letters and nonsense sentences where human listeners can in some cases outperform the machine by more than an order of magnitude. We can thus interpret that the gap between machine performance and human performance (10% vs. 1% word error rate on the WSJ large vocabulary continuous speech task in a variety of acoustic conditions) is by a large amount related to acoustico-phonetic aspects. The deficiencies probably come from a combination of factors. First, the feature representations used for ASR may not contain all the useful information for recognition. Then, the modeling assumptions may not be appropriate. Third, the applied features extraction and the modeling approaches may be too sensitive to intrinsic speech variabilities, amongst which are: speaker, gender, age, dialect, accent, health condition, speaking rate, prosody, emotional state, spontaneity, speaking effort, articulation effort.

In [START_REF] Sroka | Human and machine consonant recognition[END_REF], consonant recognition within different degradation conditions (high-pass and low-pass filtering, as well as background noise) is compared between human and automatic systems. Results are presented globally in terms of recognition accuracy, and also in more details in terms of confusion matrices as well as information transfer of different phonetic features (voicing, place, frication, sibilance). Although the test material is not degraded in the exact same fashion for the comparison tests, results clearly indicate different patterns of accuracy for human and machines, with weaker machine performance on recognizing some phonological features, such as voicing, especially under noise conditions. This happens despite the fact that the ASR system training provides acoustic models that are almost perfectly matched to the test conditions, using the same speakers, same material (CVCs) and same conditions (noise added to the training set to match the test condition).

In [START_REF] Wesker | Oldenburg logatome speech corpus (OLLO) for speech recognition experiments with humans and machines[END_REF] (experiments under way), this line of research is extended with the first controlled comparison of human and machine on speech after removing high-level knowledge (lexical, syntactic..) sources, complementing the analysis of phoneme identification scores with the impact of intrinsic variabilities (rather than high-pass/low-pass filters and noise in the previous literature..) Another goal of the research is to extend the scope of previous research (which was for instance mostly related to English) and address some procedures that can sometimes be questioned in previous research (for instance the difference of protocols used for human and machine tests).

Besides simple comparisons in the form of human intelligibility versus ASR accuracy, specific experimental designs can also provide some relevant insights in order to pinpoint possible weaknesses (with respect to humans) at different stages of processing of the current ASR recognition chain. This is summarized in the next subsection.

Specific methodologies

Some references are given here, revolving around the issue of feature extraction limitations (in this case the presence or absence of phase information) vs. modeling limitations.

It has been suggested [START_REF] Demuynck | Synthesizing speech from speech recognition parameters[END_REF][START_REF] Leonard | A database for speaker independent digit recognition[END_REF][START_REF] Peters | On the limits of speech recognition in noise[END_REF] that conventional cepstral representation of speech may destroy important information by ignoring the phase (power spectrum estimation) and reducing the spectral resolution (Mel filter bank, LPC, cepstral liftering, ...).

Phase elimination is justified by some evidence that humans are relatively insensitive to the phase, at least in steadystate contexts, while resolution reduction is mostly motivated by practical modeling limitations. However, natural speech is far from being constituted of steady-state segments. In [START_REF] Liu | Effects of phase on the perception of intervocalic stop consonants[END_REF], the authors clearly demonstrate the importance of the phase information for correctly classifying stop consonants, especially regarding their voicing property. Moreover, in [START_REF] Schroeder | Flat-spectrum speech[END_REF], it is demonstrated that vowel-like sounds can be artificially created from flat spectrum signal by adequately tuning the phase angles of the waveform.

In order to investigate a possible loss of crucial information, reports of different experiments have been surveyed in the literature. In these experiments, humans were asked to recognize speech reconstructed from the conventional ASR acoustic features, hence with no phase information and no fine spectral representation.

Experiments conducted by Leonard and reported by [START_REF] Lippmann | Speech recognition by machines and humans[END_REF], seems to show that ASR acoustic analysis (LPC in that case) has little effect on human recognition, suggesting that most of the ASR weaknesses may come from the acoustic modeling limitations and little from the acoustic analysis (i.e. front-end or feature extraction portion of the ASR system) weaknesses. Those experiments have been carried out on sequences of digits recorded in a quiet environment.

In their study, Demuynck et al. re-synthesized speech from different steps of the MFCC analysis, i.e. power spectrum, Mel spectrum and Mel cepstrum [START_REF] Demuynck | Synthesizing speech from speech recognition parameters[END_REF]. They come to the conclusion that re-synthesized speech is perfectly intelligible given that an excitation signal based on pitch analysis is used, and that the phase information is not required. They emphasize that their experiments are done on clean speech only.

Experiments conducted by Peters et al [START_REF] Peters | On the limits of speech recognition in noise[END_REF] demonstrate that these conclusions are not correct in case of noisy speech recordings. He suggests that information lost by the conventional acoustic analysis (phase and fine spectral resolution) may become crucial for intelligibility in case of speech distortions (reverberation, environment noise, ...). These results show that, in noisy environment, the degradation of the speech representation affects the performance of the human recognition almost in the same order as the machine. More particularly, ignoring the phase leads to a severe drop of human performance (from almost perfect recognition to 8.5% sentence error rate) suggesting that the insensitivity of human to the phase is not that true in adverse conditions.

In [START_REF] Paliwal | Usefulness of phase spectrum in human speech perception[END_REF], the authors perform human perception experiments on speech signals reconstructed either from the magnitude spectrum or from the phase spectrum and conclude that phase spectrum contribute as much as amplitude to speech intelligibility if the shape of the analysis window is properly selected.

Finally, experiments achieved at Oldenburg demonstrated that the smearing of the temporal resolution of conventional acoustic features affects human intelligibility for modulation cut-off frequencies lower than 32 Hz on a phoneme recognition task. Also, they conclude that neglecting the phase causes approximately 5% error rate in phoneme recognition of human listeners.

ASR techniques

In this section, we review methodologies towards improved ASR analysis/modeling accuracy and robustness against the intrinsic variability of speech. Similar techniques have been proposed to address different sources of speech variation. This section will introduce both the general ideas of these approaches and the specific usage regarding variability sources.

Front-end techniques

An update on feature extraction front-ends is proposed, particularly showing how to take advantage of techniques targeting the non-stationarity assumption. Also, the feature extraction stage can be the appropriate level to target the effects of some other variations, like the speaker physiology (through feature compensation [START_REF] Welling | Speaker adaptive modeling by vocal tract normalization[END_REF] or else improved invariance [START_REF] Mertins | Vocal tract length invariant features for automatic speech recognition[END_REF]) and other dimensions of speech variability. Finally, techniques for combining estimation based on different features sets are reviewed. This also involves dimensionality reduction approaches.

Overcoming assumptions

Most of the Automatic Speech Recognition (ASR) acoustic features, such as Mel-Frequency Cepstral Coefficients (MFCC) [START_REF] Davis | Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences[END_REF] or Perceptual Linear Prediction (PLP) coefficients [START_REF] Hermansky | Perceptual linear predictive (PLP) analysis of speech[END_REF], are based on some sort of representation of the smoothed spectral envelope, usually estimated over fixed analysis windows of typically 20 ms to 30 ms [START_REF] Davis | Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences[END_REF][START_REF] Rabiner | Fundamentals of speech recognition[END_REF] 2 . Such analysis is based on the assumption that the speech signal is quasi-stationary over these segment durations. However, it is well known that the voiced speech sounds such as vowels are quasi-stationary for 40 ms-80 ms, while stops and plosive are time-limited by less than 20 ms [START_REF] Rabiner | Fundamentals of speech recognition[END_REF]. Therefore, it implies that the spectral analysis based on a fixed size window of 20 ms-30 ms has some limitations, including:

• The frequency resolution obtained for quasi-stationary segments (QSS) longer than 20 ms is quite low compared to what could be obtained using larger analysis windows.

• In certain cases, the analysis window can span the transition between two QSSs, thus blurring the spectral properties of the QSSs, as well as of the transitions. Indeed, in theory, Power Spectral Density (PSD) cannot even be defined for such non stationary segments [START_REF] Haykin | Adaptive filter theory[END_REF]. Furthermore, on a more practical note, the feature vectors extracted from such transition segments do not belong to a single unique (stationary) class and may lead to poor discrimination in a pattern recognition problem.

In [START_REF] Tyagi | On variablescale piecewise stationary spectral analysis of speech signals for ASR[END_REF], the usual assumption is made that the piecewise quasi-stationary segments (QSS) of the speech signal can be modeled by a Gaussian autoregressive (AR) process of a fixed order p as in [START_REF] Andre-Obrecht | A new statistical approach for the automatic segmentation of continuous speech signals[END_REF][START_REF] Svendsen | An improved sub-word based speech recognizer[END_REF][START_REF] Svendsen | On the automatic segmentation of speech signals[END_REF]. The problem of detecting QSSs is then formulated using a Maximum Likelihood (ML) criterion, defining a QSS as the longest segment that has most probably been generated by the same AR process. 3Another approach is proposed in [START_REF]Efficient coding of LPC parameters by temporal decomposition[END_REF], which describes a temporal decomposition technique to represent the continuous variation of the LPC parameters as a linearly weighted sum of a number of discrete elementary components. These elementary components are designed such that they have the minimum temporal spread (highly localized in time) resulting in superior coding efficiency. However, the relationship between the optimization criterion of "the minimum temporal spread" and the quasi-stationarity is not obvious. Therefore, the discrete ele-mentary components are not necessarily quasi-stationary and vice-versa.

Coifman et al [START_REF] Coifman | Entropy based algorithms for best basis selection[END_REF] have described a minimum entropy basis selection algorithm to achieve the minimum information cost of a signal relative to the designed orthonormal basis. In [START_REF] Svendsen | On the automatic segmentation of speech signals[END_REF], Svendsen et al have proposed a ML segmentation algorithm using a single fixed window size for speech analysis, followed by a clustering of the frames which were spectrally similar for sub-word unit design. More recently, Achan et al [START_REF] Achan | A segmental HMM for speech waveforms[END_REF] have proposed a segmental HMM for speech waveforms which identifies waveform samples at the boundaries between glottal pulse periods with applications in pitch estimation and time-scale modifications.

As a complementary principle to developing features that "work around" the non-stationarity of speech, significant efforts have also been made to develop new speech signal representations which can better describe the non-stationarity inherent in the speech signal. Some representative examples are temporal patterns (TRAPs) features [START_REF] Hermansky | TRAPS: classifiers of temporal patterns[END_REF], MLPs and several modulation spectrum related techniques [START_REF] Kingsbury | Robust speech recognition using the modulation spectrogram[END_REF][START_REF] Milner | Inclusion of temporal information into features for speech recognition[END_REF][START_REF] Tyagi | Melcepstrum modulation spectrum (MCMS) features for robust ASR[END_REF][START_REF] Zhu | AM-demodualtion of speech spectra and its application to noise robust speech recognition[END_REF]. In this approach temporal trajectories of spectral energies in individual critical bands over windows as long as one second are used as features for pattern classification. Another methodology is to use the notion of the amplitude modulation (AM) and the frequency modulation (FM) [START_REF] Haykin | Communication systems[END_REF]. In theory, the AM signal modulates a narrow-band carrier signal (specifically, a monochromatic sinusoidal signal). Therefore to be able to extract the AM signals of a wide-band signal such as speech (typically 4KHz), it is necessary to decompose the speech signal into narrow spectral bands. In [START_REF] Tyagi | Fepstrum representation of speech[END_REF], this approach is opposed to the previous use of the speech modulation spectrum [START_REF] Kingsbury | Robust speech recognition using the modulation spectrogram[END_REF][START_REF] Milner | Inclusion of temporal information into features for speech recognition[END_REF][START_REF] Tyagi | Melcepstrum modulation spectrum (MCMS) features for robust ASR[END_REF][START_REF] Zhu | AM-demodualtion of speech spectra and its application to noise robust speech recognition[END_REF] which was derived by decomposing the speech signal into increasingly wider spectral bands (such as critical, Bark or Mel). Similar arguments from the modulation filtering point of view, were presented by Schimmel and Atlas [START_REF] Schimmel | Coherent envelope detection for modulation filtering of speech[END_REF]. In their experiment, they consider a wide-band filtered speech signal x(t) = a(t)c(t), where a(t) is the AM signal and c(t) is the broad-band carrier signal. Then, they perform a low-pass modulation filtering of the AM signal a(t) to obtain aLP (t). The low-pass filtered AM signal aLP (t) is then multiplied with the original carrier c(t) to obtain a new signal x(t). They show that the acoustic bandwidth of x(t) is not necessarily less than that of the original signal x(t). This unexpected result is a consequence of the signal decomposition into wide spectral bands that results in a broad-band carrier.

Finally, as extension to the "traditional" AR process (allpole model) speech modeling, pole-zero transfer functions that are used for modeling the frequency response of a signal, have been well studied and understood [START_REF] Makhoul | Linear prediction: a tutorial review[END_REF]. Lately, Kumaresan et al. [START_REF] Kumaresan | An inverse signal approach to computing the envelope of a real valued signal[END_REF][START_REF] Kumaresan | Model-based approach to envelope and positive instantaneous frequency estimation of signals with speech appications[END_REF] have proposed to model analytic signals using pole-zero models in the temporal domain. Along similar lines, Athineos et al. [START_REF] Athineos | Frequency domain linear prediction for temporal features[END_REF] have used the dual of the linear prediction in the frequency domain to improve upon the TRAP features.

Another strong assumption that has been addressed in recent papers, concern the worthlessness of the phase for speech intelligibility. We already introduced in section 3.2.1 the conclusions of several studies that reject this assumption. A few papers have tried to reintroduce the phase information into the ASR systems. In [START_REF] Paliwal | Frequency-related representation of speech[END_REF], the authors introduce the instantaneous frequency which is computed from the phase spectrum. Experiments on vowel classification show that these features contain meaningful information. Other authors are proposing features derived from the group delay [START_REF] Bozkurt | On the use of phase information for speech recognition[END_REF][START_REF] Hegde | Continuous speech recognition using joint features derived from the modified group delay function and mfcc[END_REF][START_REF] Zhu | Product of power spectrum and group delay function for speech recognition[END_REF] which presents a formant-like structure with a much higher resolution than the power spectrum. As the group delay in inherently very noisy, the approaches proposed by the authors mainly aims at smoothing the estimation. ASR experiments show interesting performance in noisy conditions.

Compensation and invariance

For other sources of speech variability (besides nonstationarity), a simple model may exist that appropriately reflects and compensate its effect on the speech features.

The preponderance of lower frequencies for carrying the linguistic information has been assessed by both perceptual and acoustical analysis and justify the success of the non-linear frequency scales such as Mel, Bark, Erb, ... Similarly, in [START_REF] Hermansky | Perceptual linear predictive (PLP) analysis of speech[END_REF], the PLP parameters present a fair robustness to inter-speaker variability, thanks to the low order (5th) linear prediction analysis which only models the two main peaks of the spectral shape, typically the first two formants. Other approaches aim at building acoustic features invariant to the frequency warping.

In [START_REF] Umesh | Scale transform in speech analysis[END_REF], the authors define the "scale transform" and the "scale cepstrum"of a signal spectrum whose magnitude is invariant to a scaled version of the original spectrum. In [START_REF] Mertins | Vocal tract length invariant features for automatic speech recognition[END_REF], the continuous wavelet transform has been used as a preprocessing step, in order to obtain a speech representation in which linear frequency scaling leads to a translation in the time-scale plane. In a second step, frequency-warping invariant features were generated. These include the auto-and cross-correlation of magnitudes of local wavelet spectra as well as linear and nonlinear transforms thereof. It could be shown that these features not only lead to better recognition scores than standard MFCCs, but that they are also more robust to mismatches between training and test conditions, such as training on male and testing on female data. The best results were obtained when MFCCs and the vocal tract length invariant features were combined, showing that the sets contain complementary information [START_REF] Mertins | Vocal tract length invariant features for automatic speech recognition[END_REF].

A direct application of the tube resonator model of the vocal tract lead to the different vocal tract length normalization (VTLN) techniques: speaker-dependent formant mapping [START_REF] Di Benedetto | Extrinsic normalization of vowel formant values based on cardinal vowels mapping[END_REF][START_REF] Wakita | Normalization of vowels by vocal-tract length and its application to vowel identification[END_REF], transformation of the LPC pole modeling [START_REF] Slifka | Speaker modification with LPC pole analysis[END_REF], frequency warping, either linear [START_REF] Eide | A parametric approach to vocal tract length normalization[END_REF][START_REF] Lee | Speaker normalization using efficient frequency warping procedures[END_REF][START_REF] Tuerk | A new frequency shift function for reducing inter-speaker variance[END_REF][START_REF] Zhan | Speaker normalization based on frequency warping[END_REF] or non-linear [START_REF] Ono | Speaker normalization using constrained spectra shifts in auditory filter domain[END_REF], all consist of modifying the position of the formants in order to get closer to an "average" canonical speaker. Simple yet powerful techniques for normalizing (compensating) the features to the VTL are widely used [START_REF] Welling | Speaker adaptive modeling by vocal tract normalization[END_REF]. Note that VTLN is often combined with an adaptation of the acoustic model to the canonical speaker [START_REF] Eide | A parametric approach to vocal tract length normalization[END_REF][START_REF] Lee | Speaker normalization using efficient frequency warping procedures[END_REF] (cf. section 4.2.1). The potential of using piece-wise linear and phoneme-dependent frequency warping algorithms for reducing the variability in the acoustic feature space of children have also been investigated [START_REF] Das | Improvements in children speech recognition performance[END_REF].

Channel compensation techniques such as the cepstral mean subtraction or the RASTA filtering of spectral trajectories, also compensate for the speaker-dependent component of the long-term spectrum [START_REF] Kajarekar | Analysis of speaker and channel variability in speech[END_REF][START_REF] Westphal | The use of cepstral means in conversational speech recognition[END_REF].

Similarly, some studies attempted to devise feature extraction methods tailored for the recognition of stressed and nonstressed speech simultaneously. In his paper [START_REF] Chen | Cepstral domain stress compensation for robust speech recognition[END_REF], Chen proposed a Cepstral Domain Compensation when he showed that simple transformations (shifts and tilts) of the cepstral coefficients occur between the different types of speech signals studied. Further processing techniques have been employed for more robust speech features [START_REF] Hansen | Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition[END_REF][START_REF] Hermansky | RASTA processing of speech[END_REF][START_REF] Hunt | A comparison of several acoustic representations for speech recognition with degraded and undegraded speech[END_REF] and some researchers simply assessed the better representations from the existing pool of features [START_REF] Hanson | Robust speakerindependent word recognition using instantaneous, dynamic and acceleration features: experiments with Lombard and noisy speech[END_REF].

When simple parametric models of the effect of the variability are not appropriate, feature compensation can be performed using more generic non-parametric transformation schemes, including linear and non-linear transformations. This becomes a dual approach to model adaptation, which is the topic of Section 4.2.1.

Additional cues and multiple feature streams

As a complementary perspective to improving or compensating single feature sets, one can also make use of several "streams" of features that rely on different underlying assumptions and exhibit different properties.

Intrinsic feature variability depends on the set of classes that features have to discriminate. Given a set of acoustic measurements, algorithms have been described to select subsets of them that improve automatic classification of speech data into phonemes or phonetic features. Unfortunately, pertinent algorithms are computationally intractable with these types of classes as stated in [START_REF] Omar | Maximum mutual information based acoustic features representation of phonological features for speech recognition[END_REF], [START_REF] Omar | An evaluation of using mutual information for selection of acoustic features representation of phonemes for speech recognition[END_REF], where a sub-optimal solution is proposed. It consists in selecting a set of acoustic measurement that guarantees a high value of the mutual information between acoustic measurements and phonetic distinctive features.

Without attempting to find an optimal set of acoustic measurements, many recent automatic speech recognition systems combine streams of different acoustic measurements on the assumption that some characteristics that are de-emphasized by a particular feature are emphasized by another feature, and therefore the combined feature streams capture complementary information present in individual features.

In order to take into account different temporal behavior in different bands, it has been proposed ( [START_REF] Bourlard | Sub-band based speech recognition[END_REF][START_REF] Tibrewala | Sub-band based recognition of noisy speech[END_REF][START_REF] Tomlinson | Modelling asynchrony in speech using elementary single-signal decomposition[END_REF]) to consider separate streams of features extracted in separate channels with different frequency bands. Inspired by the multi-stream approach, examples of acoustic measurement combination are:

• Multi-resolution spectral/time correlates ( [START_REF] Vaseghi | Multi resolution phonetic/segmental features and models for HMM-based speech recognition[END_REF], [START_REF] Hariharan | Noise robust speech parameterization using multiresolution feature extraction[END_REF]),

• segment and frame-based acoustic features ( [START_REF] Hon | Combining frame and segment based models for large vocabulary continuous speech recognition[END_REF]),

• MFCC, PLP and an auditory feature ( [START_REF] Jiang | Acoustic feature selection using speech recognizers[END_REF]),

• spectral-based and discriminant features ( [START_REF] Benitez | Robust asr front-end using spectral based and discriminant features: experiments on the aurora task[END_REF]),

• acoustic and articulatory features ( [START_REF] Kirchhoff | Combining articulatory and acoustic information for speech recognition in noise and reverberant environments[END_REF][START_REF] Tolba | Auditory-based acoustic distinctive features and spectral cues for automatic speech recognition using a multistream paradigm[END_REF]),

• LPC based cepstra, MFCC coefficients, PLP coefficients, energies and time-averages ( [START_REF] Omar | Maximum mutual information based acoustic features representation of phonological features for speech recognition[END_REF], [START_REF] Omar | An evaluation of using mutual information for selection of acoustic features representation of phonemes for speech recognition[END_REF]), MFCC and PLP ( [START_REF] Zolnay | Acoustic feature combination for robust speech recognition[END_REF]),

• full band non-compressed root cepstral coefficients (RCC), Full band PLP 16kHz,Telephone band PLP 8 kHz ( [START_REF] Kingsbury | Robust speech recognition in noisy environments: the 2001 IBM SPINE evaluation system[END_REF]),

• PLP, MFCC and wavelet features ( [START_REF] Gemello | Multiple resolution analysis for robust automatic speech recognition[END_REF]),

• joint features derived from the modified group-delay function ( [START_REF] Hegde | Speech processing using joint features derived from the modified group delay function[END_REF]),

• combinations of frequency filtering (FF), MFCC, RASTA-FF, (J)RASTA-PLP ( [START_REF] Pujol | Comparison and combination of features in a hybrid HMM/MLP and a HMM/GMM speech recognition system[END_REF]).

Other approaches integrate some specific parameters into a single stream of features. Examples of added parameters are:

• periodicity and jitter ( [START_REF] Thomson | Use of periodicity and jitter as speech recognition feature[END_REF]),

• voicing ( [START_REF] Zolnay | Robust speech recognition using a voiced-unvoiced feature[END_REF], [START_REF] Graciarena | Voicing feature integration in SRI's DECI-PHER LVCSR system[END_REF]),

• rate of speech and pitch ( [START_REF] Stephenson | Speech recognition with auxiliary information[END_REF]). To benefit from the strengths of both MLP-HMM and Gaussian-HMM techniques, the Tandem solution was proposed in [START_REF] Ellis | Tandem acoustic modeling in large-vocabulary recognition[END_REF], using posterior probability estimation obtained at MLP outputs as observations for a Gaussian-HMM. An error analysis of Tandem MLP features showed that the errors using MLP features are different from the errors using cepstral features. This motivates the combination of both feature styles. In ( [START_REF] Zhu | On using MLP features in LVCSR[END_REF]), combination techniques were applied to increasingly more advanced systems showing the benefits of the MLP-based features. These features have been combined with TRAP features ( [START_REF] Morgan | TRAPping conversational speech: extending TRAP/tandem approaches to conversational telephone speech recognition[END_REF]). In ( [START_REF] Kleinschmidt | Improving word accuracy with gabor feature extraction[END_REF]), Gabor filters are proposed, in conjunction with MLP features, to model the characteristics of neurons in the auditory system as is done for the visual system. There is evidence that in primary auditory cortex each individual neuron is tuned to a specific combination of spectral and temporal modulation frequencies.

In [START_REF] Eide | Distinctive features for use in automatic speech recognition[END_REF], it is proposed to use mixture gaussians to represent presence and absence of features.

Additional features have also been considered as cues for speech recognition failures [START_REF] Hirschberg | Prosodic and other cues to speech recognition failures[END_REF].

This section introduced several works where several streams of acoustic representations of the speech signal were successfully combined in order to improve the ASR performance. Different combination methods have been proposed and can roughly be classified as:

• direct feature combination/transformation such as PCA, LDA, HDA, ... or selection of the best features will be discussed in section 4.1.4

• combination of acoustic models trained on different feature sets will be discussed in section 4.2.2

• combination of recognition system based on different acoustic features will be discussed in section ??

Dimensionality reduction and feature selection

Using additional features/cues as reviewed in the previous section, or simply extending the context by concatenating feature vectors from adjacent frames may yield very long feature vectors in which several features contain redundant information, thus requiring an additional dimension-reduction stage [START_REF] Haeb-Umbach | Linear discriminant analysis for improved large vocabulary continuous speech recognition[END_REF][START_REF] Kumar | Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition[END_REF] and/or improved training procedures.

The most common feature-reduction technique is the use of a linear transform y = Ax where x and y are the original and the reduced feature vectors, respectively, and A is a p × n matrix with p < n where n and p are the original and the desired number of features, respectively. The principal component analysis (PCA) [START_REF] Duda | Pattern classification and scene analysis[END_REF][START_REF] Fukunaga | Introduction to statistical pattern recognition[END_REF] is the most simple way of finding A. It allows for the best reconstruction of x from y in the sense of a minimal average squared Euclidean distance. However, it does not take the final classification task into account and is therefore only suboptimal for finding reduced feature sets. A more classification-related approach is the linear discriminant analysis (LDA), which is based on Fisher's ratio (F-ratio) of between-class and within-class covariances [START_REF] Duda | Pattern classification and scene analysis[END_REF][START_REF] Fukunaga | Introduction to statistical pattern recognition[END_REF]. Here the columns of matrix A are the eigenvectors belonging to the p largest eigenvalues of matrix [S -1 w S b ], where Sw and S b are the within-class and between-class scatter matrices, respectively. Good results with LDA have been reported for small vocabulary speech recognition tasks, but for large-vocabulary speech recognition, results were mixed [START_REF] Haeb-Umbach | Linear discriminant analysis for improved large vocabulary continuous speech recognition[END_REF]. In [START_REF] Haeb-Umbach | Linear discriminant analysis for improved large vocabulary continuous speech recognition[END_REF] it was found that the LDA should best be trained on sub-phone units in order to serve as a preprocessor for a continuous mixture density based recognizer. A limitation of LDA is that it cannot effectively take into account the presence of different within-class covariance matrices for different classes. Heteroscedastic discriminant analysis (HDA) [START_REF] Kumar | Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition[END_REF] overcomes this problem, and is actually a generalization of LDA. The method usually requires the use of numerical optimization techniques to find the matrix A. An exception is the method in [START_REF] Loog | Linear dimensionality reduction via a heteroscedastic extension of LDA: The Chernoff criterion[END_REF], which uses the Chernoff distance to measure between-class distances and leads to a straight forward solution for A. Finally, LDA and HDA can be combined with maximum likelihood linear transform (MLLT) [START_REF] Gopinath | Maximum likelihood modeling with gaussian distributions for classification[END_REF], which is identical to semi-tied covariance matrices (STC) [START_REF] Gales | Semi-tied covariance matrices for hidden markov models[END_REF]. Both aim at transforming the reduced features in such a way that they better fit with the diagonal covariance matrices that are applied in many HMM recognizers (cfr. [START_REF] Pitz | Investigations on Linear Transformations for Speaker Adaptation and Normalization[END_REF], section 2.1). It has been reported [START_REF] Saon | Maximum likelihood discriminant feature spaces[END_REF] that such a combination performs better than LDA or HDA alone. Also, HDA has been combined with minimum phoneme error (MPE) analysis [START_REF] Zhang | Minimum phoneme error based heteroscedastic linear discriminant analysis for speech recognition[END_REF]. Recently, the problem of finding optimal dimension-reducing feature transformations has been studied from the viewpoint of maximizing the mutual information between the obtained feature set and the corresponding phonetic class [START_REF] Omar | Maximum mutual information based acoustic features representation of phonological features for speech recognition[END_REF][START_REF] Padmanabhan | Maximizing information content in feature extraction[END_REF].

A problem of the use of linear transforms for feature reduction is that the entire feature vector x needs to be computed before the reduced vector y can be generated. This may lead to a large computational cost for feature generation, although the final number of features may be relatively low. An alternative is the direct selection of feature subsets, which, expressed by matrix A, means that each row of A contains a single one while all other elements are zero. The question is then the one of which features to include and which to exclude. Because the elements of A have to be binary, simple algebraic solutions like with PCA or LDA cannot be found, and iterative strategies have been proposed. For example, in [2], the maximum entropy principle was used to decide on the best feature space.

Acoustic modeling techniques

Concerning acoustic modeling, good performance is generally achieved when the model is matched to the task, which can be obtained through adequate training data (see also Section 4.4). Systems with stronger generalization capabilities can then be built through a so-called multi-style training. Estimating the parameters of a traditional modeling architecture in this way however has some limitation due to the inhomogeneity of the data, which increases the spread of the models, and hence negatively impacts accuracy compared to task-specific models. This is partly to be related to the inability of the framework to properly model long-term correlations of the speech signals.

Also, within the acoustic modeling framework, adaptation techniques provide a general formalism for reestimating optimal model parameters for given circumstances based on moderate amounts of speech data.

Then, the modeling framework can be extended to allow multiple specific models to cover the space of variation. These can be obtained through generalizations of the HMM modeling framework, or through explicit construction of multiple models built on knowledge-based or data-driven clusters of data.

In the following, extensions for modeling using additional cues and features is also reviewed.

Adaptation

In Section 4.1.2, we have been reviewing techniques that can be used to compensate for speech variation at the feature extraction level. A dual approach is to adapt the ASR acoustic models.

In some cases, some variations in the speech signal could be considered as long term given the application. For instance, a system embedded in a personal device and hence mainly designed to be used by a single person, or a system designed to transcribe and index spontaneous speech, or characterized by utilization in a particular environment. In these cases, it is often possible to adapt the models to these particular conditions, hence partially factoring out the detrimental effect of these. A popular technique is to estimate a linear transformation of the model parameters using a Maximum Likelihood (ML) criterion [START_REF] Leggetter | Maximum likelihood linear regression for speaker adaptation of continuous density Hidden Markov Models[END_REF]. A Maximum a Posteriori (MAP) objective function may also be used [START_REF] Chesta | Maximum a posteriori linear regression for Hidden Markov Model adaptation[END_REF][START_REF] Zavaliagkos | Maximum a posteriori adaptation for large scale HMM recognizers[END_REF].

Being able to perform this adaptation using limited amounts of condition-specific data would be a very desirable property for such adaptation methodologies, as this would reduce the cost and hassle of such adaptation phases. Such "fast" (sometimes on-line) adaptation schemes have been proposed a few years ago, based on the clustering of the speakers into sets of speakers which have similar voice characteristics. Inferred acoustic models present a much smaller variance than speaker-independent systems [START_REF] Masaki Naito | Speaker clustering for speech recognition using the parameters characterizing vocal-tract dimensions[END_REF][START_REF] Padmanabhan | Speaker clustering and transformation for speaker adaptation in large-vocabulary speech recognition systems[END_REF]. The eigenvoice approach [START_REF] Gales | Cluster adaptive training for speech recognition[END_REF][START_REF] Nguyen | Eigenvoices : a compact representation of speakers in a model space[END_REF] takes from this idea by building a low dimension eigenspace in which any speaker is located and modeled as a linear combination of "eigenvoices".

Intuitively, these techniques rest on the principle of acquiring knowledge from the training corpora that represent the prior distribution (or clusters) of model parameters given a variability factor under study. With these adaptation techniques, knowledge about the effect of the inter-speaker variabilities are gathered in the model. In the traditional approach, this knowledge is simply discarded, and, although all the speakers are used to build the model, and pdfs are modeled using mixtures of gaussians, the ties between particular mixture components across the several CD phonemes are not represented/used.

Recent publications have been extending and refining this class of techniques. In [START_REF] Kim | Rapid online adaptation using speaker space model evolution[END_REF], rapid adaptation is further extended through a more accurate speaker space model, and an on-line algorithm is also proposed. In [START_REF] Wu | Speaker adaptation using constrained transformation[END_REF], the correlations between the means of mixture components of the different features are modeled using a Markov Random Field, which is then used to constrain the transformation matrix used for adaptation. Other publications include [START_REF] Kenny | Eigenvoice modeling with sparse training data[END_REF][START_REF] Mak | Improving eigenspace-based mllr adaptation by kernel PCA[END_REF][START_REF] Tsakalidis | Discriminative linear transforms for feature normalization and speaker adaptation in HMM estimation[END_REF][START_REF] Tsao | Segmental eigenvoice with delicate eigenspace for improved speaker adaptation[END_REF][START_REF] Wu | Speaker adaptation using constrained transformation[END_REF][START_REF] Zhou | Rapid discriminative acoustic model based on eigenspace mapping for fast speaker adaptation[END_REF].

Other forms of transformations for adaptation are also proposed in [START_REF] Padmanabhan | Maximumlikelihood nonlinear transformation for acoustic adaptation[END_REF], where the Maximum Likelihood criterion is used but the transformations are allowed to be nonlinear. Let us also mention alternate non-linear speaker adaptation paradigms based on connectionist networks [START_REF] Abrash | Acoustic adaptation using nonlinear transformations of HMM parameters[END_REF][START_REF] Watrous | Speaker normalization and adaptation using second-order connectionist networks[END_REF].

Speaker normalization algorithms that combine frequency warping and model transformation have been proposed to reduce acoustic variability and significantly improve ASR performance for children speakers (by 25-45% under various model training and testing conditions) [START_REF] Potamianos | Robust recognition of children speech[END_REF][START_REF] Potamianos | Automatic speech recognition for children[END_REF]. ASR on emotional speech has also benefited from techniques relying on adapting the model structure within the recognition system to account for the variability in the input signal. One practice has been to bring the training and test conditions closer by space projection [START_REF] Carlson | Speech recognition in noise using a projection-based likelihood measure for mixture density HMMs[END_REF][START_REF] Mansour | A family of distortion measures based upon projection operation for robust speech recognition[END_REF]. In [START_REF] Kubala | Comparative experiments on large vocabulary speech recognition[END_REF], it is shown that acoustic model adaptation can be used to reduce the degradation due to non-native dialects. This has been observed on an English read speech recognition task (Wall Street Journal), and the adaptation was applied at the speaker level to obtain speaker dependent models. For speaker independent systems this may not be feasible however, as this would require adaptation data with a large coverage of non-native speech.

Multiple modeling

Instead of adapting the models to particular conditions, one may also train an ensemble of models specialized to specific conditions or variations. These models may then be used within a selection, competition or else combination framework. Such techniques are the object of this section.

Acoustic models are estimated from speech corpora, and they provide their best recognition performances when the operating (or testing) conditions are consistent with the training conditions. Hence many adaptation procedures were studied to adapt generic models to specific tasks and conditions. When the speech recognition system has to handle various possible conditions, several speech corpora can be used together for estimating the acoustic models, leading to mixed models or hybrid systems [START_REF] Das | Towards robust speech recognition in the telephony network environment -cellular and landline conditions[END_REF][START_REF] Mokbel | Towards improving ASR robustness for PSN and GSM telephone applications[END_REF], which provide good performances in those various conditions (for example in both landline and wireless networks). However, merging too many heterogeneous data in the training corpus makes acoustic models less discriminant. Hence the numerous investigations along multiple modeling, that is the usage of several models for each unit, each model being trained from a subset of the training data, defined according to a priori criteria such as gender, accent, age, rate-of-speech (ROS) or through automatic clustering procedures. Ideally subsets should contain homogeneous data, and be large enough for making possible a reliable training of the acoustic models.

Gender information is one of the most often used criteria. It leads to gender-dependent models that are either directly used in the recognition process itself [START_REF] Odell | Large vocabulary continuous speech recognition using HTK[END_REF][START_REF] Konig | GDNN: a gender-dependent neural network for continuous speech recognition[END_REF] or used as a better seed for speaker adaptation [START_REF] Lee | Speaker adaptation based on MAP estimation of HMM parameters[END_REF]. Gender dependence is applied to whole word units, for example digits [START_REF] Gupta | Highaccuracy connected digit recognition for mobile applications[END_REF], or to context dependent phonetic units [START_REF] Odell | Large vocabulary continuous speech recognition using HTK[END_REF], as a result of an adequate splitting of the training data.

In many cases, most of the regional variants of a language are handled in a blind way through a global training of the speech recognition system using speech data that covers all of these regional variants, and enriched modeling is generally used to handle such variants. This can be achieved through the use of multiple acoustic models associated to large groups of speakers as in [START_REF] Beattie | An integrated multidialect speech recognition system with optional speaker adaptation[END_REF][START_REF] Vancompernolle | Speaker clustering for dialectic robustness in speaker independent speech recognition[END_REF]. These papers showed that it was preferable to have models only for a small number of large speaker populations than for many small groups. When a single foreign accent is handled, some accented data can be used for training or adapting the acoustic models [START_REF] Aalburg | Foreign-accented speakerindependent speech recognition[END_REF][START_REF] He | Fast model selection based speaker adaptation for nonnative speech[END_REF][START_REF] Liu | MLLR-based accent model adaptation without accented data[END_REF][START_REF] Uebler | Recognition of non-native german speech with multilingual recognizers[END_REF].

Age dependent modeling has been less investigated, may be due to the lack of large size children speech corpora. The results presented in [START_REF] D'arcy | Recognition of read and spontaneous children's speech using two new corpora[END_REF] fail to demonstrate a significant improvement when using age dependent acoustic models, possibly due to the limited amount of training data for each class of age. Simply training a conventional speech recognizer on children speech is not sufficient to yield high accuracies, as demonstrated by Wilpon and Jacobsen [START_REF] Wilpon | A study of speech recognition for children and the elderly[END_REF]. Recently, corpora for children speech recognition have begun to emerge. In [START_REF] Eskenazi | Kids: a database of children's speech[END_REF] a small corpus of children speech was collected for use in interactive reading tutors and led to a complete children speech recognition system. In [START_REF] Shobaki | The OGI kids speech corpus and recognizers[END_REF], a more extensive corpus consisting of 1100 children, from kindergarten to grade 10, was collected and used to develop a speech recognition system for isolated word and finite state grammar vocabularies for U.S. English.

Speaking rate notably affects the recognition performances, thus ROS dependent models were studied [START_REF] Mirghafori | Towards robustness to fast speech in ASR[END_REF]. It was also noticed that ROS dependent models are often getting less speakerindependent because the range of speaking rate shown by different speakers is not the same [START_REF] Pfau | Creating Hidden Markov Models for fast speech[END_REF], and that training procedures robust to sparse data need to be used. In that sense, comparative studies have shown that rate-adapted models per-formed better than rate-specific models [START_REF] Wrede | An investigation of modeling aspects for rate-dependent speech recognition[END_REF]. Speaking rate can be estimated on line [START_REF] Pfau | Creating Hidden Markov Models for fast speech[END_REF], or computed from a decoding result using a generic set of acoustic models, in which case a rescoring is applied for fast or slow sentences [START_REF] Nanjo | Speaking-rate dependent decoding and adaptation for spontaneous lecture speech recognition[END_REF]; or the various rate dependent models may be used simultaneously during decoding [START_REF] Chesta | Connected digit recognition using short and long duration models[END_REF][START_REF] Zheng | Effective acoustic modeling for rate-of-speech variation in large vocabulary conversational speech recognition[END_REF].

The Signal-to-Noise Ratio (SNR) also impacts recognition performances, hence, besides or in addition to noise reduction techniques, SNR-dependent models have been investigated. In [START_REF] Song | Speech recognition in car noise environments using multiple models according to noise masking levels[END_REF] multiple sets of models are trained according to several noise masking levels and the model set appropriate for the estimated noise level is selected automatically in recognition phase. In contrast, in [START_REF] Sakauchi | Robust speech recognition based on HMM composition and modified Wiener filter[END_REF] acoustic models composed under various SNR conditions are run in parallel during decoding.

The same way, speech variations due to stress and emotions has been addressed by the multi-style training [START_REF] Lippmann | Multistyle training for robust isolated-word speech recognition[END_REF][START_REF] Paul | A speaker-stress resistant HMM isolated word recognizer[END_REF], and simulated stress token generation [START_REF] Bou-Ghazale | Duration and spectral based stress token generation for HMM speech recognition under stress[END_REF][START_REF] Bou-Ghazale | Improving recognition and synthesis of stressed speech via feature perturbation in a source generator framework[END_REF]. As for all the improved training methods, recognition performance is increased only around the training conditions and degradation in results is observed as the test conditions drift away from the original training data.

Automatic clustering techniques have also been used for elaborating several models per word for connected-digit recognition [START_REF] Rabiner | HMM clustering for connected word recognition[END_REF]. Clustering the trajectories (or sequences of speech observations assigned to some particular segment of the speech, like word or subword units) deliver more accurate modeling for the different groups of speech samples [START_REF] Korkmazskiy | Generalized mixture of HMMs for continuous speech recognition[END_REF]; and clustering training data at the utterance level provided the best performances in [START_REF] Shinozaki | Spontaneous speech recognition using a massively parallel decoder[END_REF].

Multiple modeling of phonetic units may be handled also through the usual triphone-based modeling approach by incorporating questions on some variability sources in the set of questions used for building the decision trees: gender information in [START_REF] Neti | Phone-context specific genderdependent acoustic-models for continuous speech recognition[END_REF]; syllable boundary and stress tags in [START_REF] Paul | Extensions to phone-state decision-tree clustering: single tree and tagged clustering[END_REF]; and voice characteristics in [START_REF] Suzuki | Speech recogntion using voicecharacteristic-dependent acoustic models[END_REF].

When multiple modeling is available, all the available models may be used simultaneously during decoding, as done in many approaches, or the most adequate set of acoustic models may be selected from a priori knowledge (for example network or gender), or their combination may be handled dynamically by the decoder. This is the case for parallel Hidden Markov Models [START_REF] Brugnara | A family of parallel Hidden Markov Models[END_REF] where the acoustic densities are modulated depending on the probability of a master context HMM being in certain states. In [START_REF] Zolnay | Acoustic feature combination for robust speech recognition[END_REF], it is shown that log-linear combination provides good results when used for integrating probabilities provided by acoustic models based on different acoustic feature sets. More recently Dynamic Bayesian Networks have been used to handle dependencies of the acoustic models with respect to auxiliary variables, such as local speaking rate [START_REF] Shinozaki | Hidden mode HMM using bayesian network for modeling speaking rate fluctuation[END_REF], or hidden factors related to a clustering of the data [START_REF] Korkmazsky | Hidden factor dynamic bayesian networks for speech recognition[END_REF][START_REF] Matsuda | Speech recognition system robust to noise and speaking styles[END_REF].

Multiple models can also be used in a parallel decoding framework [START_REF] Zhang | Speaker-independent isolated word recognition using multiple Hidden Markov Models[END_REF]; then the final answer results from a "voting" process [START_REF] Fiscus | A post-processing system to yield reduced word error rates: Recognizer Output Voting Error Reduction (ROVER)[END_REF], or from the application of elaborated decision rules that take into account the recognized word hypotheses [START_REF] Barrault | Variability of automatic speech recognition systems using different features[END_REF]. Multiple decoding is also useful for estimating reliable confidence measures [START_REF] Utsuro | A confidence measure based on agreement among multiple LVCSR models -correlation between pair of acoustic models and confidence[END_REF].

Also, if models of some of the factors affecting speech variation are known, adaptive training schemes can be developed, avoiding training data sparsity issues that could result from cluster-based techniques. This has been used for instance in the case of VTL normalization, where a specific estimation of the vocal tract length (VTL) is associated to each speaker of the training data [START_REF] Welling | Speaker adaptive modeling by vocal tract normalization[END_REF]. This allows to build "canonical" models based on appropriately normalized data. During recognition, a VTL is estimated in order to be able to normalize the feature stream before recognition. The estimation of the VTL factor can either be perform by a maximum likelihood approach [START_REF] Lee | Speaker normalization using efficient frequency warping procedures[END_REF][START_REF] Zhan | Vocal tract length normalization for large vocabulary continuous speech recognition[END_REF] or from a direct estimation of the formant positions [START_REF] Eide | A parametric approach to vocal tract length normalization[END_REF][START_REF] Lincoln | A fast method of speaker normalisation using formant estimation[END_REF]. More general normalization schemes have also been investigated [START_REF] Gales | Multiple-cluster adaptive training schemes[END_REF], based on associating transforms (mostly linear transforms) to each speaker, or more generally, to different clusters of the training data. These transforms can also be constrained to reside in an reduced-dimensionality eigenspace [START_REF] Gales | Cluster adaptive training for speech recognition[END_REF]. A technique for "factoring-in" selected transformations back in the canonical model is also proposed in [START_REF] Gales | Acoustic factorization[END_REF], providing a flexible way of building factor-specific models, for instance multispeaker models within a particular noise environment, or multienvironment models for a particular speaker.

Auxiliary acoustic features

Most of speech recognition systems rely on acoustic parameters that represent the speech spectrum, for example cepstral coefficients. However, these features are sensitive to auxiliary information inherent in the speech signal such as pitch, energy, rate-of-speech, etc. Hence attempts have been made in taking into account this auxiliary information in the modeling and in the decoding processes.

Pitch, voicing and formant parameters have been used since a long time, but mainly for endpoint detection purposes [START_REF] Rabiner | A pattern recognition approach to voiced-unvoiced-silence classification with applications to speech recognition[END_REF] making it much more robust in noisy environments [START_REF] Martin | Voicing parameter and energy-based speech/non-speech detection for speech recognition in adverse conditions[END_REF]. Many algorithms have been developed and tuned for computing these parameters, but are out of the scope of this paper.

For what concerns speech recognition itself, the most simple way of using such parameters (pitch, formants and/or voicing) is their direct introduction in the feature vector, along with the cepstral coefficients, for example periodicity and jitter are used in [START_REF] Thomson | Use of voicing features in HMM-based speech recognition[END_REF] and formant and auditory-based acoustic cues are used together with MFCC in [START_REF] Holmes | Using formant frequencies in speech recognition[END_REF][START_REF] Selouani | Distinctive features, formants and cepstral coefficients to improve automatic speech recognition[END_REF]. Correlation between pitch and acoustic features is taken into account in [START_REF] Kitaoka | Speaker independent speech recognition using features based on glottal sound source[END_REF] and an LDA is applied on the full set of features (i.e. energy, MFCC, voicing and pitch) in [START_REF] Ljolje | Speech recognition using fundamental frequency and voicing in acoustic modeling[END_REF]. In [START_REF] De Wet | Evaluation of formant-like features on an automatic vowel classification task[END_REF], the authors propose a 2dimension HMM to extract the formant positions and evaluate their potential on a vowel classification task. In [START_REF] Garner | On the robust incorporation of formant features into hidden markov models for automatic speech recognition[END_REF], the authors integrate the formant estimations into the HMM formalism, in such a way that multiple formant estimate alternatives weighted by a confidence measure are handled. In [START_REF] Tolba | Comparative experiments to evaluate the use of auditorybased acoustic distinctive features and formant cues for robust automatic speech recognition in low-snr car environments[END_REF], a multi-stream approach is used to combine MFCC features with formant estimates and a selection of acoustic cues such as acute/grave, open/close, tense/lax, ... Pitch has to be taken into account for the recognition of tonal languages. Tone can be modeled separately through specific HMMs [START_REF] Yang | Hidden Markov Model for Mandarin lexical tone recognition[END_REF] or decision trees [START_REF] Wong | Decision tree based tone modeling for chinese speech recognition[END_REF], or the pitch parameter can be included in the feature vector [START_REF] Chen | New methods in continuous mandarin speech recognition[END_REF], or both information streams (acoustic features and tonal features) can be handled directly by the decoder, possibly with different optimized weights [START_REF] Shi | Discriminative HMM stream model for Mandarin digit string speech recognition[END_REF]. Various coding and normalization schemes of the pitch parameter are generally applied to make it less speaker dependent; the derivative of the pitch is the most useful feature [START_REF] Liu | The effect of fundamental frequency on mandarin speech recognition[END_REF], and pitch tracking and voicing are investigated in [START_REF] Huank | Pitch tracking and tone features for Mandarin speech recognition[END_REF]. A comparison of various modeling approaches is available in [START_REF] Demeechai | Recognition of syllables in a tone language[END_REF]. For tonal languages, pitch modeling usually concerns the whole syllable; however limiting the modeling to the vowel seems sufficient [START_REF] Chen | Recognize tone languages using pitch information on the main vowel of each syllable[END_REF].

Voicing has been used in the decoder to constrain the Viterbi decoding (when phoneme node characteristics are not consistent with the voiced/unvoiced nature of the segment, corresponding paths are not extended) making the system more ro-bust to noise [START_REF] Shaughnessy | Towards a robust/fast continuous speech recognition system using a voicedunvoiced decision[END_REF].

Pitch, energy and duration have also been used as prosodic parameters in speech recognition systems, or for reducing ambiguity in post-processing steps. These aspects are out of scope of this paper.

Dynamic Bayesian Networks (DBN) offer an integrated formalism for introducing dependence on auxiliary features. This approach is used in [START_REF] Stephenson | Speech recognition with auxiliary information[END_REF] with pitch and energy as auxiliary features. Other information can also be taken into account such as articulatory information in [START_REF] Stephenson | Automatic speech recognition using dynamic Bayesian networks with both acoustic and articulatory variables[END_REF] where the DBN utilizes an additional variable for representing the state of the articulators by direct measurement (note that these experiments require a very special X-ray Microbeam database). As mentioned in previous section, speaking rate is another factor that can be taken into account in such a framework. Most experiments deal with limited vocabulary sizes; extension to large vocabulary continuous speech recognition is proposed through an hybrid HMM/BN acoustic modeling in [START_REF] Markov | Hybrid HMM/BN LVCSR system integrating multiple acoustic features[END_REF].

Another approach for handling heterogeneous features is the TANDEM approach used with pitch, energy or rate of speech in [START_REF] Magimai-Doss | Modelling auxiliary features in tandem systems[END_REF]. The TANDEM approach transforms the input features into posterior probabilities of sub-word units using artificial neural networks (ANNs), which are then processed to form input features for conventional speech recognition systems.

Finally, auxiliary parameters may be used to normalize spectral parameters, for example based on measured pitch [START_REF] Singer | Pitch dependent phone modelling for HMM based speech recognition[END_REF], or used to modify the parameters of the densities (during decoding) through multiple regressions as with pitch and speaking rate in [START_REF] Fujinaga | Multiple-regression Hidden Markov Model[END_REF].

Pronunciation modeling techniques

As mentioned in the introduction of Section 2, some speech variations, like foreign accent or spontaneous speech, affect the acoustic realization to the point that their effect may be better described by substitutions and deletion of phonemes with respect to canonical (dictionary) transcriptions.

As a complementary principle to multiple acoustic modeling approaches reviewed in Section 4.2.2, multiple pronunciations are generally used for the vocabulary words. Hidden model sequences offer a possible way of handling multiple realizations of phonemes [START_REF] Hain | Dynamic HMM selection for continuous speech recognition[END_REF] possibly depending on phone context. For handling hyper articulated speech where pauses may be inserted between syllables, ad hoc variants are necessary [START_REF] Matsuda | Speech recognition system robust to noise and speaking styles[END_REF]. And adding more variants is usually required for handling foreign accents.

Modern approaches attempt to build in rules underlying pronunciation variation, using representations frameworks such as FSTs [START_REF] Hazen | Pronunciation modeling using a finite-state transducer representation[END_REF][START_REF] Seneff | Statistical modeling of phonological rules through linguistic hierarchies[END_REF], based on phonological knowledge, data and recent studies on the syllabic structure of speech, for instance in English [START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF] or French [START_REF] Adda-Decker | Investigating syllabic structures and their variation in spontaneous french[END_REF].

In [START_REF] Adda-Decker | Investigating syllabic structures and their variation in spontaneous french[END_REF], an experimental study of phoneme and syllable reductions is reported. The study is based on the comparison of canonical and pronounced phoneme sequences, where the latter are obtained through a forced alignment procedure (whereas [START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF] was based on fully manual phonetic annotation). Although results following this methodology are affected by ASR errors (in addition to "true" pronunciation variants), they present the advantage of being able to benefit from analysis of much larger and diverse speech corpora. In the alignment procedure, the word representations are defined to allow the dropping of any phoneme and/or syllable, in order to avoid limiting the study to pre-defined/already know phenomena. The results are presented and discussed so as to study the correlation of reduction phenomena with respect to the position of the phoneme in the syllable, the syllable structure and the position of the syllable within the word. Within-word and cross-word resylabification (frequent in French but not in English) is also addressed. The results reinforce previous studies [START_REF] Greenberg | Linguistic dissection of switchboard-corpus automatic speech recognition systems[END_REF] and suggest further research in the use of more elaborate contexts in the definition of ASR acoustic models. Context-dependent phonemes could be conditioned not only on neighboring phones but also on the contextual factors described in this study. Such approaches are currently being investigated [START_REF] Lamel | Alternate phone models for conversational speech[END_REF][START_REF] Messina | Context dependent long units for speech recognition[END_REF]. These rely on the modeling capabilities of acoustic models that can implicitly model some pronunciation effect [START_REF] Dupont | A study of implicit anf explicit modeling of coarticulation and pronunciation variation[END_REF][START_REF] Hain | Implicit modelling of pronunciation variation in automatic speech recognition[END_REF][START_REF] Jurafsky | What kind of pronunciation variation is hard for triphones to model?[END_REF], provided that they are represented in the training data. In [START_REF] Hain | Implicit modelling of pronunciation variation in automatic speech recognition[END_REF], several phone sets are defined within the framework of triphone models, in the hope of improving the modeling of pronunciation variants affected by the syllable structure. For instance, an extended phone set that incorporates syllable position is proposed. Experimental results with these novel phone sets are not conclusive however. The good performance of the baseline system could (at least partly) be attributed to implicit modeling, especially when using large amounts of training data resulting in increased generalization capabilities of the used models. Also it should be considered that "continuous" (or "subtle") pronunciation effects are possible (e.g. in spontaneous speech), where pronunciations cannot be attributed to a specific phone from the phone set anymore, but might cover "mixtures" or transitional realizations between different phones. In this case, approaches related to the pronunciation lexicon alone will not be sufficient.

The impact of regional and foreign accents may also be handled through the introduction of detailed pronunciation variants at the phonetic level [START_REF] Adda-Decker | Pronunciation variants across system configuration, language and speaking style[END_REF][START_REF] Humphries | Using accent-specific pronunciation modelling for robust speech recognition[END_REF]. Introducing multiple phonetic transcriptions that handle alterations produced by nonnative speakers is a usual approach, and is generally associated to a combination of phone models of the native language with phone models of the target language [START_REF] Bartkova | Language based phone model combination for asr adaptation to foreign accent[END_REF][START_REF] Bonaventura | Speech recognition methods for non-native pronunciation variants[END_REF][START_REF] Witt | Off-line acoustic modelling of non-native accents[END_REF]. However adding too many systematic pronunciation variants may be harmful [START_REF] Strik | Modeling pronunciation variation for ASR: a survey of the literature[END_REF].

Alteration rules can be defined from phonetic knowledge or estimated from some accented data [START_REF] Livescu | Lexical modeling of non-native speech for automatic speech recognition[END_REF]. Deriving rules using only native speech of both languages is proposed in [START_REF] Goronzy | Generating non-native pronunciation variants for lexicon adaptation[END_REF]. [START_REF] Raux | Automated lexical adaptation and speaker clustering based on pronunciation habits for non-native speech recognition[END_REF] investigates the adaptation of the lexicon according to preferred phonetic variants. When dealing with various foreign accents, phone models of several languages can be used simultaneously with the phone models of the target language [START_REF] Bartkova | Multiple models for improved speech recognition for non-native speakers[END_REF], multilingual units can be used [START_REF] Uebler | Recognition of non-native german speech with multilingual recognizers[END_REF] or specialized models for different speaker groups can be elaborated [START_REF] Cincarek | Speech recognition for multiple non-native accent groups with speaker-group-dependent acoustic models[END_REF]. Multilingual phone models have been investigated for many years in the hope of achieving language independent units [START_REF] Bonaventura | Multilingual speech recognition for flexible vocabularies[END_REF][START_REF] Dalsgaard | Crosslanguage merged speech units and their descriptive phonetic correlates[END_REF][START_REF] Köhler | Multilingual phonemes recognition exploiting acoustic-phonetic similarities of sounds[END_REF][START_REF] Schultz | Language independent and language adaptive large vocabulary speech recognition[END_REF]. Unfortunately language independent phone models do not provide as good results as language dependent phone models when the latter are trained on enough speech data, but language independent phone models are useful when little or no data exists in a particular language and their use reduces the size of the phoneme inventory of multilingual speech recognition systems. The mapping between phoneme models of different languages can be derived from data [START_REF] Weng | A study of multilingual speech recognition[END_REF] or determined from phonetic knowledge [START_REF] Uebler | Multilingual speech recognition in seven languages[END_REF], but this is far from obvious as each language has his own characteristic set of phonetic units and associated distinctive features. Moreover, a phonemic distinguishing feature for a given language may hardly be audible to a native of another language.

As mentioned in section 2.4, variations of the speaking rate may deeply affect the pronunciation. Regarding this source of variability, some approaches relying upon an explicit modeling strategy using different variants of pronunciation have been proposed; a multi-pass decoding enables the use of a dynamically adjusted lexicon employed in a second pass [START_REF] Fosler-Lussier | Effects of speaking rate and word predictability on conversational pronunciations[END_REF]. The acoustic changes, such as coarticulation, are modeled by directly adapting the acoustic models (or a subset of their parameters, i.e. weights and transition probabilities) to the different speaking rates [START_REF] Bard | Taking the hit: leaving some lexical competition to be resolved post-lexically[END_REF][START_REF] Martinez | Towards speech rate independence in large vocabulary continuous speech recognition[END_REF][START_REF] Morgan | Speech recognition using on-line estimation of speaking rate[END_REF][START_REF] Shinozaki | Hidden mode HMM using bayesian network for modeling speaking rate fluctuation[END_REF][START_REF] Zheng | Rate of speech modeling for large vocabulary conversational speech recognition[END_REF]. Most of the approaches are based on a separation of the training material into discrete speaking rate classes, which are then used for the training of rate dependent models. During the decoding, the appropriate set of models is selected according to the measured speaking rate. Similarly, to deal with changes in phone duration, as it is the case for instance for variation of the speaking rate, alteration schemes of the transition probabilities between HMM states are proposed [START_REF] Martinez | Characteristics of slow, average and fast speech and their effects in large vocabulary continuous speech recognition[END_REF][START_REF] Mirghafori | Fast speakers in large vocabulary continuous speech recognition: analysis & antidotes[END_REF][START_REF] Morgan | Speech recognition using on-line estimation of speaking rate[END_REF]. The basic idea is to put high/low transition probability (exit probability) for fast slow/speech. These compensation techniques require a priori ROS estimation using one of the measures described in section 2.4. In [START_REF] Zheng | Rate of speech modeling for large vocabulary conversational speech recognition[END_REF], the authors proposed a compensation technique that does not require ROS estimation. This technique used a set of parallel rate-specific acoustic and pronunciation models. Rate switching is permitted at word boundaries to allow within-sentence speaking rate variation.

The reader should also explore the publications from [START_REF]ISCA Tutorial and Research Workshop, Pronunciation Modeling and Lexicon Adaptation for Spoken Language Technology (PMLA-2002)[END_REF].

Larger and diverse training corpora

Driven by the availability of computational resources, there is a still ongoing trend in trying to build bigger and hopefully better systems, that attempt to take advantage of increasingly large amounts of training data. This trend seems in part to be related to the perception that overcoming the current limited generalization abilities as well as modeling assumptions should be beneficial. This however implies more accurate modeling whose parameters can only be reliably estimated through larger data sets.

Several studies follow that direction. In [START_REF] Nguyen | Large corpus experiments for broadcast news recognition[END_REF], 1200 hours of training data have been used to develop acoustic models for the English broadcast news recognition task, with significant improvement over the previous 200 hours training set. It is also argued that a vast body of speech recognition algorithms and mathematical machinery is aimed at smoothing estimates toward accurate modeling with scant amounts of data.

More recently, in [START_REF] Lamel | Alternate phone models for conversational speech[END_REF], up to 2300 hours of speech have been used. This has been done as part of the EARS project, where training data of the order of 10000 hours has been put together. It is worth mentioning that the additional very large amounts of training data are usually either untranscribed or automatically transcribed. As a consequence, unsupervized or lightly supervized approaches (e.g. using closed captions) are essential here.

Research towards making use of larger sets of speech data are also involving schemes for training data selection, semisupervised learning, as well as active learning [START_REF] Venkataraman | An efficient repair procedure for quick transcriptions[END_REF]. These allow to minimize the manual intervention required while preparing a corpus for model training purposes. 4A complementary perspective to making use of more training data consists in using knowledge gathered on speech variations in order to synthesize large amounts of acoustic training data [START_REF] Girardi | Creating speaker independent HMM models for restricted database using straight-tempo morphing[END_REF].

Finally, another approach is proposed in [START_REF] Dupont | Feature extraction and acoustic modeling: an approach for improved generalization across languages and accents[END_REF], with discriminant non-linear transformations based on MLPs (Multi-Layer Perceptrons) that present some form of genericity across several factors. The transformation parameters are estimated based on a large pooled corpus of several languages, and hence presents unique generalization capabilities. Language and domain specific acoustic models are then built using features transformed accordingly, allowing language and task specificity if required, while also bringing the benefit of detailed modeling and robustness to any tasks and language. A important study of the robustness of similarly obtained MLP-based acoustic features to domains and languages is also reported in [START_REF] Stolcke | Cross-domain and cross-language portability of acoustic features estimated by multilayer perceptrons[END_REF].

Conclusion

This paper gathers important references to literature related to the endogenous variations of the speech signal and their importance in automatic speech recognition. Important references addressing specific individual speech variation sources are first surveyed. This covers accent, speaking style, speaker physiology, age, emotions. General methods for diagnosing weaknesses in speech recognition approaches are then highlighted. Finally, the paper proposed an overview of general and specific techniques for better handling of variation sources in ASR, mostly tackling the speech analysis and acoustic modeling aspects.

Note that these widely used ASR front-end techniques make use of frequency scales that are inspired by models of the human auditory system. An interesting critical contribution to this has however been provided in[START_REF] Hunt | Spectral signal processing for asr[END_REF], where it is concluded that so far, there is little evidence that the study of the human auditory system has contributed to advances in automatic speech recognition.

[START_REF] Abrash | Acoustic adaptation using nonlinear transformations of HMM parameters[END_REF] Equivalent to the detection of the transition point between the two adjoining QSSs.

[START_REF] Tur | Combining active and semi-supervised learning for spoken language understanding[END_REF] Combining active and semi-supervised learning for spoken language understanding. Methods of similar inspiration are also used in the framework of training models for spoken language understanding.

Acknowledgments

This review has been partly supported by the EU 6th Framework Programme, under contract number IST-2002-002034 (DIVINES project). The views expressed here are those of the authors only. The Community is not liable for any use that may be made of the information contained therein.