Monja A Knoll
email: monja.knoll@port.ac.uk

Maria Uther
email: maria.uther@brunel.ac.uk

Alan Costall
email: alan.costall@port.ac.uk

EFFECTS OF FILTERED SPEECH ON AFFECT Effects of low-pass filtering on the judgment of vocal affect in speech directed to infants, adults and foreigners

Keywords: Low-pass filtering, infant-directed speech, foreign-directed speech, vocal affect

Low-pass filtering has been used in emotional research to remove the semantic content from speech on the assumption that the relevant acoustic cues for vocal affect remain intact. This method has also been adapted by recent investigations into the function of infant-directed speech (IDS). Similar to other emotion-related studies that have utilised various levels of low-pass filtering, these IDS investigations have used different frequency cut-offs. However, the effects of applying these different low-pass filters to speech samples on perceptual ratings of vocal affect are not well understood.

Samples of natural IDS, foreigner-(FDS) and British adult-directed (ADS) speech were low-pass filtered at four different cut-offs (1200 Hz, 1000 Hz, 700 Hz, 400 Hz), and affective ratings of these were compared to those of the original samples. The samples were also analyzed for mean fundamental frequency (F 0 ) and F 0 range. Whilst IDS received consistently higher affective ratings for all filters, the results of the adult conditions were more complex. ADS received significantly higher ratings of positive vocal affect than FDS with the lower cut-offs (1000 to 400 Hz), whereas no significant difference between the adult conditions was found in the original and 1200 Hz conditions. No difference between the adult conditions was found for encouragement of attention. These findings show that low-pass filtering leaves sufficient vocal affect for detection by raters between IDS and the adult conditions, but that residual semantic information in filters above 1000 Hz may have a confounding affect on raters' perception.

Introduction

The voice carries a considerable amount of information about a person's emotional state that is communicated via both semantic and vocal acoustic channels (e.g. [START_REF] Kramer | Judgment of personal characteristics and emotions from nonverbal properties of speech[END_REF][START_REF] Starkweather | Vocal behavior as an information channel of speaker status[END_REF]. Investigation of the emotional acoustic characteristics of speech requires separation of the semantic channel from the vocal acoustic aspects of speech, and this goal has been a recurring concern in emotional speech research. A common method to separate these components has been to standardize the content of the speech by asking the speakers to repeat written text or a list of numbers (e.g. [START_REF] Davitz | Nonverbal vocal communication of feeling[END_REF]. However, the difficulty of obtaining true emotional content with these artificial situations is well understood (e.g. [START_REF] Kramer | Judgment of personal characteristics and emotions from nonverbal properties of speech[END_REF]. In an attempt to avoid these problems, alternative means of deriving contentfree speech such as playing speech samples backwards (e.g. [START_REF] Knower | Analysis of some experimental variations of simulated vocal expressions of the emotions[END_REF], foreign speech (e.g. [START_REF] Kramer | Elimination of verbal cues in judgments of emotion from voice[END_REF] and randomized content-splicing (e.g. [START_REF] Scherer | Randomized splicing: a note on a simple technique for masking speech content[END_REF]) have been employed, allowing the potential use of naturally occurring speech. These approaches nonetheless presented a variety of new problems, for example playing speech samples backwards causes reversal of the acoustic intonation contours.

An alternative widely used method of separating the two channels, low-pass filtering, has been adapted from research into speech intelligibility (e.g. [START_REF] French | Factors governing the intelligibility of speech sounds[END_REF][START_REF] Pollack | Effects of high pass and low pass filtering on the intelligibility of speech in noise[END_REF] on the assumption that such filtering would leave a large number of the emotional acoustic cues of the voice intact (e.g. Soskin and Kaufman, 1961;[START_REF] Scherer | Minimal cues in the vocal communication of affect: judging emotions from content-masked speech[END_REF]. It has been suggested that these cues (e.g. intonation contour, rate of speech, pause and rhythm) should remain virtually unaffected by the removal of the upper frequencies (e.g. [START_REF] Rogers | Content filtering human speech: A simple electronic system[END_REF][START_REF] Scherer | Randomized splicing: a note on a simple technique for masking speech content[END_REF][START_REF] Scherer | Minimal cues in the vocal communication of affect: judging emotions from content-masked speech[END_REF], since it is mainly the lower frequencies and tonal quality of the voice that are supposed to be important in communicating the speakers' emotional state [START_REF] Starkweather | Vocal behavior as an information channel of speaker status[END_REF].

Because a key advantage of low-pass filtering is that it can be applied to natural speech samples, one area where this approach has recently been used to investigate affect is research into speech directed to infants (IDS). It is believed that IDS serves distinct linguistic, emotional affective and attentional roles (e.g. [START_REF] Fernald | Expanded intonation contours in mother's speech to newborns[END_REF]Uther et al., 2003), and a growing body of research has attempted to separate these functions (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Fernald | Expanded intonation contours in mother's speech to newborns[END_REF][START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF][START_REF] Trainor | Is infant-directed speech prosody a result of the vocal expression of emotion?[END_REF]. Methods to accomplish this have included investigating acoustic and perceptual aspects of IDS by comparing IDS to other linguistic [START_REF] Papousek | Tone and intonation in Mandarin babytalk to presyllabic infants: comparison with registers of adult conversation and foreign language instruction[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] and emotional comparison groups (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Trainor | Is infant-directed speech prosody a result of the vocal expression of emotion?[END_REF]. In the case of adult linguistic and emotional comparison groups for IDS, content-free speech is clearly crucial for comparing the semantically less challenging IDS to the semantically complex adult speech. Consequently, low-pass filtering has played a key role in the investigation of the affective content in such studies (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF]. Two of these comparative studies (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] into IDS, and other investigations into children's vocal emotional understanding (e.g. [START_REF] Morton | Children's understanding of emotion in speech[END_REF], have utilized different filters.

The applicability of low-pass filtering for speech research has mainly been assessed by comparing a single filter cut-off with the original speech samples (e.g. [START_REF] Starkweather | The communication-value of content-free speech[END_REF], content-spliced speech (e.g. [START_REF] Scherer | Minimal cues in the vocal communication of affect: judging emotions from content-masked speech[END_REF][START_REF] Van Bezooijen | The effects of low-pass filtering and random splicing on the perception of speech[END_REF], reiterant speech (Friend and Farrer, 1993) and foreign speech (e.g. [START_REF] Kramer | Elimination of verbal cues in judgments of emotion from voice[END_REF]. These and other studies in the area (e.g. [START_REF] Cohen | Vocal cues to the identification of language[END_REF][START_REF] Milmoe | The doctor's voice: postdictor of successful referral of alcoholic patients[END_REF] have also used a wide variety of different filter cut-offs.

Although, higher cut-offs have also been used (e.g. 1040 Hz, [START_REF] Caporael | The paralanguage of caregiving: baby talk to the institutionalized aged[END_REF], most of these studies have chosen filter cut-offs below 650 Hz. However, it is not clear how much of the relevant nonverbal information may be lost with the removal of the upper frequencies [START_REF] Kramer | Judgment of personal characteristics and emotions from nonverbal properties of speech[END_REF], and no consensus has emerged as to the optimal filter for removing the semantic content, while at the same time conserving most of the acoustic emotional content.

To our knowledge, there is only one existing study [START_REF] Ross | Contribution of the lower audible frequencies to the recognition of emotions[END_REF] that has investigated the effect of different filters (between 600 Hz and 150 Hz) on the recognition of basic emotions. The authors found that the higher filters resulted in good recognition rates, and that only the lowest filter (150 Hz) resulted in a considerable deterioration in the raters' ability to correctly identify the emotions. This study, however, was restricted to a low range of filter cut-offs and used speech samples of actors instructed to play out specific emotions whilst reciting a fixed text, rather than spontaneous emotional speech.

The extent to which a wide and higher range of different levels of low-pass filtering influences perception of affect in natural speech samples therefore remains unknown. Investigation of this factor is crucial, not only for justification of using lowpass filtering in future emotional speech research, but also for validation of previous findings from similar studies. This is clearly pertinent in the area of IDS, where the use of natural speech samples is of great importance. However, this issue is equally important to other areas of speech research including hearing impairment, hearing aid design (e.g. [START_REF] Hogan | High-frequency audibility: Benefits for hearingimpaired listeners[END_REF][START_REF] Baer | Effects of low pass filtering on the intelligibility of speech in noise for people with and without dead regions at high frequencies[END_REF], speech recognition (e.g. [START_REF] Dubno | Recognition of filtered words in noise at higher-than-normal levels: decreases in scores with and without increases in masking[END_REF] and the design of automated speech recognition systems [START_REF] White | Linear predictive residual analysis compared to bandpass filtering for automatic speech recognition[END_REF], where frequency filtering (low-pass, high-pass and band-pass) continues to be used.

Here, we attempt to address these questions by investigating how a series of progressively lower filters influence raters' perceptions of vocal affect. We chose IDS, foreigner-(FDS) and British adult-directed speech (ADS) as comparative speech recipient groups, given the perceived high emotional content of IDS relative to these other types of speech (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF]. This approach allows direct investigation of the effects of filtering across filters and between speech produced by the same speakers directed to different audiences. Using a filter of 1000 Hz, it was previously found that IDS was rated higher on positive vocal affect and encouragement of attention than ADS and FDS, whereas ADS was rated higher than FDS in positive vocal affect, but not in encouragement of attention (e.g. [START_REF] Knoll | Motherese' and 'Chin-ese': evidence of acoustic changes in speech directed at infants and foreigners[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF]. We were particularly interested to explore how the use of a lower filter in the same data set (such as the 400 Hz filter of [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF] would have influenced these affective ratings. To achieve this aim, we used low-pass filters of 400 Hz [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF], 700 Hz (as an approximate average of studies that have used filter cut-offs between 650 and 720 Hz; e.g. [START_REF] Pollack | Effects of high pass and low pass filtering on the intelligibility of speech in noise[END_REF][START_REF] Scherer | Minimal cues in the vocal communication of affect: judging emotions from content-masked speech[END_REF], 1000 Hz [START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF], and added a further filter cut-off of 1200 Hz (intended as a bridge between commonly used filters and unfiltered speech), and compared them to the original speech samples. The purpose of the present study, therefore, is to determine at which level of frequency cut-off, if any, low-pass filtering influences the perception of vocal affect.

Method

Design

We applied a mixed factorial design with two factors. The first factor consisted of five different levels of low-pass filtered speech samples (400 Hz, 700 Hz, 1000 Hz, 1200 Hz and original; between subject measures), and the second factor consisted of the three speech recipient groups (IDS, FDS and ADS; within subjects). The dependent variables were rated vocal affect, and acoustic measures (mean F 0 (Hz) and F 0 range (Hz)).

Raters

A total of 117 people took part in the experiment; 26 males and 91 females with a mean age of 26.3 years (sd 9.0). Participants were required to have no known hearing impairment and had to be fluent in English. This was clearly stipulated in the instructions, and participants had to be capable of understanding and following complex written technical instructions in English (78.4% of the participants were English native speakers). Preliminary analyses of language background (English as first or second language) and gender of the listeners were found to be non-significant, and showed that these factors did not influence the results. Calculation of intra-class correlation coefficient (two-way random model) showed that inter-rater reliability was high for each filter condition (reliability coefficient range from α = .936 (original) to α = .892 (400 Hz)).

Stimuli selection and filtering

An existing data set [START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] consisting of recordings of ten southern English mothers (mean age 30.7 years) talking to their infants (mean age 37 weeks), a foreign (Chinese) and British adult confederate (both females in their early twenties) formed the basis of the present dataset. For the interactions, the mothers were provided with three toys to retain consistency of conversation content, but otherwise the interactions were natural and spontaneous. The interactions lasted between 5 and 10 minutes, but following previous research (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF], only approximately 30 seconds of each mother's voice was extracted. Extraneous sounds (e.g. the voice of the confederate) were edited out. Each sample was chosen from approximately the start of the mother's dialogue, but speech occurring during the first 30 seconds was generally rejected since the beginning of each interaction was mostly accompanied by distractive sounds. The speech samples were then subjected to successive low-pass filtering (Hann window) using Praat 4.3.14 [START_REF] Boersma | Praat: doing phonetics by computer[END_REF], to remove frequencies above 400 Hz, 700 Hz, 1000 Hz and 1200 Hz. For all of these, the standard recommendation for smoothing at 100 Hz was employed.

Set-up of the web-based questionnaire

The questionnaire was delivered via the Internet, hosted by the University of Portsmouth (for validity of Internet obtained ratings see [START_REF] Knoll | Evaluation of the utility of the Internet for emotional-affective speech perception studies[END_REF]. We did not use basic emotional categories (e.g. joy, anger) for the investigations of affect, because, in contrast to previous research (e.g. Ross et al., 2007), we had not instructed our speakers to convey any specific emotion. Instead, we adapted a questionnaire from that of [START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF], which has been used to investigate affect in natural IDS in several studies (e.g. [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF]. The questionnaire consisted of four affective scales: 1) positive and 2) negative vocal affect, 3) encouragement of attention, and 4) comforting and soothing (see [START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF] for a full description of these scales). Ratings were obtained on a five point Likert-scale (1 (not at all) to 5 (extremely)). We focused attention on what seemed to be the most critical scales to compare with previous research [START_REF] Knoll | Motherese' and 'Chin-ese': evidence of acoustic changes in speech directed at infants and foreigners[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF], namely the scales of positive vocal affect (as an example of an emotional affective scale) and encouragement of attention (as an example of an emotional and communicative scale). We also concluded that both of these scales might be more informative in the context of adult speech than, for instance, 'comforting and soothing', which relates more specifically to investigations of IDS.

Procedure and background information

Raters were recruited via a snowballing approach, whereby messages for volunteers were sent, amongst others, to Universities in the UK. Raters were sequentially assigned to one of the five filter conditions, whereby the first person clicking on the link would be in the original condition, the second person in the 1200 Hz condition and so forth (pseudo-random assignment). However, not all of the participants completed the study. This resulted in an assignment of 24 raters in the original and 700 Hz conditions, 21 raters in the 1000 Hz and 400 Hz conditions, and 27 raters in the 1200 Hz condition. Each rater was required to listen to 30 speech samples (10 IDS, 10 FDS and 10 ADS). The order of presentation of the speech samples was randomized for each of the raters, and they were supplied with two test trials of each filtered condition for familiarization with their respective sound. Raters were instructed to adjust the loudness level to their own comfort and hearing ability, as later speech samples would only be played once to avoid habituation effects.

Acoustic analyses

Different levels of low-pass filtering might lead to a decrease in the measures of some of the acoustic features (e.g. F 0 range 1 ), possibly influencing perceptual ratings within each filter. We therefore acoustically analyzed each of the speech samples for mean F 0 (Hz) and F 0 range (Hz). Analyses were carried out with Praat 4.3.14 [START_REF] Boersma | Praat: doing phonetics by computer[END_REF]. The pitch floor was set at 75 Hz with a ceiling of 600 Hz.

All of our analyses concentrated on the whole (unmodified) 30 second speech samples, firstly because acoustic analyses (e.g. F1/F2 ratio) of individual target words of these data had already been presented in earlier work [START_REF] Knoll | Motherese' and 'Chin-ese': evidence of acoustic changes in speech directed at infants and foreigners[END_REF][START_REF] Knoll | Good performers know their audience! Identification and characterisation of pitch contours in infant and foreigner-directed speech[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF]. Secondly, we were interested in the overall pattern of acoustic characteristics for each speech sample, and what the raters would be able to perceive during their rating task. Mean F 0 and F 0 range are important acoustic characteristics of basic emotions [START_REF] Scherer | Vocal communication of emotion: A review of research paradigms[END_REF], and analyses of these might provide some insight into the cues that the raters may have used for their ratings.

Results

The data for the two affective scales (positive vocal affect and encouragement of attention) and the acoustic analyses were subjected to a mixed MANOVA with speech recipient groups (IDS, FDS and ADS) as within subjects factor, and filters (400 Hz, 700 Hz, 1000 Hz, 1200 Hz and original) as between subjects factor. Results were further explored with univariate ANOVAs and planned contrasts. Asterisks indicate Greenhouse Geissser adjustment.

1 F 0 as measured in this study is an average over the 30 seconds varying auditory signal. Therefore higher frequencies may be attenuated by the lower filters and can therefore result in lower mean F 0 and F 0 range values.

Affective scales

We found a significant two way interaction for speech recipient groups and filters (F (16, 333.6) = 2.790, p < .001, Wilks' Lambda = .681, η = .086), with both positive vocal affect (F (6.2, 172.6*) = 2.279, p = .037, η = .075) and encouragement of attention (F (5.4, 151.4*) = 3.627, p = .003, η = .115) contributing to this interaction effect (Fig. 1). This interaction is due to a slight decreasing trend for both IDS and FDS in the scale of positive vocal affect with no significant interaction between these two conditions, whereas ADS exhibits a weak parabolic curve and interacts significantly with IDS (p = .028) and FDS (p = .009). There is no significant interaction between ADS and FDS (both display a slight parabolic curve) in the scale of encouragement of attention, whereas IDS again exhibits a slight decreasing trend, interacting with both ADS (p = .003) and FDS (p = .01). We also found a significant effect for speech recipient groups (F (4, 109) = 150.679, Wilks' Lambda = .153, p < .001, η = .847) with both scales contributing to these differences (positive vocal affect: F (1.5, 172.6*) = 423.861, p < .001, η = .791; and encouragement of attention: F (1.4, 151.4*) = 343.624, p < .001, η = .754). IDS achieved significantly higher ratings of positive vocal affect and encouragement of attention than both ADS (for both scales at p < 0.001) and FDS (for both scales at p < .001).

ADS also achieved higher ratings of positive vocal affect than FDS (p < .001), however, no significant difference between the adult conditions was observed for ratings of encouragement of attention (Fig. 1).

To explore whether these differences between the speech recipient groups were consistent within each of the five filters, we ran planned contrasts between the speech groups for each of the filters for each of the scales. As can be seen in Table 1, IDS achieved consistently higher ratings of positive vocal affect and encouragement of attention than both adult groups for each of the five filters. ADS only received higher ratings of positive vocal affect than FDS in the 1000 Hz, 700 Hz and 400 Hz filters, but not in the original or the 1200 Hz condition. No difference between the two adult groups in the ratings of encouragement of attention was found for any of the five filters.

Table 1 about here

Acoustic analyses

There was no significant interaction between speech recipient groups and filter type for the acoustic data. We found a significant effect of filters on acoustic measurements (F (8, 88) = 6.733, p < 0.001, Wilks' Lambda = .385, η = .380), although the five filter groups differed only in terms of measures of F 0 range (F (4, 45) = 15.492, p < .001, η = .579) but not for measures of mean F 0 . The measures of F 0 range in the 400 Hz condition were significantly lower than in the remaining filter conditions (at p < .001). No other significant differences between the filters were observed (see Fig. 2).

Fig. 2 about here.

We also found a significant effect of speech groups on acoustic measurements (F (4, 42) = 19.967, p < .001, Wilks' Lambda = .345, η = .655). Univariate ANOVAs revealed that both mean F 0 (F (1.1, 47.9*) = 51.425, p < .001, η = .533) and F 0 range (F (1.4, 60.9*) = 15.764, p < .001, η = .259) contributed to the differences (Fig. 2). IDS measures of mean F 0 and F 0 range were significantly higher than both ADS (for both measures at p < .001) and FDS (for both measures at p < .001). Although, the ADS measures of mean F 0 were significantly higher than FDS (p = .006), there was no significant difference between the two adult conditions for F 0 range (see Fig. 2).

These findings are broadly comparable with our earlier investigations of mean F 0 and F 0 contour shape in target words, where IDS was found to comprise higher mean F 0 and more exaggerated F 0 contours than both adult conditions [START_REF] Knoll | Good performers know their audience! Identification and characterisation of pitch contours in infant and foreigner-directed speech[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF].

Discussion

The results of our study indicate that different low-pass filters influence raters' perception of positive vocal affect and, to a lesser degree, encouragement of attention.

Consistent with the assumption that removal of the upper frequencies may lead to a loss of some of the emotional cues in speech, we found that ratings for IDS in both scales, and ratings for FDS for positive vocal affect, gradually decreased as a function of the cut-off frequency. This was not the case for FDS in encouragement of attention, or for ADS in both scales. We suggest that loss of intelligibility across the filters may partly be responsible for these results, and we discuss this possibility below.

In line with previous research (e.g. [START_REF] Kitamura | Pitch and communicative intent in mother's speech: adjustments for age and sex in the first year[END_REF], we found that IDS was consistently rated more highly (for both scales) than both adult conditions across the filter conditions. This result is unsurprising, as IDS is a highly affective speech register, and previous studies using two different low-pass filter cutoffs [START_REF] Burnham | What's new pussycat? On talking to babies and animals[END_REF][START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] had already reported increased emotional affect in IDS compared to adult-speech groups. We also found increased mean F 0 and greater F 0 range in IDS, whereas neither ADS nor FDS displayed the same exaggeration or increase in F 0 , which is concordant with earlier findings (e.g. [START_REF] Fernald | Expanded intonation contours in mother's speech to newborns[END_REF][START_REF] Knoll | Good performers know their audience! Identification and characterisation of pitch contours in infant and foreigner-directed speech[END_REF]. These acoustic features, which are less affected by filtering, have been reported to have both an emotional-affective and attentional function (e.g. [START_REF] Fernald | Expanded intonation contours in mother's speech to newborns[END_REF][START_REF] Knoll | Good performers know their audience! Identification and characterisation of pitch contours in infant and foreigner-directed speech[END_REF], perhaps explaining the consistently higher ratings of IDS in both scales.

The higher ratings of positive vocal affect for ADS compared with FDS reported in [START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] were replicated only in the lower filters (1000 Hz to 400 Hz). It is possible that sufficient semantic content of the original samples remains in the relatively liberal 1200 Hz filter cut-off to provide the raters not only with acoustic cues, but also with some contextual semantic information. This notion is also potentially supported by our findings that the original (but closely followed by the 1200 Hz) condition received significantly higher ratings of positive vocal affect than the remaining conditions. [START_REF] Uther | Do you speak E-n-g-l-i-s-h? A comparison of foreigner-and infant-directed speech[END_REF] proposed that the lower ratings of positive vocal affect found in FDS might be due to speaker frustration. Our findings are consistent with these interpretations, particularly if certain aspects of speaker frustration reside only in vocal acoustic features rather than in semantic content. For instance, raters may have taken semantic content into account in the higher filters in their evaluation of positive affect, but were forced to concentrate on acoustic signals in the more unintelligible filters. It is interesting to note that ADS was not affected by this 'semantic effect', as ratings here did not decrease gradually with the cut-off frequencies (in both scales). This suggests that semantic content and acoustic features of natural ADS may be mutually supportive, whereas the same does not apply to FDS or IDS. However, the acoustic signal responsible for this effect is not clear. Although ADS was characterized by higher F 0 (which may be associated with the basic "positive" emotion of joy/elation; [START_REF] Scherer | Vocal communication of emotion: A review of research paradigms[END_REF] in four of the conditions, this was not the case for the 400 Hz condition, where the mean difference in ratings between the two adult conditions was the greatest. It is therefore possible that another acoustic modification of FDS (e.g. speech rate; [START_REF] Biersack | Fine-tuning speech registers: A comparison of the prosodic features of child-directed and foreigner-directed speech[END_REF] is responsible for the reduced positive affect in FDS, and we are currently investigating this possibility.

In contrast to positive vocal affect, we did not find a significant difference between the adult conditions for the scale of encouragement of attention. It is possible that the speakers did not attempt to gain the attention of their speech partner in the adult interactions (supported by the overall low ratings in this scale in the adult conditions), and that it was therefore not possible for the raters to detect such a difference. We suspect that encouragement of attention may be well suited to investigations involving infants, where acoustic modifications (e.g. greater F 0 range) are of great importance in attracting the attention of the speech recipient (e.g. Knoll et al., 2007), but the scale may be less sensitive in detecting differences between the two adult groups. This assumption would need further investigation of speech samples where speakers are required to engage the attention of their speech partner, for instance, teachers addressing pupils in a classroom.

Our results nonetheless indicate that affective comparisons may be carried out with confidence using filters of 1000 Hz to 400 Hz, but that the 1200 Hz filter may not remove enough of the semantic content for this specific purpose. These findings have important implications for infant research utilizing natural speech samples. For instance, the use or choice of a filter is not necessarily critical if IDS is to be compared to a less emotionally charged adult group, but a more severe filter cut-off (≤ 1000 Hz) may be needed for discrimination between speech directed to adults where affective content is similar.

As natural interactions are normally accompanied by some degree of background noise, and since the intelligibility of filtered speech varies depending on the proportion of background noise [START_REF] Pollack | Effects of high pass and low pass filtering on the intelligibility of speech in noise[END_REF]Rogers et al., 1973), selection of the appropriate filter very much depends on the level of intelligibility remaining after that filter has been applied. Our results suggest that 1000 Hz was sufficiently severe to impair intelligibility in the present study, although this might vary for other situations where speech is recorded under optimal conditions (i.e. without background noise in a laboratory; see [START_REF] French | Factors governing the intelligibility of speech sounds[END_REF]. Future studies should thus include questions designed to investigate raters' perceptions of speech sample intelligibility.

Of course, if emotional affect is compared between different speech groups, consistency of filter cut-off between conditions remains crucial to the experimental set-up.

In summary, the purpose of the present study was to investigate whether the affective salience of speech is left unaffected when different levels of low-pass filtering are used to eliminate intelligibility. The results show that perception of affect is influenced by different frequency filter cut-offs, but that this influence is relatively small, with the greatest effects occurring in the adult conditions. Low-pass filtering is thus a useful tool in affective speech research, particularly where different types of adult-directed speech are the focus of the investigation. These findings suggest that different filters (≤ 1000 Hz) can yield similar results, but that it is obviously necessary to use the same filter for the comparison of different speech types.

Figure captions

Fig. 1

 1 Fig. 1 about here

Fig. 1 .

 1 Fig. 1. Comparison of ratings for positive vocal affect (left) and encouragement of

Fig. 2 .

 2 Fig. 2. Comparison of mean F 0 (Hz; left) and F 0 range (Hz; right) for each of the three

Figure 1 Figure 2

 12 Figure1

Table captions Table 1 .

 captions1 Mean difference, F-value and significance level between IDS, ADS and FDS for each of the five filter conditions for positive vocal affect and encouragement of attention.

Table 1 Positive vocal affect Encouragement of attention

 1

			Mean diff.	F-value	Sig. level	Mean diff.	F-value	Sig. level
	Original	IDS vs ADS	1.246	93.333	p < .001	1.570	96.126	p < .001
		IDS vs FDS	1.312	129.441	p < .001	1.708	121.689	p < .001
		ADS vs FDS	Ns	Ns	Ns	Ns	Ns	Ns
	1200	IDS vs ADS	1.241	90.927	p < .001	1.715	84.484	p < .001
		IDS vs FDS	1.363	89.918	p < .001	1.623	68.102	p < .001
		ADS vs FDS	Ns	Ns	Ns	Ns	Ns	Ns
	1000	IDS vs ADS	1.333	102.19	p < .001	1.676	59.002	p < .001
		IDS vs FDS	1.633	108.924	p < .001	1.819	69.488	p < .001
		ADS vs FDS	-0.3	10.988	p = .003	Ns	Ns	Ns
	700	IDS vs ADS	1.158	125.995	p < .001	1.496	177.724	p < .001
		IDS vs FDS	1.454	225.358	p < .001	1.521	134.683	p < .001
		ADS vs FDS	-0.296	13.844	p = .001	Ns	Ns	Ns
	400	IDS vs ADS	0.776	40.825	p < .001	0.857	15.429	p < .001
		IDS vs FDS	1.205	72.318	p < .001	0.933	22.547	p < .001
		ADS vs FDS	-0.429	41.411	p < .001	Ns	Ns	Ns

ACKNOWLEDGEMENTS

This research was supported by a grant from the ESRC to Monja Knoll. We thank Pushpendra Singh (Newcastle University, UK), Paul Marshman (Portsmouth University, UK), James Uther (Sydney University, Australia), Axelle Philippon (Portsmouth University, UK) and Stig Walsh (NHM, UK) for technical support and discussion. Two anonymous reviewers are thanked for greatly improving this manuscript.


Mónica Caballero

Asunción Moreno
email: asuncion@gps.tsc.upc.edu

Albino Nogueiras
email: albino@gps.tsc.upc.edu

Multidialectal Spanish Acoustic Modeling for Speech Recognition

Keywords: Multidialectal ASR system, dialect-independent ASR system, Spanish acoustic modeling, Spanish dialects

During the last years, language resources for speech recognition have been collected for many languages and specifically, for global languages. One of the characteristics of global languages is their wide geographical dispersion, and consequently, their wide phonetic, lexical, and semantic dialectal variability. Even if the collected data is huge, it is difficult to represent dialectal variants accurately. This paper deals with multidialectal acoustic modeling for Spanish. The goal is to create a set of multidialectal acoustic models that represents the sounds of the Spanish language as spoken in Latin America and Spain. A comparative study of different methods for combining data between dialects is presented. The developed approaches are based on decision tree clustering algorithms. They differ on whether a multidialectal phone set is defined, and in the decision tree structure applied.

Besides, a common overall phonetic transcription for all dialects is proposed. This transcription can be used in combination with all the proposed acoustic modeling approaches. Overall transcription combined with approaches based on defining a multidialectal phone set leads to a full dialect-independent recognizer, capable to recognize any dialect even with a total absence of training data from such dialect.

Multidialectal systems are evaluated over data collected in five different countries: Spain, Colombia, Venezuela, Argentina and Mexico. The best results given by multidialectal systems show a relative improvement of 13% over the results obtained with monodialectal systems. Experiments with dialect-independent systems have been conducted to recognize speech from Chile, a dialect not seen in the training process. The recognition results obtained for this dialect are similar to the ones obtained for other dialects.

Introduction

Dialectal variability is a significant degrading factor in automatic speech recognition (ASR) performance. Research shows that a mismatch in dialects between training and testing speakers significantly influences recognition accuracy in several languages like French (Brosseau, 1992), Japanese [START_REF] Kudo | Data collection of Japanese dialects and its influence into speech recognition[END_REF], Dutch [START_REF] Diakolukas | Development of a dialect-specific speech recognizers using adaptation methods[END_REF], German [START_REF] Fischer | Speaker-independent upfront dialect adaptation in a large vocabulary continuous speech recognizer[END_REF] or English [START_REF] Chengalvarayan | Accent-Independent universal HMM-based speech recognizer for American, Australian and British English[END_REF], as an example. Spanish is not an exception, as it has been shown in research (de la [START_REF] De La Torre | Evaluation of the Telefónica I+D natural numbers recognizer over different dialects os Spanish from Spain and America[END_REF]Zissman, 1996;[START_REF] Aalburg | Approaches to foreign-accented speakerindependent speech recognition[END_REF]. Efforts in dialect ASR technology have followed two different goals: (i) to improve dialectal recognition rates by developing recognition systems tailored to specific dialects, and (ii) to design multidialectal ASR systems robust to dialect variation. The primary tools to achieve these goals are lexical and acoustic modeling, while the existence and availability of language resources are the main constraints.

Concerning lexical modeling, a common approach consists in adapting the lexicon to represent dialectal variants, either by adding alternative pronunciations to the lexicon or by adapting the transcription to a given dialect [START_REF] Beringer | German regional variantsa problem for automatic speech recognition?[END_REF][START_REF] Bosch | ASR, dialects, and acoustic/phonological distances[END_REF][START_REF] Baum | A Phonetic Lexicon for Adaptation in ASR for Austrian German[END_REF]. Results show that when using the same set of acoustic models for all dialects, lexical modeling does not lead to a significant improvement. Therefore, lexical modeling alone is not enough to achieve good results and has to be combined with acoustic modeling.

Statistical acoustic models have been shown to retain accent and dialect information in a consistent way; they have been widely used in the study of dialectal variation and in the identification and phonetic classification of language variants in a data-driven manner. Different acoustic measures can be applied over dialect-dependent Hidden Markov Models (HMM) to create dialect maps [START_REF] Heeringa | Norwegian dialects examined perceptually and acoustically[END_REF][START_REF] Salvi | Accent clustering in Swedish using the Battacharya distance[END_REF], or use the recognition accuracy of dialect-dependent acoustic models to evaluate dialect distances. Training dialect-dependent acoustic models is only possible if dialect data are available, and several approaches can be found in the literature to cope with data scarcity in dialectal ASR applications. If there are enough data, a specific dialect recognizer can be built totally independent of the recognizer developed for the language or standard dialect [START_REF] Fischer | Speaker-independent upfront dialect adaptation in a large vocabulary continuous speech recognizer[END_REF]. This approach requires a dialect identification module when a system has to deal with different accents or dial! ects. More recent approaches are based on sharing data and resources between dialects. Data from one or more dialects can be used to increase the amount of training data of one monodialectal system [START_REF] Kirchhoff | Cross-dialectal data sharing for acoustic modeling in Arabic speech recognition[END_REF], or to build a set of multidialectal acoustic models that can be used to recognize speech from several dialects [START_REF] Chengalvarayan | Accent-Independent universal HMM-based speech recognizer for American, Australian and British English[END_REF]. The latter approach seems to be the more robust, since the variations in the way the same phone can be pronounced in different dialects cause the resultant acoustic models to provide greater acoustical space coverage.

Adaptation methods can be applied to well-trained acoustic models to obtain a set of models that are specific to a dialect with a limited amount of dialect speech data. In [START_REF] Diakolukas | Development of a dialect-specific speech recognizers using adaptation methods[END_REF][START_REF] Fischer | Speaker-independent upfront dialect adaptation in a large vocabulary continuous speech recognizer[END_REF] adaptation is applied to models trained with the standard dialect resources. Alternatively, multidialectal acoustic models could be adapted in the same way as [START_REF] Schultz | Language independent and language adaptative acoustic modelling for speech recognition[END_REF] do in a multilingual approach with language-independent models.

Techniques similar to those used in multilingual acoustic modeling research can be used to define a multidialectal set of acoustic models (i.e. each dialect is handled as a different language). In order to define and properly train the multidialectal acoustic models, similar phonetic units have to be identified across dialects. The similarity between the sounds of different languages -or different dialects -can either be defined by an expert, or be estimated by data-driven methods. Expert methods use linguistic knowledge. The most common approach is based on IPA (or SAMPA) alphabet: phones of different languages are considered similar if they map onto the same class as defined by IPA (or SAMPA) [START_REF] Byrne | Towards language independent acoustic modeling[END_REF][START_REF] Chengalvarayan | Accent-Independent universal HMM-based speech recognizer for American, Australian and British English[END_REF]. As a result of this procedure, a global phone set is defined for all the languages. In data-driven methods, similarity between phonetic units across languages is commonly estimated by evaluating the distance of their language-dependent acoustic models (i.e. HM! Ms) using agglomerative [START_REF] Köhler | Multilingual phone models for vocabulary-independent speech recognition tasks[END_REF][START_REF] Salvi | Accent clustering in Swedish using the Battacharya distance[END_REF][START_REF] Imperl | Clustering of triphones using phoneme similarity estimation for the definition of a multilingual set of triphones[END_REF], decision tree based [START_REF] Schultz | Language independent and language adaptative acoustic modelling for speech recognition[END_REF], or a combination of decision tree and agglomerative [START_REF] Mariño | Monolingual and bilingual Spanish-Catalan speech recognizers developed from SpeechDat databases[END_REF] clustering algorithms. Other data-driven approaches find the similarity between phones by means of a confusion matrix [START_REF] Byrne | Towards language independent acoustic modeling[END_REF]. Measuring similarity between language-context-dependent phonetic units, such as demiphones [START_REF] Mariño | Monolingual and bilingual Spanish-Catalan speech recognizers developed from SpeechDat databases[END_REF], triphones [START_REF] Imperl | Clustering of triphones using phoneme similarity estimation for the definition of a multilingual set of triphones[END_REF] or pentaphones [START_REF] Schultz | Language independent and language adaptative acoustic modelling for speech recognition[END_REF] provide better recognition results than measuring similarity between language-context-independent units. In addition, [START_REF] Imperl | Clustering of triphones using phoneme similarity estimation for the definition of a multilingual set of triphones[END_REF] conclude that although an agglomerative clustering algorithm yields a limited number of clusters, the decision tree method gives better recognition results and solves modeling units that are not seen in the training data.

Concerning the structure of the decision tree in context modeling, a distinct decision tree is typically grown for each unit (or each state of each unit) in the phone set. Another approach is to build a single global decision tree structure that allows parameters to be shared by different phones. The single global decision tree structure was used in [START_REF] Duchateau | A novel node splitting criterion in decision tree construction for semi-continuous HMMs[END_REF][START_REF] Yu | Enhanced tree clustering with single pronunciation dictionary for conversational speech recognition[END_REF] for improving monolingual acoustic modeling. In [START_REF] Caballero | Data driven multidialectal phone set for Spanish dialects[END_REF] authors applied this tree structure in the multidialectal acoustic modeling of three Spanish dialects with encouraging results.

These techniques provide robustness in acoustic modeling, but the recognition system has to know the dialect of the test speaker, either because the dialects do not share grapheme-to-phoneme transcription rules and phone sets, or because dialect information is needed to browse the decision tree.

The existence of dialect data resources is a key factor in studying and solving dialectal problems, but it is difficult and expensive to collect new data. With more than 300,000 million speakers worldwide, Spanish is one of the most widely spoken languages and is considered to be one of the global languages in the world. Dialectal variants can be found across Spain and Latin American countries, as well as within countries. Databases for properly training ASR systems for Latin American dialects are appearing. Adding to the former VAHA or CALL HOME databases available in the LDC, the SpeechDat Across Latin America (SALA) project (Moreno et al., 1998) developed a set of telephone databases in most of the Latin American countries for the purposes of training ASR systems. Some research deal with recognition of Spanish dialects or its influence in a Spanish ASR. Variability due to speakers and data from different dialects is considered to be pronunciation variation; as such, it is modeled by adding alternative pronunciations to the lexicon [START_REF] Billa | Multilingual speech recognition: the 1996 byblos callhome system[END_REF][START_REF] Ferreiros | Improving continuous speech recognition in Spanish by phone-class semicontinuous HMMs with pausing and multiple pronunciations[END_REF], or by defining a simple phonetic set [START_REF] Huerta | The development of the 1997 CMU Spanish broadcast news transcription system[END_REF] in order to integrate variability in HMM. Two examples of specific dialectal modeling can be found in [START_REF] Aalburg | Approaches to foreign-accented speakerindependent speech recognition[END_REF] and (de la [START_REF] De La Torre | Evaluation of the Telefónica I+D natural numbers recognizer over different dialects os Spanish from Spain and America[END_REF]. In the first paper, Spanish as spoken in Spain is used to model non-native speech applied to a system trained with Colombian speakers. In the second, Argentinean and Spanish as spoken in Spain are considered. Both studies apply lexical modeling and adaptation techniques in order to improve recognition accuracy for a specific dialect speech. Good results are obtained, but in both cases, the acoustic models are tailored to a single dialect. In [START_REF] Nogueiras | Multidialectal Spanish speech recognition[END_REF], authors created a multidialectal ASR system for t! hree Spanish dialects that improved monodialectal performances. Authors also showed that testing Latin American dialects in a system trained with data of Spain did not improve the monodialectal performance.

The goal of this paper is to create a multidialectal speech recognition system robust to dialect variations. The intended language is Spanish including dialectal variants from Latin America and Spain. Designing a system that is completely robust to dialectal variations requires total independence to the dialect of the speaker. A number of solutions are possible, such as adding a stage to the recognizer that identifies the dialect of the speaker or having equal representation (transcription) of the recognition vocabulary for all the dialects. In this paper, we focus on the latter approach. A new overall phonetic transcription technique common to all the Spanish dialects is proposed. In defining an overall transcription, dialect information is used not to adapt the phone set and grapheme-to-phoneme rules to a particular dialect but rather to define a phone set and rules that enable the system to detect similarities and differences between sounds by applying a clustering algori! thm in the acoustic modeling stage. This new transcription approach restricts the need for prior design decisions regarding the phone sets for each dialect and decisions regarding whether to transcribe a new dialect that is to be incorporated into the system or that is to be recognized by the system.

To create a robust multidialectal set of acoustic models, different methods for combining training data based on decision tree clustering algorithms are explored. The approaches differ on whether a multidialectal global phone set is defined and in the decision tree structure applied (i.e. multiple roots or one single global decision tree). Both, the multidialectal set of acoustic models and the overall transcription are combined with the aim of finding a robust recognizer for Spanish dialects. The resulting system is designed to be able to recognize any Spanish dialect, even when no training data for a given dialect are available.

The rest of the paper is organized as follows. Section 2 deals with Spanish language and dialects. Sections 3 and 4 describe canonical transcription rules and the overall transcription proposed for Spanish dialects. Section 5 is devoted to the methodology of multidialectal acoustic modeling. Section 6 describes the recognition system used in our research and gives an overview of the experiments carried out and the results obtained. Finally, our conclusions are presented in Section 7.

Spanish Dialects

Spanish dialects across the world

As mentioned above, Spanish is one of the global languages in the world and is also one of the most widely spoken languages. It is the official language of Spain and of nearly all Latin America countries except Brazil, Guyanas and some Caribbean islands. Furthermore, its use is growing rapidly in the United States and Australia. Spanish dialectal variants have been described in the literature and they include phonetic, lexical, semantic and cultural variations. Within Spain, one can roughly distinguish between the standard Castilian and southern dialects. With regard to Latin America, many factors prompted the appearance of dialectal variants: the varieties spoken by Spanish settlers, the state of the Spanish language in the time in which these settlers occupied the territories, contact with other languages, and linguistic drift of the dialects [START_REF] Lipski | Latin American Spanish[END_REF]. It is difficult to classify dialects or accents of Spanish in Latin America, since there are no clear boundaries between the varieties; one local variety may merge gradually into another, and it is sometimes easier to find dialectal similarities across countries than it is within countries. Spanish as spoken in Latin America is often broadly classified according to whether it is spoken in the highlands and mountains or in the lowlands and coast! al areas. This broad division is due to the Andes mountains, which cross South America from north to south and favoured an initial Castilian settle in the high lands and a later southern Spanish settle in the coast lands. In addition, both dialects can be found across the entire continent, from Mexico to Chile. In this section we describe the main phonetics characteristics of the Spanish dialects.

Phonological differences between Spanish dialects

Most of the variation between Spanish dialects occurs with consonants, particularly in the fricative class. An explanation is that when Spanish settlers took their language to America, the sounds belonging to this phonetic class were still in evolution in Spain. The main variations can be classified into phonetic differences, consonant weakness in the coda position, and lenition. Next sections describe the details of each class. Examples are shown in Table 1.

Phonetic differences

These differences consist in the substitution or use of one phoneme or allophone by another.

• Seseo

Seseo affects the use of SAMPA phonemes /T/ (IPA /T/) and /s/. The name seseo applies to those dialects that do not pronounce /T/ and pronounce /s/ instead. This is the most common effect.

• Yeismo/lleismo/zeismo

They affect the use of SAMPA phonemes /L/, /jj/, and /Z/ (/L/, /y/ and /Z/ in the IPA alphabet). The most common effects are:

• Yeismo: Only /jj/ is pronounced.

• Lleismo: /L/ and /jj/ are pronounced. It typically occurs in bilingual areas were /L/ is a phoneme of the other language (Quechua, Catalan). • Zeismo: Only /Z/ is pronounced. It occurs in east Argentina and Uruguay.

Consonant weakness in the coda position

Variations between dialects in the coda position (at the end of a word or at the end of a syllable) include the following:

• Preservation of /s/ in the coda position

The most distinctive feature of variants of Spanish is the pronunciation of /s/. The main division between Latin American Spanish dialects is characterized by the preservation of /s/ at the end of a syllable or a word in highland areas. In lowlands, it is elided or aspirated, becoming /h/. • Velarization of /n/ Nasal consonants in the coda position are velarized and become the allophone /N/ (/N/ in IPA). This effect is common in lowland variants, mostly in the Caribbean area. • Distinction of /l/ and /r/ at the end of a syllable

In lowland variants, especially in the Caribbean dialect, the distinction between laterals and vibrants in the coda position tends to be eroded. Few speakers exhibit complete neutralization.

Lenition

Lenition or softening occurs when a consonant that is considered strong becomes weak. The following are the most common cases:

• Pronunciation of the fricative /x/ The fricative voiceless velar /x/ is pronounced in Argentina, Chile, Mexico, and Spain. In Chile, it is produced as a fricative voiceless palatal /C/ (IPA /C/) when it precedes the vowels [e] and [i]. In the Caribbean area and in Colombia, it is aspirated and becomes a fricative voiceless glottal /h/.

• Elision of /D/ between vowels

The elision of /D/ is characteristic of lowlands, although it has become very common in informal speech in all dialects.

Dialect phonological characteristics

The above mentioned phonetic effects can be grouped into regions to perform a dialect map. We identified a small number of dialectal regions and for each region we found the predominant dialect (normally the variant spoken in the capital) or the variant spoken by the majority of the population:

• Mexico (ME): Represents Mexico and part of Central America. The most populated area is Mexico DF. • Caribbean (CA): Includes Caribbean Islands, Venezuela, and the Atlantic Coast of Central America and Colombia. The most populated area is Cara-

Table 1 Examples of phonological differences between Spanish dialects. Table shows for each effect, a word example and its transcription in the absence or the presence of the related effect. Differences between dialects can be explained easily by the absence or presence of the effects discussed above. Table 2 summarizes the main differences between the variants.

Effect absence

Canonical transcription: following dialectal characteristics

It is well known that Spanish grapheme to phoneme transcription can be done with rules with few exceptions. In this study, phonetic transcription is based on rules. It is carried out automatically using SAMPA symbols. [START_REF] Llisterri | Spanish adaptation of SAMPA and automatic phonetic transcription[END_REF] proposed a set of rules for transcribing Spanish as spoken in Spain. Based on that work, [START_REF] Moreno | Spanish dialects: Phonetic transcription[END_REF]

developed a set of

Table 2 Phonological differences between dialects. A bullet marks the presence of an effect.

Effect

AR CA CH CO ME SP

Seseo • • • • • - [Y/Ll/Z]eismo Z Y Y Y Ll Ll Aspiration /s/ • • • - - - Velarization /n/ - • - - - - Distinction /l/-/r/ • - • • • • Pronunciation of /x/ (/x/, /h/ or /C/) x h C h x x Elision of /D/ between vowels - • - - - -
canonical transcription rules for Latin American Spanish dialects according to the specific phonetics of each dialect. To enable dialectal pronunciation to be accurately represented and to cope with all the Latin American dialects, the symbols /h/, /C/ and /Z/ were added to the standard SAMPA symbol set for Spanish [START_REF] Gibbon | Handbook of standards and resources for spoken language resources[END_REF].

In order to transcribe Latin American dialects, the [START_REF] Moreno | Spanish dialects: Phonetic transcription[END_REF] canonical transcription rules are applied:

• In all Spanish dialects, only /jj/ is considered to exist (non-existence of /L/), except in Argentina, where both are transcribed as /Z/. • Because of seseo, across Latin America /T/ is not used and becomes /s/ in the transcription. • The velar fricative /x/ is transformed into /h/ in Colombia and in the Caribbean. In Chile, /x/ is transformed into /C/ when it precedes the vowels /e/, /i/ and /j/. • In Argentina, Chile and the Caribbean, [s] in the coda position is transcribed as /h/. • Nasal consonants in the post-nuclear position are the velar /N/ in the Caribbean.

Additionally, in this research, /R/ was added to represent the post-nuclear [r], and the phonemes /l/ and /r/ were specifically tagged as 'CG' when they belonged to a consonant group (they followed a plosive or a [f]).

Tables 3 and4 show SAMPA symbols used in canonical transcriptions of Spanish dialects. Table 3 contains phones shared across dialects, while Table 4 shows phones not present in all variants.

Overall transcription common to all dialects

Canonical transcriptions should not be followed blindly. [START_REF] Foldvik | Dialect maps and dialect research; useful tools for automatic speech recognition?[END_REF] found that traditional dialect maps may be of limited use in ASR and that dialectal boundaries are never clear-cut; however, statistical models for speech recognition retain accent information and that information may be useful for the purpose of improving ASR performance. Actually, there are no exclusive dialect rules, only phenomena that may be present in dialects or not.

A question that springs to mind is, can these effects be reflected in a single overall transcription for all dialects? An overall transcription would prevent new rules from having to be designed for every new dialect added to the recognition system and would allow any variant of Spanish to be transcribed.

The overall transcription proposed in this work uses phonetic knowledge related to changes across dialects. The process consists in modifying the canonical transcription of the standard Spanish in certain situations by marking phones that are liable to be different in different dialects. In this study, Spanish as spoken in Spain is considered to be the standard variant.

Considering the effects that change across dialects, as explained in section 2, it is easy to identify and separate special cases. Note that some approaches in the acoustic modeling stage allow dialect-dependent models to be separated or joined.

• Phonetic differences

To deal with seseo, the two phonemes /T/ and /s/ are used in order not to prejudice the dialect of Spain. For apical and sibilant expression of the phoneme /s/, the same SAMPA symbol /s/ is used for all dialects. In the case of yeismo, leismo and zeismo effects, we considered only one expression between /L/ and /jj/ in the canonical transcriptions, as the majority of the population only produces one of them. Linguistic research highlights the fact that there are areas in which these phonemes are kept and /L/ is still pronounced. In order to determine if this affects acoustic models, in overall transcription both phonemes are considered and kept separate.

• Consonantic weakness in the coda position

To mark [s], nasal ([n], [m]) and liquid [l] consonants in the coda position, a special tag, 'C', is added to their SAMPA symbols. The phoneme /N/ is kept but not marked, as it is always uttered at the end of a syllable.

• Lenition

Glottal and palatal pronunciations of /x/ are considered as allophonic variations and the same SAMPA symbol is used for the three allophones. In order to have a specific symbol for /D/ between vowels, the approximant /D/ in the coda position is also marked with the tag 'C'.

The final set of phone symbols is summarized in Table 5. All the aboved mentioned differences between the dialects are included in the set. Table 6 shows a comparison of the canonical and overall transcription for some example words.

Acoustic modeling: data sharing

In this section, we describe various techniques designed to define a set of multidialectal acoustic models combining data of different dialects. These techniques are based on decision trees clustering algorithms. We propose two tree structures and two different starting points in order to ascertain which units could benefit from other data sources.

Starting point for contextual modeling

We apply and compare two different approaches for getting the contextual models. The first one is based on the definition of a multidialectal global phone set based on SAMPA alphabet and the second approach avoids a preliminary phone set definition step.

• Definition of a multidialectal Global Phone Set (GPS). The SAMPA and IPA alphabets classify sounds based on their phonetic characteristics. This linguistic knowledge has been used to define which phones could share data in the training process and to define a global phone set. The sounds of different dialects that have the same representation in the SAMPA alphabet are considered to be the same phone. The global phone set is completed by adding phonemes that are not shared between dialects.

In this approach, all the material from all the dialects that corresponds to the same SAMPA symbol is used to train the same acoustic model. This is the most common approach in the literature and is very useful if different languages/dialects share a considerable number of symbols, as in Spanish dialects.

Once the phone set has been decided, no more dialect information is needed, so this type of measure allows the resultant acoustic models to be used to recognize a dialect that is not present in the training process, whenever no new phone is needed for that variant.

• Starting with Dialect-Context-Dependent (DCD) acoustic models. A set of context-dependent acoustic models (i.e. Hidden Markov Models HMMs) is trained for each dialect. Dialect-dependent models are marked with a dialect tag in order to be capable to distinguish them in the tree. This approach gives freedom to detect similar context-dependent acoustic units. The decision tree driven by the entropy measured over dialect-dependent models define which units (and from which dialects) are similar enough to share training data.

Dialect information is needed in the regression through the decision tree, so it does not allow speech from a dialect that is not considered in the training process to be recognized.

Tree structures

Two tree structures are studied: a multiroot structure that applies SAMPA restrictions to the clustering algorithm, and an one-root structure with no SAMPA constraints.

• Multiroot structure (MR). A different tree (root) is created for each SAMPA unit. Each root contains all the context-dependent acoustic models belonging to the same phone symbol. This is typically the structure used for context modeling in monolingual systems. Parameter sharing is not allowed between units with different SAMPA representation. When using overall transcription, units belonging to the same SAMPA symbol, even if they are marked, share the same root (e.g. /n/ and /n/ C) in order to keep one tree for each SAMPA symbol.

• One-root structure (OR). A single tree is built for all the units in the phone set. Its root contains all the context-dependent acoustic models of all the units. This structure enables data to be shared between different phones.

Multidialectal acoustic modeling approaches

Four approaches for multidialectal acoustic modeling are obtained by combining the types of starting points and decision tree structures presented above, as can be seen in Table 7. These approaches are graphically represented in Figures 1 and2. • Global phone set, multiroot tree structure (GPS-MR)

In this approach, a global phone set is defined based on the SAMPA alphabet, that is, no distinction is made between units across dialects. Context modeling is achieved by applying a decision tree clustering algorithm using a multiroot structure, as in most monolingual systems. The question set only inquires about the context of the unit. When overall transcription is used, questions relative to the unit are added to those trees which have more than one unit in their roots. This is the most immediate approach, and the most intuitive. A major drawback is its dependence on the decisions made at the transcription stage, as it is totally based on the SAMPA alphabet.

• Global phone set, one-root tree structure (GPS-OR)

In this approach, a global phone set based on SAMPA is also defined. Oneroot structure decision tree algorithm is applied. The application of this structure allows models of different phones to be joined if they are similar in certain contexts or situations. The question set contains questions about the phone itself as well as the context.

Both global phone set based approaches determine a set of dialect-independent acoustic models, which can be used for any Spanish dialect, even if there are no data available for it. With canonical transcriptions, this is only possible if all the phones of the new dialect are contained in the global phone set. Using overall transcription does not have this drawback, as all the dialects share the same transcription rules and phone symbol set.

• Dialect-context-dependent models, multiroot tree structure (DCD-MR) Dialect-dependent models are created for each contextual unit. Each root of the decision tree contains all the models whose phone is represented by the same SAMPA symbol. The question set asks for the context unit and the dialect. With this approach, similarity is only evaluated across models whose phone has the same SAMPA representation. It is possible to keep sounds represented by the same SAMPA symbol across different dialects separated if they are not very similar. For example, when overall transcription is used, both /x/ for Spain and /h/ for Colombia are transcribed as /x/. This system is able to separate SP /x/ from CO /x/ if they are really different, as they are assumed to be in canonical transcriptions.

• Dialect-context-dependent models, one-root tree structure (DCD-OR)

A single tree is grown with all the dialect-dependent models in the root node. The question set asks for the unit, the context unit, and the dialect. This approach gives maximum freedom to the clustering algorithm because no SAMPA restrictions are applied. As in the DCD-MR approach, models with the same SAMPA representation can be distinguished. The one-root tree structure allows models with distinct SAMPA representation to be joined if they are similar enough. This approach makes the system totally automatic and independent of prior phonetic knowledge. This approach seems to be able to solve errors at the transcription stage. For example, Caribbean canonical transcription assumes that /s/ in the coda position is uttered as /h/. If this is not true or it is superfluous to the acoustic models, some contextual models from Caribbean /h/ can be joined to /s/ contextual models from other dialects, and the error is thus solved. Using overall transcription, in which /L/ and /jj/ are kept separated, it is possible to cluster them if their acoustic models are really similar.

Experiments

This section presents the performance of the proposed methods for multidialectal speech recognition. A brief introduction of the in-house speech recognizer is first presented followed by the experimental set-up. A comparison of the four proposed multilingual approaches with both, canonical transcriptions and overall transcriptions is presented. One baseline recognizer was built for each dialect. The purpose was to compare the results to the multidialectal approaches proposed in this work. The performance of the dialect-independent systems is evaluated with a dialect not seen in the training data. This section ends with a discussion.

Recognition System

Our research was implemented in an in-house speech recognition system. The system is based on Semicontinuous Hidden Markov Models (SCHMMs). Speech signals are parameterized using Mel Cepstrum and each frame is represented by its Cepstrum C, derivatives ∆C and ∆∆C, and the energy derivative. The first three features are represented by 512 Gaussians and the energy derivative by 128 Gaussians.

The phonetic units are demiphones [START_REF] Mariño | The demiphone versus the triphone in a decision-tree state-tying framework[END_REF], which are contextual units that model half a phoneme by taking into account its immediate context. A phone ph is modeled by two demiphones, 'lph' 'ph + r', where l and r stay for the left and the right phone context respectively. Each demiphone is modeled by a two-state left-to-right model. The main advantage of using demiphones instead of other contextual units such as triphones is the lower number of acoustic units that need to be trained, and consequently, they can be properly trained with small databases.

All the systems use decision trees clustering algorithms. For the tree to grow, the entropy of each node is computed. A discrete approximation is used to evaluate partitions. With this approximation, the entropy of a node A can be calculated using eq. ( 1), where M is the number of models in the node, S is the number of states in each model, G is the number of Gaussians in the codebook, f (m) is the frequency of the model in the train data, f (s|m) is the quotient between the number of frames of the state s and the total number of frames of the model the state belongs to, and b sg s are the mixture weights for each of the Gaussians in the codebook.

H(A) = M m=1 f (m) S s=1 f (s|m) G g=1 b sg log b sg (1)
Splitting stop criteria are defined by a minimum decrease in entropy and/or a threshold of the number of training examples in each cluster (leaf node). To train a demiphone, 100 training examples proved to be enough.

The question set inquires as to the phonetic features of the phonetic unit the model represents (type, place and manner), the dialect of the unit, and optionally non-phonetic questions (i.e. the position in the word, whether the phone is an aspiration, or whether the phone belongs to a consonant group). Compound questions about a single attribute (e.g. manner of articulation) are allowed using a logical OR link (e.g. Is the manner in which the sound is articulated nasal OR fricative? ). The question set is completely dependent on the approach, and was discussed in section 5.

Data

Experiments were carried out with databases recorded in Argentina, Colombia, Chile, Mexico, Spain and Caribbean (Venezuela). The databases consists of fixed network telephone recordings. Except the Caribbean, signals are sampled and recorded from an ISDN line at 8kHz sampling rate, 8 bits/sample and coded using A-law. The Caribbean database was recorded from an analogue line and µ-law coded. The database from Spain contains speech from 4 000 speakers. For the purposes of our research, 3 500 speakers were selected for training and 200 for the test. The databases of Latin America contains speech from 1 000 different speakers. For training purposes, 800 speakers were selected from each database, and 200 speakers were selected for the test. The canonical phonetic transcription applied to each database coincides with the predominant dialect spoken in each country, as mentioned in section 2.2.4.

The systems were trained with a set of phonetically rich words and phonetically balanced sentences. In order to evaluate dialect-independent systems, one dialect, Chilean, was kept out of the training process. Six recognition tests were defined, one for each dialect. The recognition tests are composed of phonetically rich words. Each test speaker pronounces four of these words. Truncated and mispronunciated utterances were discarded from the test set. Each test utterance contains only one word, thus isolated word recognition language model is used. The vocabulary is identical for all dialects and has a size of 4 500 words, containing all the words appearing in the tests. Table 8 shows the total amount of training and testing data for each dialect considered. Training data for Chile do not appear in this table as they did not participate in the training process.

Monodialectal systems: baseline recognizers

For comparison purposes, one baseline recognizer was built for each dialect. A multiroot decision tree based clustering algorithm was used for context modeling. Table 9 shows the number of models of each monodialectal system. The number of models depends on the amount of training data and the phone set size because of the definition of the tree growing process. The system trained with data from Spain had the largest set of models, due to the larger amount of data available. The total number of models needed for recognizing all dialects was 3 596. Table 9 also shows the percentage of Word Error Rate (WER) for the baseline recognizers and their average value calculated as the mean value of the dialect WERs. The system for Spain gave the best result: 3.62%. Caribbean and Argentinean systems obtained around 7%. The Colombian and Mexican systems gave the worst rates.

Lower rates achieved for Latin American dialects shows a problem of data scarcity. In order to prove this, another baseline system was created for Spain with the same number of speakers as the rest of the dialects (800). The number of models created in this case was 736 and the WER obtained for the Spanish test was 6.00%. The results showed that even the WER for Spain was lower than the one obtained for other dialects using a similar amount of data, there was a loss of nearly 50% in its performance comparing when using all the available data.

Multidialectal acoustic modeling 6.4.1 Using canonical transcriptions

The approaches proposed in this paper -GPS-MR, GPS-OR, DCD-MR and DCD-OR-were developed using a canonical transcription for each dialect.

The GPS-MR and GPS-OR approaches used 988 and 981 models respectively. These figures are comparable to the sizes of monodialectal sets, since the phone sets for these approaches are similar to the monodialectal ones. Dialect querying (the DCD-MR and DCD-OR approaches) made the decision tree grow to 3 600 leaf nodes. In order to determine the optimal size of the acoustic model set, experiments with a variety of different task test sets were carried out. Different acoustic model set sizes (from 500 to the total number of lead nodes) were scanned, and the best results were obtained with 2 000 models in both cases.

Table 10 summarizes the results of the experiments of the four proposed systems. All the systems improved the baseline average word error rate. In all the approaches presented, the performance of the dialect of Spain was slightly degraded. This result is not surprising since we added variability to a welltrained system. We consider that this degradation is acceptable as a minor drawback in order to achieve a multidialectal system.

The results for both GPS systems were similar. The improvement of baseline results in both cases was caused by the reduction of the WER in the Colombian, Mexican and Caribbean variants.

The DCD-MR approach improved the average performance achieved with the GPS-MR and GPS-OR approaches, as well as the baseline results. Using the one-root tree structure (the DCD-OR approach) led to the best system, as the average word error rate was reduced to 6.63%, a relative reduction of almost 7% over the baseline results. This system outperformed all the Latin American baseline results and the WER for Spanish as spoken in Spain was almost as good as for the dialect-specific system.

In addition to the results shown in Table 10, it has to be said that in most of the experiments carried out with sets of acoustic models of different sizes, the application of the one-root tree structure gave better results than the application of multiple roots. This leads us to conclude that the one-root tree structure, that is, a structure in which models can be shared between units, allows sharing data in a consistent way.

Using overall transcription

The approaches proposed were also developed using one overall transcription. These systems are referred to as OT . The GPS-MR OT and GPS-OR OT systems produced sets of 846 and 954 acoustic models respectively. The optimal size of the acoustic model set for approaches DCD-MR OT and DCD-OR OT was found to be 2 000, the same size as using canonical transcriptions.

The recognition results obtained with these new systems are summarized in Table 11. The average result for the GPS-MR OT system was better than the baseline result, but if we look at the dialect-specific rates in more detail, we can see that the results for three of the five dialects were worse than the monodialectal ones. Substantial improvement was achieved with the GPS-OR OT system. Using this system for Colombian and Mexican dialects, the WER was reduced by more than one and two points respectively, compared with the baseline results. The Caribbean rate was also improved, while rates for Spain and Argentina variants degraded only slightly. In addition, this system balanced the word error rate between dialects. It is also interesting to remark that this system outperformed the GPS-OR system trained using canonical transcriptions, with a lower number of acoustic models.

DCD-MR OT led to the best average result; it outperformed all of the systems presented in this paper. There was a relative reduction of 13% with respect to the monodialectal results. This system obtained the best results for Argentinean and Colombian dialects, and WERs pretty close to the best results presented in this work for Mexican and Caribbean variants. The result for the dialect of Spain was no better than the result for the well-trained baseline.

DCD-OR OT also improved monodialectal rates, but contrary to what occurred with canonical transcriptions, the one-root structure used in combination with a dialect-dependent contextual models failed to improve the results obtained using a multiroot tree structure.

Evaluation of dialect-independent recognition systems in absence of dialectspecific training data

As commented in section 5, approaches based on the definition of a global phone set are able to act as dialect-independent systems. However, to be able to recognize a dialect with no training data available using canonical transcriptions, no new phone can appear in the dialect to be recognized. The WERs obtained for the Chilean test with GPS-MR OT was 8.03% and with GPS-OR OT was 7.35%. These rates were similar to those obtained for Argentina, Colombia and Mexico using those systems. This result points out that these recognizers generalize across dialects; they provide models that are able to recognize speech in dialects that are not present in the training process.

Discussion

In this section, we compare the systems trained and analyze them in terms of data sharing and tree behavior.

Data sharing between dialects

Table 12 shows, for each approach, the percentage of full multidialectal (clusters containing data of all dialects) and semi-multidialectal (clusters containing data of more than one dialect) nodes. The percentages for the DCD systems were calculated for the 2 000 acoustic model set. Maximum data sharing was provided by approaches that defined a global phone set. The total percentage of full multidialectal models for both GPS-MR and GPS-OR approaches was 70%. The GPS-OR approach slightly increased the data sharing percentage as it allowed dialect-specific models to be joined with other models in the same cluster. When overall transcription is used, all the units of the phone set are shared between dialects. Thus, the GPS-MR OT and GPS-OR OT approaches allowed to share 100% of the data.

The DCD systems allowed expressions of the same unit to be separated for different dialects. Opening the decision tree up to 2 000 clusters decreased the percentage of full multidialectal nodes. When the one-root system was applied the percentage of semi-multidialectal nodes increased substantially, while the percentage of full multidialectal nodes decreased. The DCD-MR OT system had the highest percentage of data sharing between all the dialects and the best recognition results.

To analyze the multidialectal clusters we calculated, for each pair of dialects, the percentage of clusters P (d i , d j ) that share data from those dialects. P (d i , d j ) as eq. 2, where N c (d i , d j ) is the number of clusters containing data from dialects d i and d j , and N t is the total number of clusters.

P (d i , d j ) = 1 N t N c (d i , d j )100 (2)
When these percentages were measured, all the clusters containing data from the two dialects were counted, even if there were other variants in it. Table 13 shows, for each DCD approach and dialect d i , the top 2 scored dialects d j and the calculated score P (d i , d j ). Colombian, Mexican, and Argentinean variants appear in most of the cells of the 13 shows that overall transcription allowed more data sharing between those dialects than canonical transcriptions. Actually, both dialects have speakers of the other variant. This behaviour agrees with [START_REF] Foldvik | Dialect maps and dialect research; useful tools for automatic speech recognition?[END_REF], and validates the usefulness of the overall transcription.

Tree Behavior

Concerning how the trees treat the models belonging to different phones, multiroot approaches begin with all the units separated into different trees. The one-root tree structure allowed clusters to be shared by different phones. Table 14 shows the percentage of clusters shared by more than one phone. The approaches that use overall transcription allowed more clusters to be shared between different phones than canonical transcriptions based approaches. In all the one-root tree structure approaches, the units that were most frequently tied together were semivowels (/j/, /w/) and their corresponding vowel (/i/, /u/) when there was a lack of data for a given context. In the GPS-OR trees, these type of clusters were the most common. The DCD-OR approach also had clusters shared by fricative units /s/, /h/, and /T/, in coda position. Beside fricative units, GPS-OR OT also joined /n/ C with /N/ and /D/ with /R/ in coda position. It is remarkable that this tree sepa! rates /jj/ clusters from /L/ clusters, and /T/ clusters from /s/ clusters. DCD-OR OT had the largest percentage of clusters with more than one unit in them, which can cause a loss in acoustic resolution. On the other hand, the experiments validate the use of the same symbol for allophonic variations across dialects in overall transcription if dialect information is given to the tree. With DCD-MR OT , the system that obtained the best recognition results, the decision tree detects and clusters allophone variation between dialects. For example, the tree clearly separates velar /x/ from glottal /x/ and /s/ from aspirated /s/.

Conclusions

In this paper we compared several approaches to build a robust multidialectal set of context dependent acoustic models for Spanish. The acoustic models were achieved by applying a decision tree clustering algorithm. Two tree structures were tested, multiroot, where there is a root for each considered SAMPA symbol and one-root, where the tree starts with a single root. To train the trees, two approaches were considered: dialect-independent models trained with data of all dialects and defined from a global phone set, and dialect-dependent models where each model is defined and trained with data from a single dialect.

To solve the necessity of using a canonical phonetic transcription per each dialect, a new approach, an overall transcription common to all the dialects has been proposed. The overall transcription has two advantages. First, it avoids knowing in advance the dialect of the speaker and, consequently, can be used to develop a multidialectal system to recognize speech from a broad number of dialects. Second, it overcomes errors in the phonetic transcription of the training databases due to a lack of knowledge of the dialect spoken per each speaker. The overall transcription has been successfully tested in four multidialectal approaches and compared with results obtained using a canonical transcription for each dialect. The recognition results with overall transcription overcome results with canonical transcriptions. Overall transcription is simpler and can be applied to recognize dialects non-seen during the training stage.

All the systems proposed improve monodialectal performance. Building the tree with dialect-dependent contextual models shows better performance than using context-dependent models defined from a global phone set. Concerning decision tree structure, in most of the experiments, one-root structure performs better than multiple root structure. The combination of dialect-dependent contextual models, multiroot structure and overall transcription outperforms all the other systems.

Overall transcription, in combination with the definition of a multidialectal global phone set led to a totally dialect-independent system with a reduced set of models. Its performance with one-root structure was nearly as good as the best found and it used half the number of models. This system is suitable for use with all Spanish speakers even if their dialect was not seen in the training phase

  ).• High Land (CO). Represents the high land dialect of Colombia, Ecuador and Peru. The most populated area is Bogota (Colombia). • Chile (CH). The most populated area is Santiago.• Argentina (AR) Represents Argentina and Uruguay. The most populated area is Buenos Aires. • Spain (SP). The most common dialect is spoken in Madrid.

  L jj r rr R l CG r CG

Fig. 1 .

 1 Fig. 1. Global phone set based approaches proposed to define a set of multidialectal acoustic models sharing data across dialects. (a) Approach using a multiroot tree structure. (b) Approach using one-root tree structure.

Table 3

 3 Shared phones across dialects using canonical transcriptions.

Table 4

 4 Phones not shared across all dialects using canonical transcriptions. Right column indicates dialects where phone is present.

	Allophone Phonetic Attributes	Dialects
	jj	Voiced, palatal, fricative	CA CO ME SP
	x	Voiceless, velar, fricative	AR ME SP
	h	Voiceless, glottal, fricative	AR CA CO

Z

Voiced, palatoalveolar, fricative AR T Voiceless, interdental, fricative SP C Voiceless, palatal, fricative CH

Table 5

 5 Phone symbols used in overall transcription. Symbols in bold face have been added to standard SAMPA symbols in order to reflect the differences across dialects.

Table 6

 6 Comparison between canonical and overall transcription.

	Word	Dialect	Canonical trans. Overall trans.
	caza (hunting)	SP AR CA CH CO ME	k a T a k a s a	k a T a
	halla (find) / haya (beech)	AR CA CH CO ME SP	a Z a a jj a	a L a / a jj a
	manta (blanket)	CA AR CH CO ME SP	m a N t a m a n t a	m a n t a
	más (more)	AR CA CH	m a h	m a s C
		CO ME SP	m a s
	caja (box) / mujer (woman)	CH CA HI AR ME SP	k a h a / m u C e R k a h a / m u h e R k a x a / m u x e R	k a x a / m u x e R
	red (net)	AR CA CH CO ME SP	r e D	r e D C

Table 7

 7 Multidialectal acoustic modeling approaches as a combination of the starting point of the approach and tree structure used.

				Starting point
			Global phone set Dialect-context-dep. models
	Tree	Multiroot	GPS-MR	DCD-MR
	Structure One-root	GPS-OR	DCD-OR

Table 8

 8 Training and testing data amount for dialects considered in this study. The Chilean dialect is not represented in the training process.

	DIALECT	AR	CA	CO	ME	SP	CH
	train. utterances	9 568	9 303	8 874	11 506	40 936 --
	train. running words 412 859 425 591 476 559 558 884 956 300 --
	test. utt.	722	686	640	624	718	735
	Table 9
	Number of models and WER (%) for monodialectal systems developed for Argentina,
	Caribbean, Colombia, Mexico and Spain.
	DIALECT	AR CA CO ME SP AVERAGE
	Number of models 662 688 683	716	847	WER %
	WER %	7.34 6.71 9.22 10.10 3.62	7.40

Table 11

 11 Word error rate for multidialectal recognition systems using overall transcription.

	DIALECT	GPS-

MR OT GPS-OR OT DCD-MR OT DCD-OR OT

	No Models	846	954	2 000	2 000
	ARGENTINA	7.89	7.76	5.68	6.23
	CARIBBEAN	6.85	6.56	6.71	6.71
	COLOMBIA	7.19	7.50	7.03	8.28
	MEXICO	8.50	7.70	8.66	9.29
	SPAIN	5.44	4.04	4.17	4.04
	AVERAGE	7.17	6.71	6.45	6.91

  Chilean test data was used in this research to evaluate the performance of the dialect-independent systems developed. Even most of the phenomena of Chilean dialect are seen in other dialects, the Chilean test includes one new phone /C/, not included in the training data. This phone is present in 77 utterances of the Chilean phonetically rich words test, which means a percentage of 10,5% over the whole test. The presence of the /C/ phone did not allow to use GPS approaches developed using canonical transcriptions to recognize Chilean speakers. In systems that use overall transcription, /C/ phone is modeled with /x/ models trained with all the /x/ and /h/ realizations of other dialects. Thus, only the systems that use overall transcription (the GPS-MR OT and GPS-OR OT approaches) were able to recognize Chilean speakers.

Table 12

 12 Percentage of clusters that share data between dialects for multidialectal approaches. Full multidialectal clusters contain data of all dialects. Semi-multidialectal clusters contain data of few dialects.

	Approach	Full multidialectal clusters Semi-multidialectal clusters
	GPS-MR	69.23%	20.65%
	GPS-OR	69.72%	21.61%
	GPS-MR OT	100.00%	0.00%
	GPS-OR OT	100.00%	0.00%
	DCD-MR	6.80%	11.20%
	DCD-OR	6.20%	14.85%
	DCD-MR OT	9.25%	11.35%
	DCD-OR OT	7.2%	13.5%

Table 13

 13 Percentage of clusters that share data from each pair of dialectal variants. For each dialect and approach, the table shows the top 2 dialectal variants. ME 14.2 CO 15.6 ME 13.4 CO 20.2 ME 17.5 CO 16.8 ME 14.6 CA CO 15.6 ME 13.2 CO 14.8 ME 12.4 CO 20.2 ME 17.3 CO 17.4 ME 14.2 CO AR 16.1 ME 15.5 AR 15.6 ME 15.4 ME 20.4 CA 20.2 CA 17.4 ME 17.0 ME CO 15.5 AR 14.2 CO 15.4 AR 13.4 CO 20.4 AR 17.5 CO 17.0 AR 14.6 SP AR 10.8 ME 8.3 AR 10.6 ME 7.6 AR 12.6 CO 9.5 AR 11.0 CO 7.6

	Dialect	DCD-MR	DCD-OR	DCD-MR OT	DCD-OR OT
	AR	CO 16.1

  Table, pointing out that those dialects were present in the majority of the clusters, sharing data with the rest of the variants. On the other hand, the variant of Spain shares less clusters than the rest of variants. This fact indicates that Latin American dialects did not borrow an excessive amount of data from speakers of Spain, but rather just what they need. An interesting result is that overall transcription seems to overcome the barrier to data being shared established by canonical transcriptions, which specify different transcription rules for different dialects. The behaviour of data sharing bet! ween Colombian and Caribbean dialects is an example. Table

Table 14

 14 Percentages of clusters shared by more than one phone for one-root tree structure approaches.

	GPS-OR DCD-OR GPS-OR OT DCD-OR OT
	6.6	10.80	8.5	15.3

Acknowledgements

This work was granted by Spanish Government TEC2006-13694-C03.