D Moraru

L Besacier
email: laurent.besacier@imag.fr

S Meignier
email: sylvain.meignier@univ-lemans.fr

C Fredouille
email: corinne.fredouille@lia.univ-avignon.fr

J.-F Bonastre

SPEAKER DIARIZATION IN THE ELISA CONSORTIUM OVER THE LAST 4 YEARS

This paper summarizes the collaboration of the LIA and CLIPS laboratories, members of the ELISA consortium, along the last 4 year NIST speaker diarization system evaluation campaigns. In this context, two individual approaches, quite different, have been developed individually by each lab, to respond to the specific task of speaker segmentation. The first one relies on a classical two-step speaker segmentation strategy, based on the detection of speaker turns followed by a clustering process, while the second one corresponds to an integrated strategy where both segment boundaries and speaker tying of the segments are extracted simultaneously and challenged during the whole process. From these two main methods, various strategies were investigated for the fusion of segmentation results. Through the performance achieved along the different evaluation campaigns as well as the experience gained by the LIA and CLIPS labs in the speaker diarization task, a discussion about the overall work done in this evaluation context is drawn in this paper, proposing further investigation and progression.

INTRODUCTION

Since 1996, NIST has organized yearly speaker recognition evaluation campaigns, focusing on the automatic speaker detection task. In 2000, the evaluation of speaker segmentation systems was introduced as a new task. Also called speaker diarization, this task consists in segmenting a conversation involving multiple speakers into homogeneous parts which contain the voice of only one speaker, and in grouping together all the homogeneous segments that correspond to the same speaker. In parallel, the progress made in broadcast news transcription, moves the focus on a new task, denoted "rich transcription", for which the semantic information is not the * is at LIUM lab of the University of Le Mans since September 2004 only element of interest. Indeed, acoustic based information (sounds, speech qualities, speaker information, ...), discourse based information (disfluencies, emotion, ...), as well as linguistic information (topic, named entities, ...) may also be used to enrich the transcription and to help for indexing audio documents. Speaker characteristics are obviously an important information in this context. For this reason, the speaker diarization system evaluation has joined in 2003 the Rich Transcription system evaluation campaign. The LIA and CLIPS labs, members of the ELISA consortium, have participated since 2000 (only LIA in 2000 and 2001) in these evaluation campaigns [START_REF] Moraru | The ELISA consortium approaches in speaker segmentation during the NIST 2002 speaker recognition evaluation[END_REF][START_REF] Moraru | The ELISA consortium approaches in broadcast news speaker segmentation during the NIST 2003 rich transcription evaluation[END_REF][START_REF] Fredouille | The NIST 2004 spring rich transcription evaluation : two-axis merging strategy in the context of multiple distance microphone based meeting speaker segmentation[END_REF]. Two main strategies have been proposed and improved over the last four evaluation campaigns. One of the main characteristics of the NIST evaluation campaigns has been to propose different kinds of environment to evaluate the speaker diarization systems: telephone conversational speech, broadcast news shows as well as meeting room recordings. This paper presents the progression of the LIA and CLIPS systems in terms of speaker segmentation strategies as well as in terms of performance, according to the targeted environment. It is organized as follows: section 2 is dedicated to the LIA and CLIPS baseline speaker segmentation system description. Section 3 presents the evolution of these systems as well as further investigation to improve performance, based for instance on system fusion. Performance of the best ELISA systems over the last four years is provided in section 4. Regarding this performance, a discussion is proposed, underlining the issues raised over the last evaluation campaigns. Finally, section 5 concludes this paper and gives some perspectives.

BASELINE OF SPEAKER DIARIZATION SYSTEMS

Two different speaker segmentation systems are presented in this section. They have been developed individually by the CLIPS and LIA labs in the framework of the ELISA consortium [START_REF] Moraru | The ELISA consortium approaches in speaker segmentation during the NIST 2002 speaker recognition evaluation[END_REF][START_REF] Moraru | The ELISA consortium approaches in broadcast news speaker segmentation during the NIST 2003 rich transcription evaluation[END_REF][START_REF] Fredouille | The NIST 2004 spring rich transcription evaluation : two-axis merging strategy in the context of multiple distance microphone based meeting speaker segmentation[END_REF], using AMIRAL, the LIA Speaker Recognition system [START_REF] Fredouille | AMIRAL: a block-segmental multirecognizer architecture for automatic speaker recognition, Digital Signal Processing (DSP)[END_REF]. The CLIPS system relies on a classical two-step strategy. It involves a BIC (Bayesian Information Criterion) detector based strategy for speaker turn detection followed by a hierarchical clustering. This approach will be denoted as "step-by-step strategy" in the rest of this paper, as the information retrieved during the speaker turn detection is not questioned during the clustering phase. The second system developed by the LIA differs from the previous one by proposing an "Integrated" strategy, for which all the information is iteratively questioned along the segmentation process. It is based on a HMM modeling of the conversation and an iterative process which adds the speakers one by one.

CLIPS Step-by-Step approach

Approach overview

The CLIPS system is based on a BIC (Bayesian Information Criterion) [START_REF] Jain | Data clustering: A review[END_REF][START_REF] Chen | Speaker, environment and channel change detection and clustering via the bayesian information criterion[END_REF][START_REF] Delacourt | DISTBIC: A speaker based segmentation for audio data indexing[END_REF]

System specification

The signal is characterized by 16 Mel Cepstral features (MFCC) computed every 10ms on 20ms windows, augmented by the energy. No frame removal or any coefficient normalization is applied.

LIA integrated approach

Approach overview

The LIA system shows a different strategy, based on a Hidden Markov Modeling (HMM) of the conversation and an iterative process which adds the speakers one by one [START_REF] Meignier | E-HMM approach for learning and adapting sound models for speaker indexing[END_REF].

Each state of the HMM characterizes a speaker and the transitions model the changes between speakers. During the segmentation, the HMM is generated using an iterative process, which detects and adds a new state (i.e. a new speaker) at each iteration. The speaker detection process is composed of four steps:

• Step 1-Initialization. A first "speaker" model is trained on the whole test utterance (it is rather a generic acoustic model than a given speaker model). The conversation is modeled by a one-state HMM and the whole signal is set to the initial "speaker".

• Step 2-Adding a new speaker. A new speaker model is trained using 3 seconds of test speech that maximize the likelihood ratio computed using the first model and a world model (learned using development data). A corresponding state is added to the previous HMM.

• Step 3-Adapting speaker models. First, all the speaker models are adapted, using a MAP approach, according to the current segmentation. Then, a Viterbi decoding is done and produces a new segmentation. The adaptation and decoding steps are performed while the segmentation differs between two successive "adaptation/decoding" phases.

• Step 4-Assessing the stop criterion. The likelihood of the previous solution and the likelihood of the last solution are computed using the last HMM model (for example, the solution with two speakers detected and the solution with three speakers detected). The stop criterion is reached when no gain in terms of likelihood is observed or when no more speech is left to initialize a new speaker. Two heuristic criteria are added to the likelihood-based criterion:

-The first one removes the new speaker if the length of its segments is less than 4 seconds. The 3 second segment used for the initialization of the speaker is then marked as unavailable for the speaker initialization (step 2). The process continues with the segmentation of the previous iteration.

-The second one discards the previous speaker from the segmentation if the length of their segments is lower than the new one. This rule, which forces the detection of the longest speaker first, is closely related to the evaluation metric used in NIST campaigns where it is more important to find the longest speaker segments than the shortest ones.

System specification

The signal is characterized by linear Cepstral features (LFCC) 1computed every 10 ms using a 20ms window, augmented by the energy. No frame removal or any coefficient normalization is applied. GMM with 128 components (diagonal covariance matrix) are used for the speakers and world / background models.

CONTEXT DEPENDENT SYSTEMS

Overview of evaluation campaigns

The speaker segmentation task (also named speaker diarization or "who spoke when" task in the NIST terminology) was introduced in the NIST speaker recognition system evaluation campaign in 2000, in addition to the classical tasks based on the speaker detection. The main difference between speaker segmentation and the other tasks lies in the unavailability of prior information concerning the speakers involved in the speech signal: neither speaker identities nor speaker numbers. This particular context obviously increases the difficulties of the speaker segmentation task. Chronologically, from 2000 to 2002, this specific task was proposed during the NIST speaker recognition system evaluation campaigns [START_REF]The NIST year 2000 speaker recognition evaluation plan[END_REF][START_REF]The NIST 2001 speaker recognition evaluation plan[END_REF][START_REF]The NIST year 2002 speaker recognition evaluation plan[END_REF]. Since information retrieved from the speaker segmentation of an audio document can be easily seen as a way of enriching the transcription of this same document, the speaker segmentation system evaluation has joined the Rich Transcription system evaluation campaigns, also organized by NIST, in 2003 [START_REF]The rich transcription spring 2003 (RT-03S) evaluation plan[END_REF]13]. Along these evaluation campaigns, speaker segmentation systems were evaluated on different kinds of data:

• conversational telephone speech corpora, which normally involve two speakers, and one acoustic class only for the signal (telephone speech) for NIST-SpRec-2000, NIST-SpRec-2001, and NIST-SpRec-2002 evaluation campaigns;

• broadcast news shows, which may contain a large set of speakers, over various acoustic classes (studio speech, telephone speech, degraded speech, speech over music, music, ...) for NIST-SpRec-2002 and NIST-RT-2003 evaluation campaigns;

• meeting data, collected through (distant) table or head microphones, involving few speakers (compared with broadcast news data), but more spontaneous speech (disfluencies, voice overlapping, long silences, ...), and eventually multi-channel signals, for NIST-SpRec-2002 (mono-channel signal only) and NIST-RT-2004 (Spring) (mono-and multi-channel signals) evaluation campaigns.

These differences between corpora raise two main observations. First of all, the second main difficulty2 of the speaker segmentation task strongly depends on the type of targeted data (from fixed number of speakers to "unlimited", from one acoustic class to various ones, mono-channel vs multichannel, ...). Secondly, speaker segmentation system has to be adapted according to the type of processed data, in order to increase performance.

From baseline to context dependent systems

The CLIPS and LIA labs have participated in the speaker segmentation system evaluation campaigns since 2001 (LIA only for this year), in association with the ELISA consortium. Considering the baseline speaker segmentation systems described in the previous section, various evolutions have been proposed to cope with the different kinds of data and to increase system performance. This section will present the most important ones for each individual system and those concerning both of them. It has to be noted that LIA initiated its participation in speaker segmentation evaluation campaigns, in 2000, in collaboration with the EURECOM Institute (France). The system proposed consisted in two steps: the first step relying on a BIC based speaker turn detection, developed by the EURECOM Institute, followed by a step, based on the LIA speaker verification algorithms for segment aggregation. Some results concerning this system are provided in section 4.

Step-by-Step approach

Two main evolutions were implemented in the CLIPS stepby-step approach. The first one relies on the parameterization step for which more filter banks were used to process broadcast news data during the NIST-RT-2003 evaluation campaigns (56 filter banks) compared with the NIST-SpRec-2002 (24 filter banks only). Experimental results show in the former case, a performance improvement in terms of speaker segmentation error rates. Secondly, during the NIST-SpRec-2002 evaluation, the CLIPS work was mainly focused on telephone speech data (considered as the primary task of the evaluation). Therefore, the speaker segmentation system was constrained to find only two speakers during the speaker segmentation process, whatever the data processed. Obviously, this configuration was well-suited for conversational telephone speech, but unrealistic for broadcast news and meeting corpora, involving drastic speaker segmentation performance in 2002.

For NIST-RT-2003 in which the speaker segmentation task was dedicated to broadcast news data, the CLIPS lab investigated an original method for estimating the number of speakers, appearing in the audio documents. In this approach, the number of speakers in the conversation (N Sp ) is estimated using a penalized BIC (Bayesian Information Criterion). The number of speakers is constrained between 1 (if we are working on an isolated acoustic pre-segmentation class, see section 3.2.3 for more details) or 2 (if we are working on the entire audio file) and 25. The upper limit is related to the recording duration. The number of speakers (N sp ) is selected to maximize:

BIC(M ) = logL(X|M ) -λ m 2 N sp logN X
where M is the model composed of the N sp speaker models, N X is the total number of speech frames involved, m is a parameter that depends on the complexity of the speaker models and λ is a tuning parameter equal to 0.6. The first term is the overall log-likelihood of the data. The second term is used to penalize the complexity of the model. We need the second term because the log-likelihood of the data increases with the number of models (speakers) involved in the calculation of L(X|M ).

For NIST-RT-2004 evaluation campaign, no particular evolution was made on the system compared with the one used for broadcast news corpus in 2003.

Integrated approach

Few evolutions were implemented on the LIA integrated system, depending on the data processed. First of all, as this system relies on a UBM world model for speaker model adaptation, suitable data were chosen to estimate it (Switchboard phase II for conversational telephone speech for instance).

Secondly, as the CLIPS system, the LIA system was constrained to find two speakers only while processing conversational telephone data whereas it was unconstrained for the other kinds of data. This constraint on the number of speakers obviously improves the performance of the speaker segmentation system, when dealing with conversational telephone speech data. Thirdly, the last evolution was made at the parameterization step for which Linear frequency Cepstral Coefficients (LFCC) were preferred to the MEL ones (MFCC), especially while processing broadcast news data, for which experiments showed performance improvement.

Common evolution Prior acoustic macro-class segmentation

In 2003, the speaker evaluation campaign was focused on broadcast news data. As underlined in the previous section, various acoustic classes may be found in such data like studio speech, telephone speech, speech over music, music, ... Therefore, the CLIPS and LIA labs investigated the use of a prior macro-class acoustic segmentation before applying speaker segmentation systems, in order to:

• discard non speech signals (music, silence, ...), which have not to be processed by the speaker segmentation systems (otherwise, they involve speaker segmentation errors since they are labeled as speakers);

• provide a prior knowledge which may be interesting for the speaker segmentation systems, like gender detection or bandwidth classification (telephone vs non telephone speech).

Practically, the acoustic macro-class segmentation is first applied on each audio document, providing four sets of segments (bandwidth and gender detection), on which the speaker segmentation systems are applied individually. Finally, the segmentation outputs yielded on each individual acoustic macro-classes are merged to provide an overall segmentation.

The acoustic macro-class system used in this context relies on a hierarchical segmentation performed in three successive steps as illustrated in figure 1:

• during the first step, a speech / non speech segmentation of signal (representing a show) is performed using M ixS and N S models. The first model represents all the speech conditions while the second one represents the non speech conditions. Basically, the segmentation process relies on a frame-by-frame best model search. A set of morphological rules are then applied to aggregate frames and label segments.

• during the second step, a segmentation based on 3 classes -clean speech (S model), speech over music (SM model) and telephone speech (T model) -is performed only on the speech segments detected by the previous segmentation step. All the models involved during this step are gender-independent. The segmentation process is a Viterbi decoding applied on an ergodic HMM, composed, here, of three states (S, T , and SM models). The transition probabilities of this ergodic HMM are learnt on 1996 HUB 4 broadcast news corpus.

• the last step is devoted to gender detection. According to the label given during the previous step, each segment will be identified as female or male speech by the use of models dependent on both gender and acoustic class (GT -F e and GT -M a for female and male telephone resp., GS -F e and GS -M a for clean speech, GSM -F e and GSM -M a for speech over music, and GDS -F e and GDS -M a, representing speech recorded over degraded conditions used to refine the final segmentation). The segmentation process, described in the previous step, is applied in the same way here.

All the state models mentioned above are diagonal GMMs except N S and M ixS models which are characterized by 1 and 512 Gaussian components respectively, all the other models are characterized by 1024 Gaussian components. They were trained on the 1996 HUB 4 broadcast news corpus.

Different levels of acoustic macro-class segmentation had been proposed and evaluated in terms of speaker diarization improvement. Results of this work may be found in [START_REF] Meignier | Benefits of prior acoustic segmentation for automatic speaker segmentation[END_REF].

Multi-channel meeting evaluation

In 2004 (Spring), the focus was made on meeting data and more precisely on multi-channel speaker segmentation. In this context, the speaker segmentation system has to process multiple speech channels coming from different microphones. Therefore, the choice of an efficient merging strategy in order to discard the irrelevant information becomes a crucial issue. This point will be discussed in the next section.

No particular tuning has been done on both the LIA and CLIPS speaker segmentation systems to participate at this evaluation campaign, except the use of a speech/non speech segmentation as a preliminary phase to deal with the specificities of meeting data (background noise, long silence, ...). This speech/non speech segmentation system consisted in a silence detection based only on a bi-gaussian modeling of the energy distribution associated with a detection threshold. The silence segment minimal length was set to 0.5s.

Strategies of fusion

In 2002, and more largely in 2003, the experience gained on the individual speaker segmentation system behavior (observed during the previous evaluation campaigns) leads the LIA and CLIPS labs to investigate various possibilities for combining their systems (for more details about these strategies, see [START_REF] Moraru | The ELISA consortium approaches in speaker segmentation during the NIST 2002 speaker recognition evaluation[END_REF][START_REF] Moraru | The ELISA consortium approaches in broadcast news speaker segmentation during the NIST 2003 rich transcription evaluation[END_REF]).

Hybridization ("piped" system)

The hybridization strategy, illustrated in figure 2, consists in using the segmentation results of the CLIPS system to initialize a variant of the LIA system. Indeed, the speakers detected by the CLIPS system (number of speakers and associated audio segments) are used as an initialization of the LIA system HMM model (the models are trained using the information issued by the clustering phase), followed by an iterative process, during which adaptation and decoding steps are performed (similarly to the step 3 of the baseline LIA system). The process, involving a variant of the LIA system, is called the LIA re-segmentation process. This solution associates the advantages of longer and (quite) pure segments of the step-by step strategy with the HMM modeling and decoding power of the integrated strategy.

Merging Strategy ("fusion" system)

The idea of "fusion" is to use the segmentations issued from as many experts as experts, as shown in figure 3. The merging strategy relies on a frame based decision which consists in grouping the labels proposed by each of the four systems at the frame level. Since two many virtual speakers are thus generated, the LIA re-segmentation process (described in the previous strategy), associated with some empirical rules, is applied to discard as many irrelevant speakers as possible.

Individual Microphone Segmentation Merging Strategy

A third strategy of fusion was especially designed in 2004 by the LIA and CLIPS labs to process multi-channel audio documents, relative to the meeting corpora. The goal of this strategy was to merge the multiple distant microphone seg-mentations in a single meeting speaker segmentation output. Since no single signal is representative of the overall meeting, this strategy had to rely on some segment selection rules over the multiple distant microphone speaker segmentations. In this way, a specific merging algorithm was investigated. It relies on an iterative process which aims at detecting the longest speaker interventions over the set of distant microphone segmentations. Based on 3 successive steps, this iterative algorithm consists basically in selecting the longest speaker intervention over all microphone segmentation outputs taken separately, deleting in each distant microphone segmentation all the segments attributed to this new speaker, verifying the presence of not selected segments over all the distant microphone segmentations (more details on this algorithm are given in [START_REF] Fredouille | The NIST 2004 spring rich transcription evaluation : two-axis merging strategy in the context of multiple distance microphone based meeting speaker segmentation[END_REF]). Performance shown on the fifth line of the table illustrates the increasing difficulty of the tasks. Indeed, broadcast news and meeting data were introduced in the 2002 speaker segmentation system evaluation campaign. The ELISA systems were not really prepared for these new kinds of data. For telephone conversations, only two persons are involved, making the segmentation process easier. For broadcast news, there are obviously more speakers on the audio documents, but this is mostly prepared speech with a large part of "studio quality" voice. The hardest task definitely corresponds to meeting data with very spontaneous speech, recovering voices, disfluencies, distant speakers (in case of table microphones) and background noise. The first three lines illustrate the performance improvement of the LIA systems (LIA/EURECOM system for 2000) on telephone data over the three years. This performance was measured using the NIST 2000 metric (see the NIST evaluation plan for more details on this metric [START_REF]The NIST year 2000 speaker recognition evaluation plan[END_REF]), which was changed from the NIST 2002 evaluation campaign [START_REF]The NIST year 2002 speaker recognition evaluation plan[END_REF]. The third and fourth lines illustrate this change of the evaluation metric through the 2002 LIA system performance. The sixth line shows the best performance obtained in 2003 on broadcast news data. This performance was achieved by the ELISA "piped" system, which is based on the fusion strategy involving the LIA and CLIPS systems, as described in section 3.2.4. This line illustrates also the progress made from 2002 to 2003 on these data. Finally, the last line presents the performance of the ELISA system during the NIST 2004 (Spring) speaker segmentation evaluation. Two different results are given, relating to the mono-or multi-channel speaker segmentation tasks. Despite the simplicity of the strategy used to merge the multiple channel segmentation outputs, the multi-channel speaker segmentation system obtained the best speaker diarization performance for the corresponding task [START_REF] Fredouille | The NIST 2004 spring rich transcription evaluation : two-axis merging strategy in the context of multiple distance microphone based meeting speaker segmentation[END_REF].

CONTEXT AND PERFORMANCE COMPARISON

System evaluation

Discussion

The participation of the LIA and CLIPS labs, every year since 2000, enforced by their strong collaboration, have enabled them to gain some experience on the speaker segmentation task and more precisely on the proposed approaches: step-by-step and integrated strategies.

First of all, the work done on the system fusion has highlighted the advantages and drawbacks of each individual approach. Indeed, through the piped strategy, it has been underlined the power of the E-HMM in modeling the conversation between speakers, which can be strongly enforced when the frontiers between speakers are provided by a robust BIC based speaker turn detection strategy. Secondly, the evaluation of the E-HMM strategy over various kinds of data (conversational telephone speech, broadcast news, or meeting data) has shown the difficulty of controlling the E-HMM parameters, mainly based on heuristics. In the same way, another issue lying on the acoustic adaptation of the UBM model, used during the iterative adaptation/decoding process, was raised. Indeed, this acoustic adaptation currently relies on the speaker recognition algorithms, for which only mean parameters are adapted. In this context, it will be interesting to investigate more complex acoustic adaptation, involving either variance, weight, or both of them in order to take into account the difficulty of the task, compared with speaker recognition.

Moreover, it has been largely observed, over these different evaluation campaigns, that the characteristics of the signal files used to measure speaker segmentation system performance are very important to understand the behavior of the speaker segmentation system. The size of the signal file is the first factor. Indeed, the increase in terms of signal file duration (if implying an increase in the number of speakers), may induce some issues with the integrated approach, since the adaptation and decoding process may become less controllable, because of the lack of robustness of the current stop criterion. This last point remains an important issue of the Integrated approach, which will demand further investigation to find a more robust one. The clus-tering phase of the step-by-step approach however seems to perform better as the file duration increases since the UBM used is trained directly from the entire test file. However, the drawback with this approach is the computational time which increases exponentially with the file duration. The variability of acoustic conditions observed in the files is also important. Indeed, the presence of large amount of telephone speech in the files may help the speaker segmentation system and therefore increases overall performance. Lastly, the number of signal files has to be sufficient in order to provide a robust evaluation measure. These different points have been largely discussed during the evaluation campaigns.

Speaker diarization on meeting data may be considered as a quite new task in the domain, particularly when the segmentation process has to deal with signals coming from multiple distant (table) microphones, scattered inside the meeting room. This multi-channel task has been introduced in 2004 in addition to the mono-channel one (meeting data used for the NIST-SpRec-2002 evaluation campaign being only mono-channel ie only based on one signal file to process, corresponding most often to the most informative microphone). Currently, the LIA and CLIPS systems, despite their proposed multi-channel fusion algorithm, do not really take advantage of the multi-channel information as shown by the small difference in terms of system performance between mono-and multi-channel (26.5% vs. 22.4% resp. in table 1). As underlined before, meeting data speaker segmentation remains, far from the other environments, the hardest one. The proposed speaker segmentation approaches cannot currently deal with a large part of issues, raised by this particular context: robust speech/non speech segmentation, robust speech vs background speech segmentation, voice overlapping inside one signal file but also between signal files, fusion of segmentation outputs without signal synchronization, etc. This long list shows that speaker segmentation process in the context of meeting data remains an open-domain.

Finally, in most research works on speaker segmentation, one of the main assumption is that there is no a priori information available on the test data. This means that there is no knowledge of the number or the identity of the speakers involved and in consequence there is no reference speaker data available for any one of them. This limitation may however not be so rough for certain applications and conditions where a priori data might be available. Generally the type of conversation is known (broadcast news, telephone or meeting). This gives us information on speech quality and average speaker turn length. In some cases, reference data might be available for some of the speakers. For broadcast news data for instance we can easily obtain reference data for the news host directly from the previous shows. A simple tracking system of this particular speaker can then lead to an error reduction. Some studies have been done in this sense by the authors (see [START_REF] Moraru | Using a priori information for speaker diarization[END_REF]) and have shown the interest of a such approach. Further investigation has to be done in this way.

CONCLUSION

This paper presents the work done by the LIA and CLIPS labs since their first participation in the speaker segmentation system evaluation campaigns in 2001 (LIA only for this year). Over these four years, two main strategies have been proposed individually by each lab, to perform speaker diarization task: the CLIPS Step-by Step approach, based on a BIC detection criterion followed by a clustering process and the LIA integrated approach, based on a modeling of the speaker conversation (E-HMM).

The various range of data proposed during these evaluations: telephone conversational speech, broadcast news shows, as well as meeting data (mono-, or multi-channel signals), allows the LIA and CLIPS labs to evaluate their own strategy through different environments and to propose further evolution to improve each of them. Besides, the strong collaboration between the LIA and CLIPS labs allows to design some fusion strategy based systems, especially on the broadcast news data, involving both the Step-By-Step and Integrated approaches. This overall work, in addition to improve speaker segmentation system performance, allows to better understand the behavior of each individual approaches and to analyze the advantages and drawbacks of each of them. Particularly, it has been highlighted that the power of the Integrated approach should be enforced when robust frontiers between speakers can be provided by a robust BIC based speaker turn detection strategy (first step of the Step-By-Step approach). Further investigation will be done in this way, in order to really integrate both of them in a same system. Indeed, a future project launched by the LIA, CLIPS, and LIUM labs will be to develop a new speaker segmentation system, based on the free speaker recognition ALIZE toolkit [START_REF]Alize toolkit[END_REF], designed and developed in the framework of the ALIZE project, a part of the French research Ministry Technolangue program [START_REF]French minister technolangue project[END_REF].

  speaker change detector (Step one) followed by a hierarchical clustering (Step two). A BIC curve is extracted by computing a distance between two 1.75s adjacent windows that go along the signal. Mono-Gaussian models with diagonal covariance matrices are used to model the two windows. A threshold is then applied on the BIC curve to find the most likely speaker change points which correspond to the local maximums of the curve. Clustering starts first by training a 32 component GMM background model (with diagonal covariance matrices) on the entire test file maximizing a ML criterion thanks to a classical EM algorithm. Segment models are then trained using MAP adaptation of the background model (means only). Next, BIC distances are computed between segment models and the closest segments are merged at each step of the algorithm until N segments are left (corresponding to the N speakers in the conversation).

Fig. 1 .

 1 Fig. 1. Hierarchical acoustic segmentation.

  Example of the piped strategy.

		LIA Re-Segmentation
	CLIPS segmentation	t	t
	Fig. 2. t
	t	Label Merge	LIA Re-Segmentation
	t	New Segmentation
	t

t Fig. 3. Example of the merging strategy.

Table 1

 1 presents a summary of the LIA and CLIPS (ELISA consortium) results obtained since 2000 on the different kinds of data: telephone speech conversations, broadcast news documents, and meeting room recordings (over various con-

ditions: head microphone, or (distant) table microphone/mono-, or multi-channel signals).

The number of parameters may differ according to the task context.

the first one being directly linked to the intrinsic constraint of the task: no prior information about speakers involved in the audio documents.


Claude Barras
email: barras@limsi.fr

Xuan Zhu

Sylvain Meignier
email: meignier@limsi.fr

Jean-Luc Gauvain
email: gauvain@limsi.fr

IMPROVING SPEAKER DIARIZATION

This paper describes the LIMSI speaker diarization system used in the RT-04F evaluation. The RT-04F system builds upon the LIMSI baseline data partitioner, which is used in the broadcast news transcription system. This partitioner provides a high cluster purity but has a tendency to split the data from a speaker into several clusters when there is a large quantity of data for the speaker. In the RT-03S evaluation the baseline partitioner had a 24.5% diarization error rate. Several improvements to the baseline diarization system have been made. A standard Bayesian information criterion (BIC) agglomerative clustering has been integrated replacing the iterative Gaussian mixture model (GMM) clustering; a local BIC criterion is used for comparing single Gaussians with full covariance matrices. A second clustering stage has been added, making use of a speaker identification method: maximum a posteriori adaptation of a reference GMM with 128 Gaussians. A final post-processing stage refines the segment boundaries using the output of the transcription system. Compared to the best configuration baseline system for this task, the improved system reduces the speaker error time by over 75% on the development data. On evaluation data, a 8.5% overall diarization error rate was obtained, a 60% reduction in error compared to the baseline.

INTRODUCTION

Acoustic diarization is the process of partitioning an input audio stream into acoustically homogeneous segments according to the speaker identity and the background and channel conditions. Speaker diarization is a useful preprocessing step for an automatic speech transcription system. By separating out speech and nonspeech segments, the recognizer only needs to process audio segments containing speech, thus reducing the computation time. By clustering segments of the same acoustic nature, condition specific models can be used to improve the recognition performance. By clustering segments from the same speaker, the amount of data available for unsupervised speaker adaptation is increased, which can significantly improve the performance of the transcription system. Speaker diarization can also improve readability of an automatic transcription by structuring the audio stream into speaker turns and in some cases by providing the identity of the speakers, Such information can also be of interest for the indexation of multimedia documents.

There are two predominant approaches to the speaker diarization problem. In most situations the number of speakers and the speaker characteristics are unknown a priori, and need to be automatically determined. The first approach relies on a two step procedure [START_REF] Chen | Speaker, Environment and Channel Change Detection and Clustering via the Bayesian Information Criterion[END_REF][START_REF] Hain | Segment generation and clustering in the HTK broadcast news transcription system[END_REF][START_REF] Siegler | Automatic Segmentation, Classification, and Clustering of Broadcast News Audio[END_REF]. First is the segmentation step, which locates seg- § now with Laboratoire d'Informatique de l'Universite du Maine ment boundaries based on acoustic changes in the signal. Second is the clustering step, which regroups segments coming from the same speaker into a clusters. A limitation of this method is that errors made in the segmentation step are not only difficult to correct later, but can also degrade the performance of the subsequent clustering step.

An alternative is to optimize jointly the segmentation and the clustering, via, for example, an iterative segmentation and clustering procedure as described in [START_REF] Gauvain | Partitioning and Transcription of Broadcast News Data[END_REF] which uses a set of Gaussian Mixture Models (GMMs). An iterative method based on an ergodic hidden Markov model (HMM) is also proposed in [START_REF] Ajmera | A robust speaker clustering algorithm[END_REF].

The remainder of paper is organized as follows: Section 2 briefly reviews the speaker diarization task and experimental conditions. Section 3 describes the baseline partitioning system, and Section 4 describes the BIC clustering and speaker ID clustering used to improve the partitioning system. The experimental results are presented in Section 5 followed by some conclusions.

EXPERIMENTAL SETUP

The experimental setup followed the RT-04F evaluation plan [START_REF]Fall 2004 Rich Transcription (RT-04f) Evaluation Plan[END_REF].

Here we briefly describe the task, the performance measures and the development and evaluation corpora.

Task

Diarization in RT-04F consisted of the "Who spoke when" speaker segmentation task, including gender classification (adult male, adult female, child). It was to be performed on English Broadcast News datasets only.

The "Who spoke when" task requires a system to identify all regions of time produced from the same speaker. Unlike the speaker identification or tracking tasks where a priori knowledge of the speaker voices is provided and an absolute identification is required, the speaker diarization task is relative to a given show, and thus only a relative, show-internal speaker identification is output by the system.

Performance measures

The speaker diarization task performance is measured via an optimum mapping between the (absolute) reference speaker IDs and the (relative) hypotheses. The primary metric for the task is the fraction of speaker time that is not attributed to the correct speaker, given the optimum speaker mapping. It is a time-based measure, computed on non-overlapping speech segments. Small pauses of less than 0.5 seconds are not considered to be segmentation breaks. Also, a time collar of 0.25 seconds is allowed at each speaker transition in the reference in order to take into account possible errors in the reference timing due to the automatic forced-alignment.

The overall speaker diarization error includes the missed and false alarm speaker times, thus taking speech/non-speech detection errors into account. Speech activity detection (SAD) is of a different nature than speaker clustering, and it is of interest to provide separate error measures for SAD and speaker clustering, when analyzing a system's performance.

Other performance measures can provide better insight into the speaker clustering stage of the system than the single, global measure provided by the speaker error time. For some experiments, we report the average frame-level cluster purity and cluster coverage measures [START_REF] Gauvain | Partitioning and Transcription of Broadcast News Data[END_REF]. Similar to segment-level cluster purity proposed in [START_REF] Chen | Speaker, Environment and Channel Change Detection and Clustering via the Bayesian Information Criterion[END_REF], frame-level cluster purity is defined as the ratio between the number of frames by the dominating speaker in a cluster and the total number of frames in the cluster. Cluster coverage is the dual measure, and accounts for the dispersion of a given speaker's data across clusters. Both measures are complementary, and the speaker error time can be interpreted as a combination of both.

Databases

Speaker diarization was assessed on the the Broadcast News transcription task for the English language. Development and training databases were provided by NIST along with a reference labeling determined by the LDC for system development. Evaluation references were made available after the evaluation. The data were taken from US radio or TV shows. The training database is a subset of the LDC 1997 English Broadcast News Hub-4 corpus, completed with precise structural metadata annotations. We report some experimental results on this database; however we did not use it directly for model training. We used the entire 1996 Hub-4 (LDC97S44) and 1997 Hub-4 corpora for building several acoustic models, as detailed in the system description. The combined corpora have a total of about 150 hours of annotated audio data.

BASELINE PARTITIONING SYSTEM

Our baseline data partitioning system is the first stage of the system developed of the LIMSI English broadcast news transcription system [START_REF] Gauvain | Partitioning and Transcription of Broadcast News Data[END_REF]. It was shown to provide a high cluster purity (about 96%) and a cluster coverage slightly below 80% on 1996 and 1997 NIST evaluation data. The baseline partitioner is structured as follows (cf. Figure 1):

• Feature extraction: Mel frequency cepstral parameters are extracted from the speech signal every 10ms using a 30ms window. The 38 dimensional feature vector consists of 12 cepstrum coefficients, 12 delta and 12 delta-delta coefficients plus the delta and delta-delta energy. It is similar to the features used for speech transcription, except that the energy coefficient is discarded.

• Speech Activity Detection: Speech is extracted from the signal with a Viterbi decoding using Gaussian Mixture Models (GMM) for speech, speech over music, music, silence and noise. The GMMs, each with 64 Gaussians, were trained on about 1 hour of acoustic data extracted from the 1996/1997 Broadcast News data.

• Chopping into small segments: Segmentation of the signal is performed by taking the maxima of a local Gaussian divergence measure between two adjacent sliding windows of 0.5 seconds, similar to [START_REF] Siegler | Automatic Segmentation, Classification, and Clustering of Broadcast News Audio[END_REF]. A single diagonal Gaussian is used for each window. The detection threshold was set on training data in order to provide small acoustically homogeneous segments lasting at least 0.25 seconds.

• Iterative GMM segmentation/clustering procedure: Each initial segment is used to seed one cluster, and a GMM with 8 Gaussians and a diagonal covariance matrix is trained by maximum likelihood estimation (MLE) on the segment data. Then, given a sequence of N non-overlapping segments (s 1 , . . . , s N ) with their associated segment cluster labels (c 1 , . . . , c N ), where c i 2 [1, K] and K ∑ N , the objective function used is a penalized log-likelihood of the form:

N X i=1 log f (s i |M c i ) °AEN °ØK
where f (•|M ) is the likelihood given the model M , and AE > 0 and Ø > 0. The terms AEN and ØK can be seen as segment and cluster penalties. The algorithm alternates Viterbi resegmentation and GMMs reestimation steps, where

P i log f (s i |M c i )
°AEN is maximized, with GMM clustering steps, as long as the resulting log-likelihood loss per merge is less than Ø. The merging criterion between two GMMs is estimated as the log-likelihood loss for merging the 16 initial Gaussians of both GMMs into a final set of 8 Gaussians (cf. [START_REF] Gauvain | Partitioning and Transcription of Broadcast News Data[END_REF]).

• Viterbi resegmentation: The segment boundaries are refined using the last set of GMMs and an additional relative energybased boundary, within a 1 second interval. This is done to locate the segment boundaries at silence portions, so as to avoid cutting words.

• Bandwidth and gender labeling: Band (studio or telephone) and gender (male or female) labeling is performed on the segments using 4 GMMs with 64 diagonal covariance matrices, trained on a subset of the 1996/1997 Broadcast News data.

IMPROVED SPEAKER PARTITIONING

The baseline partitioning system (c-std) results submitted in the RT-03S speaker diarization task, had an overall diarization error rate of 24.5%. Other approaches, e.g. BIC clustering methods, had a better performance on this task [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF]. We therefore tested a modified system, replacing the iterative GMM clustering with BIC-based clustering (cf. Figure 1, (c-bic)). We also pipelined the output of the system into a second clustering stage which uses a speaker identification module (c-sid). Finally, a SAD post-filtering stage was added to taking into account short pauses. The other parts of the system were kept unchanged.

BIC clustering

A hierarchical clustering is applied to the segments output by the iterative GMM segmentation. At the beginning, each segment seeds one cluster, modeled by a single Gaussian with a full covariance matrix. At each step, the two nearest clusters are merged until the stop criterion is reached. The BIC criterion [START_REF] Chen | Speaker, Environment and Channel Change Detection and Clustering via the Bayesian Information Criterion[END_REF] is used both for the inter-cluster distance measure and the stop criterion. The BIC penalty weight was optimized on the dev1 and dev2 data.

In order to decide whether to merge two clusters ci and cj, the ∆BIC value is computed as:

∆BIC = (ni + nj) log |Σ| °ni log |Σi| °nj log |Σj| °∏P
where Σ is the covariance matrix of the merged cluster (ci and cj), Σ i of cluster c i , Σ j of cluster c j , and n i and n j are respectively the number of the acoustic frames in cluster c i and c j . The penalty P is:

P = 1 2 (d + 1 2 d(d + 1
)) log n where d is the dimension of the feature vector space. The merging criterion is that two clusters should be merged if ∆BIC < 0. At each step, the two nearest clusters (i.e., those which have the most negative ∆BIC values) are merged into one cluster, and the ∆BIC value between the new cluster and all the other clusters is computed. The clustering procedure terminates when ∆BIC > 0.

In our BIC clustering procedure, the size of the two merged clusters, i.e. n = n i +n j , is used in the penalty P for the BIC criterion, as described in [START_REF] Cettolo | Segmentation, Classification and Clustering of an Italian Broadcast News Corpus[END_REF]. We refer to this as a local BIC penalty. But in general the size of the whole set of cluster, i.e. n = P N k=1 n k has to be used in the penalty, which we refer to as a global BIC penalty. Since we use the BIC criterion as the distance measure for merging the clusters, using the total size will make the penalty constant, so the choice of the two merged clusters is decided just by the increase in likelihood for the global BIC case. The local BIC thus seems to be a better choice for a merging criterion, even if it is not optimal as a stop criterion.

SID clustering

Speaker clustering methods performed by either the iterative GMM or the BIC agglomerative clustering procedures have to deal in the beginning of the process with short duration segments, and thus use a limited set of parameters per cluster: a GMM with 8 diagonal components for the former, and a single Gaussian with full covariance matrix for the latter. After several iterations, the amount of data per cluster increases, and a more complex model can be used. Also, purely acoustic clustering tends to split a speaker's data into several clusters as a function of the various background conditions (clean speech, speech with noise, speech with music...). Acoustic background normalization is necessary to regroup the data for a given speaker.

A state-of-the-art speaker recognition methods [START_REF] Reynolds | Speaker verification using adapted gaussian mixture models[END_REF][START_REF] Barras | Feature and score normalization for speaker verification of cellular data[END_REF] were thus employed to improve the quality of the speaker clustering. The first clustering stage is tuned to provide the highest possible cluster purity, not the lowest speaker error, since wrong merges can not be canceled by a further agglomerative process. The SID clustering process is as follows:

• Front-end: The feature vectors consist of 15 Mel frequency cepstral coefficients plus delta coefficients and delta energy.

Feature warping [START_REF] Pelecanos | Feature warping for robust speaker verification[END_REF] is performed on each segment using a sliding window of about 3 seconds in order to reduce the effect of the acoustic environment.

• Models: For each gender (male, female) and each channel condition (studio, telephone) combination, a Universal Background Model (UBM) with 128 diagonal Gaussians is trained on the 1996/1997 Broadcast News data. For each initial cluster, maximum a posteriori (MAP) adaptation [START_REF] Gauvain | Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains[END_REF] of the means of the matching UBM is performed.

• Clustering: Agglomerative clustering is performed separately for each gender and band condition, using a cross loglikelihood ratio as in [START_REF] Reynolds | Blind clustering of speech utterances based on speaker and language characteristics[END_REF]. For each cluster ci, its model Mi is MAP adapted from the gender and channel matched UBM R using the set of feature vectors x i belonging to the cluster. Then, given two clusters c i and c j , their cross log-likelihood ratio is defined as:

clr(ci, cj) = log f 0 (x i |M j ) f 0 (xi|R) + log f 0 (x j |M i ) f 0 (xj|R) where f 0 (•|M )
is the likelihood of the acoustic frames given the model M , normalized by the length of the signal. This is a symmetric similarity measure. After each merge, a new model is trained for the cluster ci[j. The clustering stops when the cross log-likelihood ratio between all clusters is below a given threshold ± estimated on the development data sets.

SAD post-filtering

The output of the LIMSI Broadcast News Speech-To-Text system is used in a post-processing stage for filtering out shortduration silence segments not detected by the initial speech detection. Only inter-word silences lasting at least 1 second are filtered out. This duration was chosen to be the sum of the minimal 0.5 sec inter-segment gap plus two collars, and its relevance was verified on development data.

EXPERIMENTAL RESULTS

Several configurations were tested for the systems. By default, the configuration used is the one that provided the best results on development data, i.e. AE = Ø = 230 for c-std, ∏ = 5.5 for c-bic and ∏ = 3.5, ± = 0.1 for c-sid and p-asr. A local BIC merging and stop criterion was also used.

Results on the development data

As expected, the standard partitioner c-std in its default configuration provides a high purity, and but a relatively poor coverage, resulting in a high overall diarization error over 30% on dev1 data (cf. Table 1). Setting the penalty AE and Ø to optimize these values reduces this error below 25%. The c-bic system also provides a high purity, with much better coverage (resp. 97% and 90%), reducing the overall error rate by almost 50%. The c-sid system obtains a large increase of the coverage without degradation of the purity, resulting in a global error rate about 7%, a reduction of almost 50% compared to c-bic.

A global BIC merging and stop criterion was also tested, but always performed worse than the local BIC criterion in our experiments, as can be seen for c-bic on dev1 (cf. Table 2), thus only the local criterion was used in the remaining experiments.

Looking in more detail at the performance of c-sid system, we can see that the speech detection error rate is 1.7% for dev1 and 3.6% for dev2 (cf. Table 3). The speaker clustering error is 5.4% for dev1 and 4.1% for dev2; but this average value hides a large variation across shows, ranging from the lowest error of 0.1% for the C-SPAN show to over 12% for the ABC and NBC shows.

Results on the training data

On the training corpus, the c-sid system has a much higher overall speaker diarization error: 16.1% compared to 7.1% and 7.6% on dev1 and dev2 respectively. The setting of the SID clustering threshold ± (0.1) on the development data is not optimal for the training corpus (cf. Table 4): an optimal value ± = 0.5 provides a 40% relative reduction of the speaker error rate, from 12.9% to 7.8%. A possible reason is the variability observed in the duration of the shows of the training set, between 30 minutes and 2 hours. By restricting the training set to the subset of shows with a matching duration (30 minutes), the speaker error rate and overall error in the standard configuration are 9.7% and 12.3% respectively, which shows that the SID clustering threshold ± is dependent on the show duration.

Results on the evaluation data

On the evaluation test set, the trend observed on the development data was confirmed, with a slight increase to 17% overall diarization error for c-bic and 9.1% for c-sid. The final SAD postprocessing stage gives an improvement of 0.6%, mainly by reducing false alarms in speech detection.

CONCLUSIONS

The baseline partitioning system provides a high cluster accuracy, but may split data from a single speaker into several clusters according to the background acoustic conditions. This behavior is desirable as a preprocessing stage of a speech transcription system, where unsupervised adaptation of the acoustic models is performed using the clustering output. The lower cluster coverage is not an issue, and has only a small impact on the quality of the transcription. On the other hand, the speaker diarization task gives equal value to cluster purity and coverage, which led us to improve upon the baseline partitioner.

We have thus explored several modifications to the baseline system. First, the iterative GMM clustering has been replaced by an agglomerative BIC clustering, using mono Gaussians with full covariance matrices. A local BIC merging and stop criterion was shown to outperform the global criterion which would be more in agreement with the theory. This result remains to be further interpreted but may be due to an inadequacy between the BIC modelization and the real distribution of the data. A similar result was observed in [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF]. A second clustering module has been applied to the output of the system, relying on techniques used for speaker identification and verification: acoustic channel normalization, and MAP adaptation of a reference GMM with a large number of Gaussians.

On the development data, the overall speaker diarization error was reduced from 24.8% for the best setting of the baseline system to 13.2% using BIC clustering, and to 7.1% with the additional SID clustering step. These figures include the speech/non-speech detection errors, which remain constant at about 1.7%. This corresponds to a relative speaker error time reduction of over 75%. Consistent, but somewhat higher overall error rates are observed on the evaluation data: 17% for c-bic and 9.1% for c-sid. This dramatic improvement over the baseline system results from several changes: a model complexity which increases with the average amount of speech data per cluster, and the combination of two different systems and models, each one focusing on a different acoustic aspect. The final post-processing filtering using the ASR output provides a further reduction of the overall error rate from 9.1% to 8.5% on the evaluation data, mainly due to a reduction of false alarm speaker time on long pauses.

Several issues remain to be investigated in order to improve the robustness and the efficiency of the system. It was observed that the clustering threshold needs to be set according to the length of the audio document, and that the system still has a large variability across individual shows. However the speaker error does not provide a stable and continuous measure of a clustering system: splitting a speaker in two classes, which is a single decision, results in doubling of the error rate for this speaker. Only with a large amount of files can statistically consistent results be obtained. No specific optimizations were made for speed in the systems described, and for a 30 minutes show, c-bic system speed is between 0.2 and 0.3xRT, and c-sid speed is between 0.5 and 1.5xRT on a single 2.4GHz CPU. Finally, most speaker diarization systems rely on a purely acoustic segmentation and clustering. An essential part of the information in speech is of a linguistic nature, and obviously in TV and radio shows most speakers are presented and identified. Combining the acoustic with the linguistic layer as explored in [START_REF] Canseco-Rodriguez | Speaker diarization from speech transcripts[END_REF] would clearly improve the robustness of a speaker diarization system and make it more exploitable by a human reader.

Figure 1 :

 1 Figure 1: Standard LIMSI partitioning system (c-std on the left side of the diagram) and speaker partitioning system improved for RT-04F (p-asr to the right, along with c-bic and c-sid intermediate steps).

  data

Table 1 :

 1 The cluster purity, cluster coverage and the overall diarization error from the systems c-std (both in initial configuration and best configuration), c-bic and c-sid on dev1 dataset.

	system	cluster		coverage	overall
		purity (%)	(%)	error (%)
	c-std (AE = Ø = 160)	95.0		71.6	32.3
	c-std (AE = Ø = 230)	90.6		82.1	24.8
	c-bic (∏ = 5.5)	97.1		90.2	13.2
	c-sid (∏ = 3.5, ± = 0.1)	97.9		95.8	7.1
	BIC	∏	overall
	criterion	error (%)
	local 5.0	13.32%
		6.0	12.77%
		7.0	13.78%
	global 5.0	16.39%
		6.0	15.46%
		7.0	18.22%

Table 2 :

 2 The overall diarization error for c-bic system on the dev1 database, as a function of the penalty weight ∏ for the local and global BIC merging and stop criterion.

Table 3 :

 3 Performance of c-sid system on the dev1 and dev2 data sets.

		missed	false alarm	speaker	overall
	set speech (%) speech (%) error (%) error (%)
	dev1	0.4	1.3	5.4	7.1
	ABC	1.6	1.3	12.4	15.2
	VOA	0.3	1.2	2.2	3.7
	PRI	0.1	0.9	2.8	3.8
	NBC	0.1	1.1	12.0	13.2
	CNN	0.5	1.4	5.6	7.6
	MNB	0.2	1.8	0.8	2.8
	dev2	0.5	3.1	4.1	7.6
	CSPAN	0.3	2.9	0.1	3.3
	CNN	0.6	4.2	5.0	9.8
	PBS	0.1	2.8	7.4	10.3
	ABC	2.1	6.7	12.5	21.2
	CNNHL	0.0	1.4	0.5	1.9
	CNBC	0.2	1.0	0.9	2.1
		±	speaker	overall
			error (%) error (%)
		0.1	12.9%	16.1
		0.2	12.5%	15.7
		0.3	9.0%	12.2
		0.4	8.9%	12.1
		0.5	7.8%	11.0
		0.6	8.1%	11.3
		0.7	8.6%	11.8

Table 4 :

 4 Speaker diarization error on the training corpus for c-sid system, as a function of the SID clustering threshold ±.

Table 5 :

 5 Performances of c-bic, c-sid and p-asr systems on the RT-04F evaluation data.

	system	missed	false alarm	speaker	overall
		speech (%) speech (%) error (%) error (%)
	c-bic	0.4	1.8	14.8	17.0
	c-sid	0.4	1.8	6.9	9.1
	p-asr	0.6	1.1	6.8	8.5