Max Jensen

Iain Smears

On the notion of boundary conditions in comparison principles for viscosity solutions

Keywords: Viscosity boundary conditions, comparison principles, Hamilton-Jacobi-Bellman equations, Monge-Ampère equations, Barles-Souganidis theorem AMS classification. 49L25, 65N99

We collect examples of boundary-value problems of Dirichlet and Dirichlet-Neumann type which we found instructive when designing and analysing numerical methods for fully nonlinear elliptic partial differential equations. In particular, our model problem is the Monge-Ampère equation, which is treated through its equivalent reformulation as a Hamilton-Jacobi-Bellman equation. Our examples illustrate how the different notions of boundary conditions appearing in the literature may admit different sets of viscosity sub-and supersolutions. We then discuss how these examples relate to the validity of comparison principles for these different notions of boundary conditions.

Introduction

In this short note we collect a small number of examples which we found instructive when designing and analysing numerical methods for fully nonlinear elliptic partial differential equations (PDE). In particular we are interested in the comparison principle between sub-and supersolutions, as used in the convergence proof by Barles and Souganidis [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF] for the approximation of viscosity solutions by monotone numerical schemes.

Our model problem is the following simple Monge-Ampère equation

M (D 2 u) = 0, M (A) := 1 2 f 2 -det A (1.1)
on a domain Ω ⊂ R d , d ≥ 2 with f ≥ 0. The problem is complemented with either Dirichlet or mixed Dirichlet-Neumann boundary conditions, as well as the requirement that u be a convex function. In order to conform to the standard framework of degenerate elliptic operators, we consider the following reformulation of (1.1) as a Hamilton-Jacobi-Bellman (HJB) equation [START_REF] Krylov | Nonlinear elliptic and parabolic equations of the second order[END_REF][START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF] H(D 2 u) = 0, H(A) := sup

B∈S 1 (-B : A + f √ det B), (1.2)
where S 1 is the set of symmetric positive semidefinite matrices in R d×d with trace equal to 1. In particular, it was shown in [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF] that (1.1) (including the convexity constraint) is equivalent to (1.2) in the sense of viscosity solutions.

Remark 1.1. Observe that the Barles-Souganidis theorem cannot be considered directly for (1.1) because (1.1) is only elliptic on the set of convex functions and its test functions are usually assumed to be convex [START_REF] Gutiérrez | The Monge-Ampère equation[END_REF]Definition 1.3.1]. This is the reason why we shall work with the equivalent formulation (1.2).

Comparison principles are central to the theory of viscosity solutions, both for the analysis of well-posedness of the PDE and for the analysis of numerical methods. While conceptually the statement of a comparison principle requires that subsolutions lie below supersolutions, the different formulations of the boundary conditions and the different sets of available test functions raise the question of the validity of the corresponding comparison principle. For instance, the boundary conditions can be imposed in the following variety of ways:

(i) In the classical sense, where the Dirichlet boundary condition is understood pointwise everywhere on the boundary; this is the setting for the comparison principle of Theorem 3.3 in the User's Guide [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF] by Crandall, Ishii and Lions.

(ii) As in the setting of the Barles-Souganidis theorem [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF], where the Dirichlet boundary condition is relaxed from its classical pointwise sense, and is understood in a generalised sense that allows extensions of the PDE onto the boundary. This notion of the boundary conditions is the subject of section 2 below. We remark that in the Barles-Souganidis theorem [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF], the comparison principle required for the analysis was stated as an assumption.

(iii) As in Definition 7.4 of the User's Guide [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF], where boundary conditions are relaxed similarly to the Barles-Souganidis approach, but semi-continuity of suband supersolutions is assumed from the outset and a closure operation is applied to the second-order jets. See also [START_REF] Barles | Exit time problems in optimal control and vanishing viscosity method[END_REF], where the semi-continuity for sub-and supersolutions of Hamilton-Jacobi equations is imposed, but the closure of the jets is not introduced. We also refer the reader to [3, Definition 7.1] on the intermediate notion of the boundary condition named therein as the strong viscosity sense. The sets of sub-and supersolutions are usually chosen within (i) the spaces USC(Ω) of bounded upper semi-continuous functions and LSC(Ω) of bounded lower semi-continuous functions, (ii) or within the function space C(Ω) of continuous functions, (iii) or, in the classical setting, within the function space C(Ω) ∩ C 2 (Ω) of twice continuously differentiable functions. Here, we shall focus our attention on the semi-continuous case because this is the relevant one for the analysis of numerical methods, where only the semi-continuity of upper and lower envelopes of sequences of numerical solutions is known a priori. Nevertheless, it is worth observing that the existence of a comparison principle may well be conditional to further regularity or structure assumptions on the set of suband supersolutions. We point to Section 7.C of [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF] for a general discussion of the subject. In this note we focus on the question of whether or not a comparison principle is available for the different notions of the boundary condition, without additional restrictions on the set of sub-and supersolutions. We take as a reference problem the simple Monge-Ampère equation and illustrate with examples how the different types of Dirichlet conditions impose a constraint on sub-and supersolutions. In turn this also informs us how a numerical convergence analysis may be approached.

While we consider in the subsequent text different notions of viscosity sub-and supersolutions, a function u is always said to be a viscosity solution if it is simultaneously a viscosity subsolution and supersolution.

Given a function v we denote its upper semi-continuous envelope by v * and its lower semi-continuous envelopes by v * , respectively. More precisely, for all x ∈ Ω,

v * (x) := sup {yn}n⊂Ω yn→x limsup n→∞ v(y n ), v * (x) := inf {yn}n⊂Ω yn→x liminf n→∞ v(y n ).

Dirichlet boundary conditions as in the Barles-Souganidis theorem

Let Ω be a open subset of R d and consider the model problem (1.2) with a homogeneous Dirichlet boundary condition u = 0 on ∂Ω. In line with Definition 1.1 and equations (1.8), (1.9) of [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF], we say that a locally bounded function v is a viscosity subsolution of the boundary value problem if

F * (D 2 φ(x), v * (x), x) ≤ 0 for all φ ∈ C 2 (Ω) such that v * -φ has a local maximum at x ∈ Ω,
where F * denotes the lower semicontinuous envelope of F defined by

F * (A, w, x) = H(A) : x ∈ Ω, min{H(A), w} : x ∈ ∂Ω.
Analogously, v is a viscosity supersolution whenever

F * (D 2 φ(x), v * (x), x) ≥ 0 (2.1)
for all φ ∈ C 2 (Ω) such that v * -φ has a local minimum at x ∈ Ω, where F * is the upper semicontinuous envelope of F given by

F * (A, w, x) = H(A) : x ∈ Ω, max{H(A), w} : x ∈ ∂Ω.
We consider in the following example the Monge-Ampère equation on possibly one of the simplest domains with a boundary, namely a d-dimensional half-space. In particular, let Ω = H d , with d ≥ 2, where

H d = {x = (x 1 , . . . , x d ) ∈ R d , x 1 > 0},
and consider the problem (1.2) with vanishing source term f = 0, corresponding to the degenerate elliptic case, complemented with homogeneous Dirichlet boundary conditions on

∂Ω = {x = (x 1 , . . . , x d ) ∈ R d , x 1 = 0}.
It is clear that the function u ≡ 0 is a viscosity solution of the problem in the sense of [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF]. However, we show below that uniqueness of the viscosity solution fails in this example.

Proposition 2.1. Let d ≥ 2 and let Ω = H d as above. For a fixed but arbitrary constant c > 0, let the locally bounded function v c be defined by

v c (x) = 0 if x ∈ Ω and v c (x) = -c if x ∈ ∂Ω. Then v c is a viscosity solution of (1.2) in the sense of [2]. Proof. It follows from the definition of v c that (v c ) * ≡ 0 identically in Ω, whereas (v c ) * = v in Ω since v c is lower semi-continuous. It is thus clear that v c is a viscosity subsolution of the problem.
We now prove that the function v c is also a viscosity supersolution and hence a viscosity solution of the problem in the sense of [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF]; in particular, we must show that v c is a viscosity supersolution, i.e. that (2.1) holds for all φ ∈ C 2 (Ω) such that (v c ) * -φ has a local minimum at x ∈ Ω. It is clear that (2.1) is satisfied whenever x ∈ Ω is an interior point, since v * ≡ 0 in Ω. Hence we need only to consider boundary points x ∈ ∂Ω. Suppose now that φ ∈ C 2 (Ω) is such that (v c ) * -φ has a local minimum at x ∈ ∂Ω. Then, since d ≥ 2, we may take a unit tangent vector y = (0, y 1 , . . . , y d-1 ) to the boundary, with |y| = 1, noting that for any ε ∈ R, x ± εy ∈ ∂Ω. Then, we deduce that, for ε > 0 sufficiently small,

φ(x + εy) -2φ(x) + φ(x -εy) ε 2 ≤ 0, (2.2)
where we have used the fact that

(v c ) * (x ± εy) -φ(x ± εy) ≥ (v c ) * (x) -φ(x)
whenever ε is small enough, and that that

(v c ) * (x ± εy) = (v c ) * (x) since (v c ) * ≡ -c
on ∂Ω. Therefore, taking the limit ε → 0, we deduce from (2.2) that the second-order directional derivative (y ⊗ y ) : D 2 φ(x) ≤ 0. Note that the matrix B y := y ⊗ y belongs to the set S 1 appearing in (1.2), since B y is positive semi-definite and has trace equal to |y| 2 = 1 (recall that y was chosen as a unit vector). Therefore, using the definition of H(D 2 φ(x)) from (1.2), we see that H(D 2 φ(x)) ≥ -B y : D 2 φ(x) ≥ 0, and hence

F * (D 2 φ(x), (v c ) * (x), x) = max{H(D 2 φ(x)), (v c ) * (x)} ≥ 0,
as required by (2.1). Hence v c is also a viscosity supersolution and thus a viscosity solution of (1.2).

Proposition 2.1 shows that in general, there may be infinitely many viscosity solutions for (1.1) and (1.2) with Dirichlet boundary conditions understood in the sense of [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF]. Therefore, by the equivalence of (1.1) and (1.2), in general there cannot be a comparison principle between sub-and supersolutions for the Monge-Ampère equation when the Dirichlet boundary conditions are understood in the sense of Barles-Souganidis, even on smooth convex domains! Remark 2.2. In Proposition 2.1, we considered negative perturbations on the boundary, i.e. v c (x) = -c, with c > 0. For the case of positive perturbations, i.e. v c = c, it is possible to construct test functions showing that the subsolution property does not hold.

Dirichlet boundary conditions as in the User's Guide

The definition of viscosity solution is formulated in a different way in the User's Guide [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]. There the gradient and Hessians obtained from the test functions define the jets J 2,+ u(x) := (Dφ(x), D 2 φ(x)) : φ ∈ C 2 and u -φ has local maximum at x , J 2,-u(x) := (Dφ(x), D 2 φ(x)) : φ ∈ C 2 and u -φ has local minimum at x . These jets may no be rich enough to replace the notion of the classical gradient and Hessian in the proof of a comparison principle in [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF], which is why one considers the closures

J 2,+ Ω u(x) := (p, X) ∈ R d × S : ∃ (x n , p n , X n ) ∈ Ω × R × S so that (p n , X n ) ∈ J 2,+ u(x n ) and (x n , u(x n ), p n , X n ) → (x, u(x), p, X) , J 2,- Ω u(x) := (p, X) ∈ R d × S : ∃ (x n , p n , X n ) ∈ Ω × R × S so that (p n , X n ) ∈ J 2,-u(x n ) and (x n , u(x n ), p n , X n ) → (x, u(x), p, X) ,
which 'inherit' nearby gradients and Hessians.

In line with Example 1.11, Definition 7.4 and equation (7.24) of [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF], we keep the above definitions of F , F * and F * . We say that a function v is a viscosity subsolution of the boundary value problem if u is upper semi-continuous on Ω and

F * (A, v(x), x) ≤ 0 ∀ (A, p) ∈ J 2,+ Ω v(x).
Similarly v is a viscosity supersolution whenever v is lower semi-continuous on Ω and

F * (A, v(x), x) ≥ 0 ∀ (A, p) ∈ J 2,- Ω v(x).
Consequently, there are two differences with the Barles-Souganidis definition:

(a) The equation is tested with a larger set of 'derivatives' as a result of the closure of the semi-jets.

(b) Both u and v are assumed to be semi-continuous, rather than taking their lower and upper semi-continuous envelopes.

The functions v c from Proposition 2.1, which are lower semi-continuous by definition, are not affected by the closure of the jets (a) in the sense that the above arguments from the previous section related to the supersolution property of v c remain valid without change.

However, the requirement of semi-continuity (b) means that now, the functions v c do not qualify as subsolutions, (and thus are not viscosity solutions) in the sense of [START_REF] Crandall | User's guide to viscosity solutions of second order partial differential equations[END_REF]. Nevertheless, since u ≡ 0 is a viscosity solution and hence is also a subsolution and yet u ≥ v c for all c > 0, we have found a subsolution that does not lie below the supersolution v c ; thus there is again no comparison principle between semi-continuous sub-and supersolutions. We note that there is no contradiction between our example and [3, Theorem 7.9], which asserts only a comparison principle between continuous sub-and supersolutions. However, recall that the case of semi-continuous sub-and supersolutions is the relevant one for the study of numerical approximations.

Dirichlet boundary conditions in the classical sense

As in [3, Definition 2.2] we now say that a function u is called a viscosity subsolution (resp. supersolution) if u ∈ USC(Ω) (resp. u ∈ LSC(Ω)) and if for all ϕ ∈ C 2 (Ω) such that u -ϕ has a local maximum (resp. minimum) at x ∈ Ω we have

F (D 2 ϕ(x), ∇ϕ(x), u(x), x) ≤ 0 (resp. F (D 2 ϕ(x), ∇ϕ(x), u(x), x) ≥ 0).
Lemma 8 in [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF], in the spirit of [3, Section 5.C], states that if u is a subsolution and v is a supersolution of (1.2) and crucially if u ≤ v on ∂Ω, then u ≤ v on Ω. Hence a viscosity solution that satisfies the boundary conditions in a pointwise sense is necessarily unique, if it exists. The general setting of paper [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF] is that of a bounded strictly convex domain Ω; however, neither boundedness nor convexity are used in the proof of Lemma 8 of [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF]. The existence and uniqueness of viscosity solutions holds with classical boundary conditions on strictly convex domains. Yet, on non-convex domains the Monge-Ampère problem is in general not well-posed.

Since the Barles-Souganidis theorem on the convergence of numerical approximations is also a proof of the existence of a unique viscosity solution, theorems of this type are therefore bound to fail for (1.2) on general non-convex domains. It is interesting to pinpoint the step at which the argument breaks down. Lemma 6.4 of [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF] shows how the upper and lower semi-continuous envelopes of the numerical solutions in the small-mesh limit satisfy the classical boundary conditions; this argument relies on the existence of certain test functions, for which the strict convexity of the domain is needed.

We shall therefore consider the scheme of [START_REF] Feng | Convergent semi-Lagrangian methods for the Monge-Ampère equation on unstructured grids[END_REF] for (1.2) on the L-shape domain Ω = (0, 1) × (-1, 1) ∪ (-1, 1) × (0, 1) , noting that the existence and uniqueness of numerical solutions also holds on nonconvex domains. A numerical solution is depicted in Figure 1 while Figure 2 shows the cross sections on {(x 1 , x 2 ) ∈ Ω : x 1 = x 2 } of the numerical solutions over several levels of refinement, where mesh 1 is the coarsest with 328 degrees of freedom while mesh 5 has 83968 DoFs. The figures illustrate how a mesh-dependent boundary layer appears in the vicinity of the re-entrant corner. Thus it is reasonable to expect that the lower semi-continuous envelope

u(x) := liminf y→x h→0 u h (y), ∀ x ∈ Ω,
of the sequence (u h ) h of numerical solutions will not satisfy the boundary conditions in the classical sense, so that the above mentioned comparison principle may not be used to guarantee existence of the viscosity solution. We now show some generalisations of the example of section 2 to problems with mixed boundary conditions on bounded convex domains in order to highlight some further subtleties and challenges of treating the boundary conditions in a generalised sense.

We therefore return to the definition of viscosity sub-and supersolutions of [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF], as detailed in section 2.

Consider the unit square domain Ω = (0, 1) 2 in two space dimensions, and consider the simple Monge-Ampère equation ( 1 note that Γ D and Γ N partition ∂Ω. To formalize the definition of the viscosity suband super-solutions, we define the operator B :

R d × R × ∂Ω → R by B(p, r, x) := r = 0 if x ∈ Γ D , p • n Γ N = 0 if x ∈ Γ N ,
where n Γ N is the unit outward normal on Γ N , which in this example is simply given by n Γ N = (0, 1) when x 2 = 1, and n Γ N (0, -1) when x 2 = 0. The lower and upper envelopes of B are given by

B * (p, r, x) := B(p, r, x) : x ∈ Γ D ∪ Γ N , min{r, p • n Γ N } : x ∈ ∂Ω \ (Γ D ∪ Γ N ) and B * (p, r, x) := B(p, r, x) : x ∈ Γ D ∪ Γ N , max{r, p • n Γ N } : x ∈ ∂Ω \ (Γ D ∪ Γ N ).
Following [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF] and [3, Section 7.B], a locally bounded function v is called a viscosity subsolution of the boundary value problem (5.1) if

F * (D 2 φ(x), ∇φ(x), v * (x), x) ≤ 0 for all φ ∈ C 2 (Ω) such that v * -φ has a local maximum at x ∈ Ω, where F * is defined by F * (A, p, w, x) = H(A) : x ∈ Ω, min{H(A), B * (p, w, x)} : x ∈ ∂Ω.
Analogously, v is a viscosity supersolution of (5.1) whenever

F * (D 2 φ(x), v * (x), x) ≥ 0 (5.2)
for all φ ∈ C 2 (Ω) such that v * -φ has a local minimum at x ∈ Ω, where F * is given by

F * (A, w, x) = H(A) : x ∈ Ω, max{H(A), B * (p, w, x)} : x ∈ ∂Ω.
It is clear that the function u ≡ 0 is a viscosity solution of the boundary value problem (5.1). However, we show in Proposition 5.1 below that again uniqueness of the viscosity solution fails due to the lack of a comparison principle. Proposition 5.1. For a fixed but arbitrary constant c > 0, let the locally bounded function v c be defined by v c = 0 on Ω ∪ Γ N and v c = -c on Γ D . Then v c is a viscosity solution of (5.1).

Proof. The upper envelope (v c ) * ≡ 0 in Ω, so we see that v c is a subsolution. To show the supersolution property, consider a function φ ∈ C 2 (Ω) such that (v c ) * -φ has a local minimum at x ∈ Ω. First, it is clear that (5.2) holds for whenever x ∈ Ω is an interior point or when x ∈ Γ N is a 'Neumann' boundary point. It remains only to consider 'Dirichlet' points x ∈ Γ D and corner points x ∈ ∂Ω \ (Γ N ∪ Γ D ).

If x ∈ Γ D is a 'Dirichlet' point, i.e. x = (x 1 , x 2 ) with x 1 ∈ {0, 1} and x 2 ∈ (0, 1), then we can follow the same argument used in the proof of Proposition 2.1 to deduce that ∂ 2

x 2 x 2 φ(x) ≤ 0 and hence that H(D 2 φ(x)) ≥ 0. This implies that (5.2) holds whenever x ∈ Γ D .

The only remaining case is when x is a corner point, i.e. x = ∂Ω \ (Γ N ∪ Γ D ). For this case, we note that for ε > 0 sufficiently small, x -εn Γ N ∈ Γ D since n Γ N = ±(0, 1) is the outward normal for the 'Neumann' part of the boundary. Therefore, we deduce that, for all ε > 0 sufficiently small, φ(x) -φ(x -εn Γ N ) ε ≥ 0, (

where we have used the facts that (v c ) * (x -εn Γ N ) -φ(x -εn Γ N ) ≥ (v c ) * (x)φ(x) for ε > 0 sufficiently small and that that (v c ) * (x -εn Γ N ) = v c (x) = -c. Therefore, taking the limit ε → 0 in (5.3) gives ∇φ(x) • n Γ N ≥ 0, and hence B * (∇φ(x), (v c ) * (x), x) = max{∇φ(x) • n Γ N , (v c ) * (x)} ≥ 0. Thus we find that (5.2) is satisfied in the case where x is a corner point. Hence v c is also a viscosity supersolution and thus a viscosity solution of (5.1).

The conclusion from Proposition 5.1 is that uniqueness again fails when considering the boundary conditions in the generalised sense as in [START_REF] Barles | Convergence of approximation schemes for fully nonlinear second order equations[END_REF].

Figure 1 .

 1 Figure 1. Numerical solution of (1.2) on an L-shaped domain with homogeneous boundary conditions and f ≡ 1.

Figure 2 .

 2 Figure 2. Cross sections of the numerical solution along the first diagonal x 1 = x 2 .

Figure 3 .

 3 Figure 3. Construction of the viscosity solutions v c in Proposition 5.1.

Author information

Max Jensen, Department of Mathematics, University of Sussex, Brighton BN1 9QF, England. E-mail: m.jensen@sussex.ac.uk Iain Smears, INRIA Paris, Paris 75589 & Université Paris-Est, CERMICS (ENPC), 77455 Marne-la-Vallée, France. E-mail: iain.smears@inria.fr


Claude Barras
email: barras@limsi.fr

Xuan Zhu

Sylvain Meignier
email: sylvain.meignier@lium.univ-lemans.fr

Member, IEEE Jean-Luc Gauvain
email: gauvain@limsi.fr

MULTI-STAGE SPEAKER DIARIZATION OF BROADCAST NEWS

Keywords: Speaker diarization, speaker identification, speaker segmentation and clustering, BIC clustering

This paper describes recent advances in speaker diarization with a multi-stage segmentation and clustering system, which incorporates a speaker identification step. This system builds upon the baseline audio partitioner used in the LIMSI broadcast news transcription system. The baseline partitioner provides a high cluster purity, but has a tendency to split data from speakers with a large quantity of data into several segment clusters. Several improvements to the baseline system have been made. First, the iterative Gaussian mixture model (GMM) clustering has been replaced by a Bayesian information criterion (BIC) agglomerative clustering. Second an additional clustering stage has been added, using a GMM-based speaker identification method. Finally a post-processing stage refines the segment boundaries using the output of a transcription system. On the NIST RT-04F and ESTER evaluation data, the multi-stage system reduces the speaker error by over 70% relative to the baseline system, and gives between 40% and 50% reduction relative to a single-stage BIC clustering system.

I. INTRODUCTION

Speaker diarization, also called speaker segmentation and clustering, is the process of partitioning an input audio stream into homogeneous segments according to speaker identity. It is one aspect of audio diarization, along with categorization of music, background noise and channel conditions. Speaker diarization can improve the readability of an automatic transcription by structuring the audio stream into speaker turns and in some cases by providing the true speaker identity. Such information can also be of interest for the indexing of multimedia documents. As defined by NIST for the 2004 Rich Transcription evaluation [START_REF]Fall 2004 Rich Transcription (RT-04F) evaluation plan[END_REF], the speaker diarization task is relative to a given show and no a priori knowledge of the speaker's voice or even of the number of speakers is available. Therefore only a relative, show-internal speaker identification is produced by the diarization system. This definition has been adopted in this work, even though a speaker diarization system could obviously make use of such information if combined with a speaker identification or tracking system. Audio diarization is a useful preprocessing step for an automatic speech transcription system. By separating out speech and non-speech segments, the recognizer only needs to process audio segments containing speech, thus reducing the computation time and avoiding word insertions in these portions. By clustering segments of the same acoustic nature, condition specific models can be used to improve the quality of the transcription. By clustering segments from the same speaker, the amount of data available for unsupervised speaker adaptation is increased, which can significantly improve the transcription performance.

Automatic speech transcription and speaker diarization rely on similar methods for segmentation and clustering. However differences in their objectives leads to different needs, particularly concerning where accuracy is most important. Automatic transcription requires accurate segment boundaries. Although the rejection of non-speech segments is useful in order to minimize insertion of words and to save computation time, it is important that the segment boundaries are located in non-informative zones such as silences or breaths. Indeed, having a word cut by a boundary disturbs the transcription process and increases the word error rate.

Diarization also aims to produce homogeneous speech segments; however, the main objectives are the purity and the correct labeling of the segments. Errors such as having more than one cluster for a given speaker, or conversely, merging the segments of two different speakers into one cluster, are penalized more heavily.

The effects of both boundary inaccuracy and mislabeled segments were measured in [START_REF] Tranter | An investigation into the interactions between speaker diarisation systems and automatic speech transcription[END_REF] for English broadcast news. The experiments showed that segment boundary errors have a greater impact on the transcription task, while label errors have a greater impact on the diarization task.

The NIST Rich Transcription evaluation has been the major evaluation for speaker diarization of broadcast news data in 2003 and 2004 [START_REF]Fall 2004 Rich Transcription (RT-04F) evaluation plan[END_REF], [START_REF]The Rich Transcription Spring 2003 (RT-03S) evaluation plan[END_REF], [START_REF]Spring 2004 (RT-04S) Rich Transcription meeting recognition evaluation plan[END_REF]. In 2005, the Technolangue ESTER evaluation was conducted on a similar task using French radio broadcast news data [START_REF] Gravier | The ESTER evaluation campaign of rich transcription of French broadcast news[END_REF], [START_REF] Galliano | The ESTER phase II evaluation campaign for the rich transcription of French broadcast news[END_REF].

Most speaker diarization systems for BN data have a similar general architecture. First the signal is chopped into homogeneous segments. The segment boundaries are located by finding acoustic changes in the signal and each segment is expected to contain speech from only one speaker. The resulting segments are then clustered so that each cluster corresponds to one speaker, a major issue being that the number of speakers is unknown a priori and needs to be automatically determined. Each system also presents specific aspects which can be classified following different criteria:

• Link between segmentation and clustering: segmentation can be done first, followed by clustering with no connection between the two parts [START_REF] Moh | Towards domain independent speaker clustering[END_REF], [START_REF] Ben | Speaker diarization using bottom-up clustering based on a parameter-derived distance between GMMs[END_REF] inspired from the work presented in [START_REF] Siegler | Automatic segmentation and clustering of broadcast news audio[END_REF]- [START_REF] Hain | Segmentation and classification of broadcast news audio[END_REF]; alternatively the segmentation and clustering can be jointly optimized, via, for example, the iterative segmentation and clustering procedures described in [START_REF] Gauvain | Partitioning and transcription of broadcast news data[END_REF]- [START_REF] Ajmera | A robust speaker clustering algorithm[END_REF]. A limitation of the first method is that errors made in the segmentation step are not only difficult to correct later, but can also degrade the performance of the subsequent clustering step.

• Clustering strategy: it relies either on an agglomerative clustering [START_REF] Moh | Towards domain independent speaker clustering[END_REF], [START_REF] Gauvain | Partitioning and transcription of broadcast news data[END_REF], [START_REF] Ajmera | A robust speaker clustering algorithm[END_REF], or on a divisive clustering method [START_REF] Meignier | E-HMM approach for learning and adapting sound models for speaker indexing[END_REF], [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF].

• Modeling strategy: each speaker can be modeled by a Gaussian Mixture Model (GMM) with diagonal covariance matrices composed of 8 to 64 components. As is done in the speaker recognition task, larger models with 2048 components have been proposed [START_REF] Ben | Speaker diarization using bottom-up clustering based on a parameter-derived distance between GMMs[END_REF], [START_REF] Meignier | E-HMM approach for learning and adapting sound models for speaker indexing[END_REF], [START_REF] Ajmera | A robust speaker clustering algorithm[END_REF]. In this case, a more robust estimation of the models despite the limited amount of data per speaker can be obtained by performing the maximum a posteriori (MAP) adaptation of a prior model [START_REF] Gauvain | Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains[END_REF]. On the other hand, using a single Gaussian with a full covariance matrix for the modeling of a speaker also provides good results [START_REF] Moh | Towards domain independent speaker clustering[END_REF].

In our experiments several variants and combinations of systems have been tested, in particular to study the link between segmentation and clustering and the modeling strategy.

The remainder of this paper is organized as follows:

Section II describes the baseline partitioning system which was developed for the automatic Broadcast New transcription task. Section III describes the multi-stage partitioning system specifically built for the speaker diarization task. This system is based upon a Bayesian information criterion (BIC) clustering followed by speaker identification (SID) clustering. Experimental results are presented in Section IV, followed by some conclusions.

II. BASELINE PARTITIONING SYSTEM

The baseline audio partitioning system was developed as a preprocessing step for the LIMSI English broadcast news transcription system [START_REF] Gauvain | Partitioning and transcription of broadcast news data[END_REF], [START_REF] Gauvain | Audio partitioning and transcription for broadcast data indexation[END_REF]. It was shown to provide a high cluster purity (about 96%) and a cluster coverage slightly below 80% on 1996 and 1997 NIST evaluation data. This baseline partitioner c-std shown in Figure 1 is structured as follows:

A. Feature extraction Mel frequency cepstral parameters are extracted from the speech signal every 10 ms using a 30 ms window on a 0-8kHz band. For each frame the Mel scale power spectrum is computed, and the cubic root taken followed by an inverse Fourier transform. Using a process similar to that of PLP computation [START_REF] Hermansky | Perceptual linear predictive (PLP) analysis of speech[END_REF], 12 LPC-based cepstral coefficients are then extracted. The 38 dimensional feature vector consists of 12 cepstral coefficients, ∆ and ∆-∆ coefficients plus the ∆ and ∆-∆ log-energy. This is essentially the same set of features that is used in a standard transcription system, except for the energy [START_REF]The LIMSI broadcast news transcription system[END_REF]. This set is used in all steps of the c-std system, except for the segmentation into small segments where only the static features are used. No cepstral mean or variance normalization is performed to the acoustic vector in the baseline partitioning system.

B. Speech Activity Detection (SAD)

Speech is extracted from the signal with a Viterbi decoding using Gaussian Mixture Models (GMM) for speech, noisy speech, speech over music, pure music, and silence or noise. The aim of the SAD is to remove only long regions without speech such as silence, music and noise, so the penalty of switching between models in the Viterbi decoding was set to minimize the loss of speech signal. The GMMs, each with 64 Gaussians, were trained on about 1 hour of the specific type of data, selected from English Broadcast News data from 1996 and 1997 distributed by the LDC (Linguistic Data Consortium).

C. Chopping into small segments

The segmentation process consists of finding segment boundaries that correspond to the instantaneous speaker change points. It is generally a good choice to minimize the miss rate for speaker change points even if the false alarm rate is high, because the false change points can be easily removed later during a clustering process.

The segmentation needs to provide pure segments (i.e.

G(s 1 , s 2 ) = (µ 2 -µ 1 ) Σ -1/2 1 Σ -1/2 2 (µ 2 -µ 1 ) (1)
It is the Mahalanobis distance between µ 1 and µ 2 weighted by the geometric mean of Σ 1 and Σ 2 , which reduces to a weighted Euclidean distance because of the diagonal assumption. The detection threshold was optimized on the training data in order to provide acoustically homogeneous segments. The window size was set to 5 seconds with a minimal segment length of 2.5 seconds. Due to the simple diagonal assumption, this segmentation phase is very quick. This approach is similar to the segmentation proposed in [START_REF] Siegler | Automatic segmentation and clustering of broadcast news audio[END_REF] using the symmetric KL2 metric. Other popular segmentation methods are based upon the BIC metric [START_REF] Chen | Speaker, environment and channel change detection and clustering via the Bayesian information criterion[END_REF], [START_REF] Delacourt | DISTBIC: a speaker-based segmentation for audio data indexing[END_REF], [START_REF] Cettolo | Segmentation, classification and clustering of an Italian broadcast news corpus[END_REF],

but these methods show a much higher complexity [START_REF] Cettolo | Evaluation of BICbased algorithms for audio segmentation[END_REF].

An analysis of various speaker change point techniques based on models, metrics or energy is given in [START_REF] Kemp | Strategies for automatic segmentation of audio data[END_REF].

D. Iterative GMM segmentation/clustering procedure

Each initial segment is used to seed one cluster, and an 8-component GMM with a diagonal covariance matrix is trained on the segment's data. The algorithm alternates the Viterbi resegmentation and the GMM re-estimation and merging steps, with the goal of maximizing the objective function:

N i=1 log f (s i |M ci ) -αN -βK (2)
where

S = (s 1 , • • • , s N ) is the partitioning of the speech segments into a sequence of N segments, c i ∈ [1, K]
is the cluster label for the segment s i among the K different clusters, f (s i |M ci ) is the likelihood of the segment s i given the model of its cluster M ci , and α and β are the segment and cluster penalties. The procedure stops when no more merges are possible. More details on the clustering procedure can be found in [START_REF] Gauvain | Partitioning and transcription of broadcast news data[END_REF]. This procedure is similar to BIC using a global penalty as described in Section III-A.

E. Viterbi resegmentation

The segment boundaries are refined using the last set of GMMs and an additional relative energy-based boundary penalty, within a one second interval. The boundaries are thus shifted to the nearest point of low energy within this interval. This is done so as to locate the segment boundaries at silence portions, thereby avoiding cutting words. This is especially important when using the resulting segmentation as a pre-processing step of an automatic transcription system.

F. Bandwidth and gender labeling

Bandwidth (wide band studio or narrow band telephone) detection for each segment is first performed using two GMMs. The gender (male or female) labeling is then carried out on the segments using two pairs

III. MULTI-STAGE DIARIZATION

Recent research has shown BIC clustering methods to obtain good performance on the speaker diarization task [START_REF] Moh | Towards domain independent speaker clustering[END_REF], [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF]. We therefore tested a modified system, replacing the iterative GMM clustering with a BIC-based clustering (cf. Figure 1) and using Gaussian models with a full covariance matrix. We believe the iterative resegmentation to be more important in the context of speaker partitioning for the transcription task than for the speaker diarization task. Since different models can capture different and complementary aspects of the data, we decided to combine them in a multi-stage system.

We decided to pipeline the output of the system with the BIC clustering into a second clustering stage which uses a speaker identification module. The SID clustering uses a more aggressive acoustic channel normalization and a more complex speaker model, enabled by the larger amount of data per cluster at the beginning of this stage. Finally, an SAD post-filtering stage was added in order to remove short intra-speaker pauses.

These short pauses while indeed harmless for a speech transcription system, are penalized as false alarms in a speaker diarization system. The other parts of the system were kept unchanged.

A. BIC clustering

Agglomerative clustering is applied to the segments resulting from the GMM segmentation. Initially, each segment seeds one cluster, modeled by a single Gaussian with a full covariance matrix trained on the 12 Mel frequency cepstrum coefficients and the energy (but without the ∆ coefficients). At each iteration, the two nearest clusters are merged until the stopping criterion is reached. The BIC criterion [START_REF] Chen | Speaker, environment and channel change detection and clustering via the Bayesian information criterion[END_REF] is used both for the inter-cluster distance measure and the stop criterion.

In order to decide whether to merge two clusters c i and c j , the ∆BIC value is computed as:

∆BIC = (n i +n j ) log |Σ|-n i log |Σ i |-n j log |Σ j |-λP (3)
March 22, 2017 DRAFT

where Σ is the covariance matrix of the merged cluster (c i and c j ), Σ i of cluster c i , Σ j of cluster c j , and n i and n j are respectively the number of the acoustic frames in clusters c i and c j . The penalty P is:

P = 1 2 (d + 1 2 d(d + 1)) log n ( 4
)
where d is the dimension of the feature vector space. The ∆BIC between all cluster pairs is greater than zero.

term n i log |Σ i | is related
In our BIC clustering procedure, the size of the two merged clusters, i.e. n = n i + n j , is used to compute the penalty P , as described in [START_REF] Cettolo | Segmentation, classification and clustering of an Italian broadcast news corpus[END_REF]. We refer to this as a local BIC penalty. Another solution is to use the size of the whole set of clusters, i.e. n = N k=1 n k to compute the penalty, which we refer to as a global BIC penalty and corresponds to an exponential prior for the number of clusters. In this case the penalty is constant and the decision to merge two clusters is decided just by the increase in likelihood. This in fact corresponds to the objective function in Equation ( 2) used in the baseline partitioner when the number of segments is fixed. For broadcast news documents, our experimental results as presented in Section IV demonstrate the local BIC to be a better choice for a merging criterion. 1

more precisely, log f (c i |Mc i ) = -n i 2 log |Σ i |-n i d 2 (1+log 2π
), but the constant factor 1 2 was simplified in Equation 3.

B. SID clustering

After the initial segmentation, both the iterative GMM and the agglomerative BIC clustering methods have to deal in the beginning of the process with short duration segments, and thus use a limited set of parameters per cluster. After several iterations, the amount of data per cluster increases, so a more complex model can be used.

Our approach is to stop the initial clustering stage early, and use the results to seed a second clustering stage with more initial data per cluster. This second stage can therefore estimate more complex models for the speakers. In addition, purely acoustic clustering tends to split a speaker's data into several clusters as a function of the various background conditions (clean speech, speech with noise, speech with music etc.), so an acoustic background normalization is necessary to regroup the data for a given speaker.

After the BIC clustering stage, state-of-the-art speaker recognition methods [START_REF]Digital Signal Processing (DSP)[END_REF], [START_REF] Barras | Feature and score normalization for speaker verification of cellular data[END_REF] were used to improve the quality of the speaker clustering. The feature vector consists of 15 Mel frequency cepstral coefficients plus delta coefficients and delta energy. Feature warping normalization, which reshapes the histogram of the cepstral coefficients into a Gaussian distribution [START_REF] Pelecanos | Feature warping for robust speaker verification[END_REF] is performed on each segment using a sliding window of 3 seconds in order to reduce the effect of the acoustic environment.

For each gender and channel condition (studio, telephone) combination, a Universal Background Model (UBM [START_REF] Reynolds | Speaker verification using adapted Gaussian mixture models[END_REF]) with 128 diagonal Gaussians is trained on the 1996/1997 English Broadcast News data. The GMM for each remaining cluster is obtained by maximum a posteriori (MAP) adaptation [START_REF] Gauvain | Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains[END_REF] of the means of the matching UBM.

Agglomerative clustering is performed separately for DRAFT [START_REF] Canseco-Rodriguez | Speaker diarization from speech transcripts[END_REF] each gender and bandwidth condition, using a cross log-likelihood ratio as in [START_REF] Reynolds | Blind clustering of speech utterances based on speaker and language characteristics[END_REF]. For each cluster c i , its model M i is MAP adapted from the gender and channel matched UBM B using the feature vectors x i belonging to the cluster. Given two clusters c i and c j , the cross log-likelihood ratio S is defined as:

S(c i , c j ) = 1 n i log f (x i |M j ) f (x i |B) + 1 n j log f (x j |M i ) f (x j |B) (5)
where f (•|M ) is the likelihood of the acoustic frames given the model M , and n i is the number of frames in cluster c i . This is a symmetric similarity measure.

After each merge, a new model is trained for the cluster c i ∪c j . The clustering stops when the cross log-likelihood ratio between all clusters is below a given threshold δ optimized on the development data.

C. SAD post-filtering

In order to filter out short-duration silence segments that were not removed in the initial speech detection step to further reduce the speaker diarization error, a post-processing stage uses the word segmentation output by the LIMSI Broadcast News Speech-To-Text system [START_REF] Nguyen | The 2004 BBN/LIMSI 10xRT English broadcast news transcription system[END_REF] relying on the c-std system for the segmentation and clustering. Only inter-word silences shorter than 1 second are filtered out, this value being determined empirically.

IV. EXPERIMENTS AND RESULTS

Experiments are reported for diarization systems that were submitted to the NIST and ESTER evaluations [START_REF] Zhu | Combining Speaker Identification and BIC for Speaker Diarization[END_REF].

Several configurations were tested for the systems. Unless otherwise specified, the configuration used is the one that provided the best results on development data, i.e. α = β = 230 for c-std, λ = 5.5 for c-bic and λ = 3.5, δ = 0.1 for c-sid and p-asr. In the c-sid system, the BIC penalty weight λ was optimized to cluster only the closest segments in the BIC clustering stage so as to give more degrees of freedom to the SID clustering stage; while in the c-bic system, λ was optimized directly to give the lowest diarization error the BIC clustering could bring. A local BIC merging and stop criterion was also used.

A. Corpora

The experiments were conducted on the US English data used in NIST RT-04F (Fall 2004 Rich Transcription Evaluation) [START_REF]Fall 2004 Rich Transcription (RT-04F) evaluation plan[END_REF] and on the French data from the French ESTER broadcast news evaluation [START_REF] Gravier | The ESTER evaluation campaign of rich transcription of French broadcast news[END_REF].

B. Performance measures

The speaker diarization task performance is measured via an optimum one-to-one mapping between the reference speaker IDs and the hypothesis speaker IDs. The primary metric for the task, referred to as the speaker match error, is the fraction of speaker time that is not attributed to the correct speaker, given the optimum speaker mapping. Another measure is the overall speaker diarization error rate (DER) which includes the missed and false alarm speaker times, thus taking speech/nonspeech detection errors into account [START_REF]Fall 2004 Rich Transcription (RT-04F) evaluation plan[END_REF]. All the measurements mentioned above are illustrated in Figure 2.

In order to more closely analyze the performance of speaker clustering methods, average frame-level cluster purity and cluster coverage are used as defined by [START_REF] Gauvain | Partitioning and transcription of broadcast news data[END_REF].

Cluster purity is defined as the ratio between the number of frames by the dominating speaker in a cluster and the total number of frames in the cluster. Cluster coverage is a dual measure, and accounts for the dispersion of a given speaker's data across clusters; for a given speaker, it is defined as the percentage of its frames in the cluster which has most of the data of the speaker. Cluster coverage was also expressed as the purity of reference clusters in [START_REF] Cettolo | Segmentation, classification and clustering of an Italian broadcast news corpus[END_REF]. In these experiments, cluster purity and cluster coverage errors are reported.

Cluster purity and coverage measures are complementary, and the speaker match error time can be interpreted as a combination of both. Moreover, it is interesting to note that if, the hypothesized speaker for a cluster is always taken to be the majority reference speaker in this cluster, then the speaker match error will be exactly the cluster purity error; it is easy to demonstrate that it is also a lower bound for the match error. Thus, starting with an initial segmentation, the cluster purity error will be the lowest possible match error on this segmentation after performing an agglomerative clustering. The same holds for further clustering of the output of a previous clustering stage, as long as the segment boundaries are not modified.

C. Results on the RT-04F development data

The performance at different stages of the system was compared for different system configurations, as presented in Table I. On RT-04F dev1 data, the initial segmentation c-seg, with the minimal duration constraint of 2.5 sec per segment, has a purity error of 1% which is also the lowest possible speaker match error. The As expected, the standard partitioner c-std in its default configuration provides good purity, but relatively poor coverage, resulting in a high overall diarization error of over 30% on dev1 data. Setting the penalty α and β to optimize these values reduces this error below 25%. The c-bic system also provides a high purity, with a much better coverage (2.9% purity error and 9.8% coverage error), reducing the overall error rate by almost 50%. The c-sid system achieves a large decrease of the coverage error with a further small improvement of the purity, resulting in an overall DER of 7.1%, a reduction of almost 50% compared to the c-bic system.

A global BIC merging and stop criterion was also tested, but always performed worse than the local BIC criterion in our experiments, as can be seen for c-bic system on RT-04F dev1 in Table II. A similar result was observed in [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF]. This result needs further investigation, but may be due to a mismatch between the BIC model and the real distribution of the data. Thus only the local criterion was used in the remaining experiments.

The effect of the SID detection threshold δ on the speaker match error and the cluster purity error was measured on both the dev1 and dev2 data. A lower threshold reduces the number of final speaker clusters.

As shown in Figure 3, reducing the threshold in the positive range results in a decrease of the match error rate with almost no degradation of the purity. Moreover, there is a large range of thresholds around zero with a low speaker match error. The cluster purity error shows the speaker match error that could be achieved with the best clustering decision, as explained in section IV-B.

Looking at the performance of the c-sid system in

D. Results on the ESTER training and development data

For the French ESTER data, SAD was performed using the same American English speech/non-speech acoustic models as were used for RT-04F plus an additional speech over music model trained on French broadcast news data for a better recognition of the jingles found in the French radio data. The optimal threshold for the SID clustering on development data was δ = 1.5.

As can be seen in Table I, the c-sid system also has low purity and coverage error rates

E. Results on the evaluation data

The trends observed on the development data were confirmed on the RT-04F evaluation data. The results given in Table V, show a slight increase in overall diarization error to 17% for the c-bic system and to 9.1% for the c-sid system. The final SAD post-processing stage gives an improvement of 0.6%, mainly by reducing false alarms in speech detection. As mentioned in [START_REF] Reynolds | Approaches and applications of audio diarization[END_REF],

the p-asr system had the best performance of all the participants of the RT-04F evaluation by a significant margin.

Results on the ESTER evaluation data are given in Table V, with the setting optimized on the development data. The overall diarization error was reduced from 13.8% for the c-bic system to 11.5% for the c-sid system.

The submitted system also had the best performance for this task in the ESTER evaluation [START_REF] Galliano | The ESTER phase II evaluation campaign for the rich transcription of French broadcast news[END_REF]. In a postevaluation experiment, a 20% relative reduction of the overall diarization error was observed for the c-sid system with the best a posteriori threshold, showing that the error rate is highly dependent on the clustering threshold.

V. CONCLUSIONS

In this paper, a multi-stage architecture for speaker diarization of broadcast news has been described. It builds upon a baseline speaker partitioning system which had been optimized for the automatic transcription task, but the constraints of the speaker diarization task are different. Several modifications to the baseline system have thus been explored. First, the iterative GMM clustering was replaced by an agglomerative BIC clustering, using single full-covariance Gaussian models. A local BIC merging and stop criterion was shown to outperform the global criterion; a similar result was observed in [START_REF] Tranter | Speaker diarisation for broadcast news[END_REF].

A second clustering module was applied to the output of the system, relying on techniques used for speaker identification and verification: acoustic channel normalization, and MAP adaptation of a reference GMM with a large number of Gaussians.

The multi-stage system was demonstrated to perform much better than the baseline system for the diarization task. On the RT-04F development data, a relative error reduction of over 70% was achieved when compared to the baseline system. This system obtained the best diarization performance in both the NIST RT-04F and the ESTER evaluations by a significant margin. An overall diarization error rate under 10% was obtained on the RT-04F evaluation data, while the error rate of the BIC-based systems were over 15%. Focusing on the speaker error only, the multi-stage system provides an error reduction of up to 50% relative to a standard BIC clustering system. This dramatic improvement over the baseline system results from several changes: the combination of March 22, 2017 DRAFT two different clustering stages, each one focusing on a different acoustic aspect with more complex modeling in the second stage, and the use of acoustic channel normalization methods suited to speaker identification.

A system following this architecture recently developed at Cambridge University demonstrated similar improvements [START_REF] Sinha | The Cambridge University March 2005 speaker diarisation system[END_REF], where it was observed that a very important part of the gain was obtained by the feature warping normalization.

Several remaining issues need further investigation in order to improve the robustness and the efficiency of the system. It was observed that the clustering threshold needs to be set according to the type of the audio document, and that the system still has a large variability across individual shows. Only with a large amount of files can statistically consistent results be obtained. This is especially important since the speaker error does not provide a stable and continuous measure of a clustering system: splitting a speaker in two classes, which is a single decision, results in doubling of the error rate for this speaker.

Finally, most speaker diarization systems rely on a purely acoustic segmentation and clustering, whereas an essential part of the information in speech is of a linguistic nature, and obviously in TV and radio shows most speakers are presented and identified. Combining the acoustic information with the linguistic layer as explored in [START_REF] Canseco-Rodriguez | Speaker diarization from speech transcripts[END_REF] would improve the robustness of a speaker diarization system and make it more exploitable by a human reader.

Fig. 1 .

 1 Fig. 1. Architecture of the baseline partitioning system (c-std on the left side of the diagram) and the multi-stage speaker diarization system (p-asr to the right, along with c-seg, c-bic and c-sid intermediate steps).

  to the log likelihood of the cluster c i given its estimated Gaussian M ci 1 . Singular covariance matrices were not an issue because of the minimal length constraint during the initial segmentation. The merging criterion is that two clusters should be merged if ∆BIC < 0. At each step the two nearest clusters (i.e. those which have the most negative ∆BIC value) are merged into one cluster, and the ∆BIC values between this new cluster and remaining clusters are computed. This clustering procedure terminates when the

  For the RT-04F evaluation, the training and development corpora were provided for system development along with a reference speaker labeling determined by LDC. Evaluation references were made available after the evaluation. The data are from US radio or TV broadcast news shows. The development data has two portions, 'dev1' and 'dev2' each consisting of 6 30minute audio files. Dev1 was recorded in February 2001, with programs from ABC, CNN, NBC, PRI and VOA; and dev2 was recorded in November and December 2003, with programs from ABC, CNBC, CNN, CSPAN and PBS. The evaluation data is comprised of 12 audio files each lasting approximately 30 minutes, recorded in December 2003, extracted from the shows of ABC, CNBC, CNN, CSPAN, PBS and WBN. The dev2 and evaluation and corpora are very similar to each other since the shows are recorded from the same channels, at the same period, whereas dev1 is an older corpus coming from the previous RT-03 evaluation. The data used in ESTER were extracted from French radio broadcast news shows, provided by ELDA (Evaluations and Language resources Distribution Agency) and the DGA (Délégation Générale pour l'Armement). The training corpus contains 82 hours of data from the France March 22, 2017 DRAFT Inter, France Info, RFI and RTM radio stations, recorded in 1998, 2000 and 2003, with the audio file durations ranging from 10 minutes to 1 hour. The development corpus contains 8 hours of data, in 14 audio files recorded from April to July 2003 from the same stations as the training corpus. The evaluation corpus is comprised of 18 audio files recorded from October to December 2004, with a total duration of 10 hours. The evaluation corpus contains data from two radio stations ('France Culture' and 'Radio Classique') not present in the training or development corpora. There is also a 14 month interval between the recording period of the development and test data (July 2003 to October 2004) of these two corpora. Both data sets present a large variability in audio file durations (10 minutes to 1 hour).

Fig. 2 .

 2 Fig. 2.Example of the performance measures for the speaker diarization task used in the NIST RT-04F evaluation.

( 4 .Fig. 3 .

 43 Fig. 3. Speaker match error and purity error rates on RT-04F dev1 and dev2 for the c-sid system as a function of the SID clustering threshold δ.

TABLE I THE

 I PURITY, COVERAGE AND OVERALL DIARIZATION ERROR RATES FROM THE c-std, c-bic AND c-sid SYSTEMS ON THE RT-04F AND THE ESTER DEVELOPMENT DATASETS.

		system		purity	coverage	overall
				error (%)	error (%)	DER
			RT-04F dev1 data set
			c-seg	1.0	73.2	N/A
	c-std (α = β = 160)	5.0	28.4	32.3
	c-std (α = β = 230)	9.4	17.9	24.8
		c-bic (λ = 5.5)	2.9	9.8	13.2
	c-sid (λ = 3.5, δ = 0.1)	2.1	4.2	7.1
			RT-04F dev2 data set
	c-sid (λ = 3.5, δ = 0.1)	1.7	3.5	7.6
			ESTER development data set
		c-bic (λ = 5.5)	7.2	10.6	15.8
	c-sid (λ = 3.5, δ = 1.5)	4.7	5.2	8.0
			TABLE II
	THE OVERALL DIARIZATION ERROR FOR c-bic SYSTEM ON THE
	RT-04F dev1 DATA, AS A FUNCTION OF THE PENALTY WEIGHT λ
	FOR THE LOCAL AND GLOBAL BIC CRITERION.
	BIC	λ	overall	BIC	λ	overall
	criterion		DER (%) criterion	DER (%)
		5.0	13.3		5.0	16.4
	local	6.0	12.8	global	6.0	15.5
		7.0	13.8		7.0	18.2
	more detail, a large variation in the speaker error is
	observed across shows, as shown in Table III. The
	speaker error ranges from a low of 0.1% to over 12%.
	Having only very few speakers (3), the CSPAN show
	has the lowest speaker error. The ABC and NBC shows
	have more speakers, occurring in different background
	conditions, which is more challenging for the diarization
	system.
	March 22, 2017					DRAFT

TABLE III PERFORMANCE

 III OF c-sid SYSTEM ON THE RT-04F DEVELOPMENT DATASET, SCORES ARE GIVEN FOR MISS (MS), FALSE ALARM (FA),

	SPEAKER ERROR (SPK) AND OVERALL DIARIZATION ERROR RATE
	(DER), #REF AND #SYS ARE RESPECTIVELY THE REFERENCE
		AND SYSTEM SPEAKER NUMBER.
	show #REF #SYS MS FA SPK DER
	dev1	121	161	0.4 1.3	5.4	7.1
	ABC	27	37	1.6 1.3 12.4 15.2
	VOA	20	22	0.3 1.2	2.2	3.7
	PRI	27	30	0.1 0.9	2.8	3.8
	NBC	21	35	0.1 1.1 12.0 13.2
	CNN	16	21	0.5 1.4	5.6	7.6
	MNB	10	16	0.2 1.8	0.8	2.8
	dev2	90	130	0.5 3.1	4.1	7.6
	CSPAN	3	4	0.3 2.9	0.1	3.3
	CNN	17	22	0.6 4.2	5.0	9.8
	PBS	27	29	0.1 2.8	7.4 10.3
	ABC	23	29	2.1 6.7 12.5 21.2
	CNNHL	9	26	0.0 1.4	0.5	1.9
	CNBC	11	20	0.2 1.0	0.9	2.1
	δ on all of the training data is the same as the one for the
	development data. The show duration was taken here as
	a rough indicator of the speaker count, however a more
	appropriate model of the show type would enable a finer
	analysis.

TABLE IV RESULTS

 IV OF THE DIFFERENT OPTIMAL SID CLUSTERING THRESHOLD δ FOR THE ESTER TRAINING SUBSETS WITH THE DIFFERENT SHOW DURATIONS.

	training	δ	purity	coverage	speaker match
	subset		error (%)	error (%)	error (%)
	10 min 0.9	1.2	1.5	2.6
	15 min 0.9	3.6	0.5	3.8
	20 min 1.1	2.2	1.6	3.5
	1 hour 1.5	3.5	5.2	7.6
	all 1.5	3.0	4.7	6.7

TABLE V PERFORMANCES

 V OF c-bic, c-sid AND p-asr SYSTEMS ON THE EVALUATION DATA OF RT-04F AND ESTER.

	system	missed	false alarm speaker overall
		speech	speech	error	DER
		RT-04F test dataset
	c-bic	0.4	1.8	14.8	17.0
	c-sid(δ = 0.1)	0.4	1.8	6.9	9.1
	p-asr	0.6	1.1	6.8	8.5
		ESTER test dataset
	c-bic	0.7	1.0	12.1	13.8
	c-sid(δ = 1.5)	0.7	1.0	9.8	11.5
	post-evaluation result on ESTER test dataset
	c-sid(δ = 2.0)	0.7	1.0	7.4	9.1

[START_REF] Canseco-Rodriguez | Speaker diarization from speech transcripts[END_REF] DRAFT

This work was partially financed by the European Commission under the FP6 Integrated Project IP 506909 CHIL and by the Defense Advanced Research Projects Agency under the GALE Program. Any opinions expressed in this paper are those of the authors and do not necessarily reflect the views of DARPA.