Anthony Larcher
email: anthony.larcher@univ-avignon.fr

Jean-Francois Bonastre
email: jean-francois.bonastre@univ-avignon.fr

John S D Mason

Constrained Viterbi decoding for embedded user-customised password speaker recognition AB-105

Keywords: H.4 [Information Systems Applications]: Miscellaneous Speaker Recognition, Viterbi decoding, Embedded system, Password recognition Training Sequence Test

Embedded speaker recognition in mobile devices could involve several ergonomic constraints and a limited amount of computing resources. GMM/UBM systems have proved their efficiency in more classical contexts where good accuracy depends on a relatively large quantity of speech data. The proposed GMM/UBM extension addresses the situations with limited resources and takes advantage from the temporal structure of speech by using client-customised utterances harnessed by a Markov model. New temporal information is then used to enhance discrimination with Viterbi decoding by increasing the gap between client and impostor tests. Experiments on the MyIdea database are performed when impostors know the client-utterance and also when they do not, highlighting the potential of this new approach. A relative gain up to 64% in terms of EER is achieved when impostors do not know the client utterances and performance is equivalent to the GMM/UBM baseline system in other configurations.

INTRODUCTION

Classical speaker recognition engines, working on text-independent inputs, allow a high level of performance as shown during NIST evaluation [START_REF] Przybocki | NIST speaker recognition evaluations utilizing the mixer corpora -2004[END_REF]. In a realistic application, the efficiency of these systems could be influenced by several constraints. An embedded application for example often has computational and memory limitations as well as ergonomic aspects; these can mean small quantities of model data and short test audio sequences. State-of-the-art speaker recogntion systems, which often follow the GMM/UBM paradigm [START_REF] Bimbot | A Tutorial on Text-Independent Speaker Verification[END_REF] depend strongly on the quantity of training data available to enrol a speaker and usually requires a few minutes of speech to achieve good performance [START_REF] Fauve | Improving the performance of text-independent short duration SVM-and GMM-based speaker verification[END_REF]. A positive contribution to this problem is to increase the amount of information taken into account by the system by including text dependencies, like in a user-customised password scenario [START_REF] Benzeghiba | User-customized password speaker verification using multiple reference and background models[END_REF]. In this case, the Temporal Structure Information (TSI) gathered from the password helps to compensate for the lack of data due to short duration of audio sequences. In order to model the TSI of speech while achieving statistical modelling, a word recognition system could be combined with a speaker recognition system [START_REF] Hebert | Phonetic class-based speaker verification[END_REF]. An embedded system is often confronted with strongly variable inputs and implementation constraints. Due to this constraint, the acoustic modelling used in the recognition has to be adapted to the environment. Hidden Markov Model (HMM) modelling does not seem well suited as full model adaptation requires a large amount of data and a significant resource consumption. Furthermore the conventional HMM/Viterbi approach does not seem to fit the passwordbased speaker recognition task. Indeed, the information gathered from the user-customised passwords aims to discriminate clients who pronounce their own password from impostors i) the correct client password, or ii) an unknown utterance. Here we consider particularly the latter case. The Viterbi decoding provides an optimal alignment for client but also impostor test sequences and the maximisation of the likelihood computed for impostor test sequences could lead to a drop in performance. A way to improve discrimination between client and impostor tests is to add a priori information on the password temporal structure. Including such information within the Viterbi decoding aims to degrade scores for impostor accesses. In this paper, two approaches are associated in an architecture able to deal with the application context constraints. The first is the standard GMM/UBM paradigm, well known for the general acoustic space modelling and its text-independent speaker recognition abilities. The second one is an HMM/Viterbi approach used to take advantages of the text-dependent and TSI aspects by using a Semi-Continuous HMM (SCHMM) [START_REF] Young | The general use of tying in phoneme-based HMM speech recognisers[END_REF]. Such a combined system was originally proposed in [START_REF] Bonastre | Gaussian Dynamic Warping (GDW) Method Applied to Text-Dependent Speaker Detection and Verification[END_REF] for speaker recognition and extended to word recognition in [START_REF] Lévy | Mobile Phone Embedded Digit-Recognition[END_REF].

A key point here is the inclusion of an external, additional synchronisation process during testing. The process is a Viterbi decoding and aims to enhance discrimination by using the temporal structure of the pronounced utterance. The aim is to reduce score values when the speaker does not know the client password, by making the Viterbi path sub-optimal. Ideally, the separate information source should be complementary and external to the SCHMM TSI, such as video synchronisation. In this work, we use a word-based synchronisation from an automatic alignment process. Preliminary results of this approach were shown in [START_REF] Larcher | Reinforced temporal structure information for embedded utterance-based[END_REF] and [START_REF] Larcher | Short utterance-based video aided speaker recognition[END_REF]; this paper focus on the constrained Viterbi decoding with more accuracy and shows more precise results. The specific acoustic architecture is described in Section 2. The constrained Viterbi algorithm is described in Section 3. Section 4 describes the experimental protocol and results. It includes a description of the MyIdea database. Section 5 summarises the benefits of this approach and presents future work direction.

A THREE LEVEL ACOUSTIC ARCHI-TECTURE

The proposed three level architecture shown in Figure 1 is called EBD for Embedded LIA_SpkDet and is an extension of the well known GMM/UBM paradigm configured to deal with the usercustomised speaker recognition task.

GMM/UBM the architecture of the two first layers of the EBD is similar to a classical GMM/UBM speaker recognition system. A text-independent model of every client is trained by adapting the UBM. This adaptation is described below.

User-customised extension the previous text-independent speaker model is then used to obtain a SCHMM with the goal of harnessing the TSI of the utterance chosen by this speaker. Each state of the SCHMM is trained from a part of that utterance using an iterative Viterbi decoding process. During the test, classical Viterbi decoding is again performed with this SCHMM. Details of the training and test are given below.

Two scores are computed, the first is obtained with only the GMM/UBM modelling and the second is computed with the SCHMM model. These scores are combined to provide a final score for the decision stage.

Training step

The EBD model is trained in three steps, each corresponding to one level of the architecture. The UBM is firstly trained to model the acoustic speech space. It is built off line using a suitably large amount of representative data. It is trained with a classical EM/ML algorithm [START_REF] Dempster | Maximum Likelihood from Incomplete Data via the EM Algorithm[END_REF]. The training of the speaker text-independent models consists in adapting the UBM/GMM with the available data pronounced by the client. An energy labelling is performed on the signal and only the frames deemed to be speech are kept. The model is obtained by adapting the UBM using the EM algorithm with the MAP criterion [START_REF] Gauvain | Maximum a Posteriori estimation for Multivariate Gaussian Mixture Observations of Markov Chains[END_REF]. The third level is to initialise an utterance SCHMM model; the utterance sequence is cut into S segments {segi} of the same length. Each state i of the SCHMM is adapted from the speaker text-independent model using the speech-labelled frames of segi. An EM/MAP algorithm is applied on the weight parameters. Then the SCHMM is optimised using a classical Viterbi algorithm (a new segmentation is achieved by Viterbi and is used to adapt the state models). The number of states of the SCHMM is experimentally determined. Finally, the transition probabilities of the SCHMM are computed using the relative length of each segment.

Testing step

During a test, the score between the input signal and an utterance SCHMM model is derived from the corresponding Viterbi path. As for the training, the log-likelihood for an input frame is only computed for each Gaussian component of the text-independent model. The system is computational efficient since scores from the original GMM/UBM are used throughout, with only the weightings changed according to state occupancy of the SCHMM. This scoring process is equivalent in terms of computation to a classical GMM/UBM system producing two scores. The first is obtained with only the text-independent speaker model and the second is computed with the SCHMM model, which itself has two operational modes, namely without further constraints or constrained by the external information as described in Section 3. These two scores are normalised using the log-likelihood of the UBM. They are then combined to give a final score for the decision stage. An empirically-tuned weighted linear combination is used.

THE CONSTRAINED VITERBI

Synchronisation points are generated from an external source during both the training and testing phase. These points are used to strongly constrain the Viterbi decoding. This constraint is obtained by allowing or forbidding transitions of the SCHMM corresponding to the synchronisation points (labelled S in Figure 2). In this case, the bottom layer of the EBD system could be compared to a succession of sub-SCHMM. The Viterbi algorithm is then processed from one synchronisation point until the next with the corresponding sub-SCHMM. Two types of consequences could be expected from the proposed contraining procedure. The addition of complementary information could improve the training of SCHMM models. Such an improvement could potentially lead to increase client scores during the testing phase. The second effect, mainly expected when constraining the Viterbi decoding consists in increasing the discrimination power of the EBD approach. This effect is described below. The external constraint is illustrated on Figure 3. The Viterbi path computed under this constraint is not allowed to pass through the blackened states area. Free zones correspond to the intersection of the synchronisation computed on both the training and the testing utterances. Figure 3 shows paths obtained on a same model for two different password occurrences:

• the occurrence used to train this SCHMM model;

• one occurrence of the same password, pronounced by the client. The temporal structure of a client test utterance is supposed to be close to the temporal structure of the training utterance. With this hypothesis the path computed for the client test utterance goes through the non-forbidden area only. This is illustrated by Figure 3. In this case, the synchronisation constraint has no effect on the Viterbi decoding and the resulting score remains high.

Figure 4 shows two expected alignment path for a given impostor test utterance, one with the synchronisation constraint (the more central one) with or without the synchronisation constraint. The temporal structure of an impostor utterance is assumed to be strongly different from the client password structure. The path resulting from the alignment of this utterance on the client-password SCHMM without applying the external constraint is assumed to go through the forbidden area. Thus, the synchronisation constraint forces the algorithm to find a path through the authorised area. This path is not optimal and the impostor score is therefor lower than the one computed without any constraint.

EXPERIMENTS

The MyIdea database

Experiments are performed on the BIOMET part of the MyIdea database. This database contains audio-video records from 30 male speakers. In this subpart of MyIdea, 25 sentences are recorded in 3 sessions for each speaker. Twelve of these sentences are the same for all the speakers, ten short (about 3 seconds) and 2 long sentences (about 6 seconds). Other occurrences are speaker or session dependent. The three sessions are recorded under controlled acoustic and illumination conditions. MyIdea presents several drawbacks for our work. The recordings are not made in a real environment. The sentence duration variability is limited (2 or 3 seconds for the utterance occurrences), the sentences are too long for a real password dedicated system, and the number of speakers is small for a speaker verification experiment. However it use is reported here with future work aimed at incorporating visual information.

Experimental protocol

The 30 male speakers are separated into two groups -A and Beach with 15 speakers. Each group is successively used as the Client-set with the others used to train the UBM. The UBM is trained using the whole recorded material of the 15 speakers of the UBM-set. When using the A-group data set to train the UBM model, the speakers from the B-group are used for enrolment and tests. Due to the small number of speakers available, a jacknifing process is used by training a client model for each available speaker session. Each of the 15 speakers of this Client-set is successively considered as a client for which the 14 other speakers of the Client-set are impostors.

Each client text-independent GMM model is derived from the UBM by using two long sentences and one occurrence of the selected short sentence (around 8 seconds of speech). The utterance-dependent model is trained with the same short sentence occurrence (around 2 seconds of speech). With the jacknifing process, 900 utterance models are trained (10 short sentences, 3 sessions and 30 clients). ALL configuration the impostor tests are all tests from both the KNOWN and the UNKNOWN configurations.

For both KOWN and UNKNOWN configurations, the number of impostor tests is constant for a given client. Moreover the global number of impostor tests is 37, 800 in those two configurations and 75, 600 in the ALL configuration.

System configuration

Mel-scaled frequency cepstral coefficients (MFCC) are used, computed every 10ms. An energy labelling is applied to separate the speech frames from the non-speech frames. Acoustic feature frames are 32-dimension vectors, 15 cepstral coefficients, the log-energy and the corresponding ∆ coefficients.

In the experimental configuration, the number of components in GMMs is fixed to 256 and all mean parameters of the text-independent speaker models are adapted. Only the 32 most significant weights parameters are adapted for each SCHMM state.

Results

Experiments are conducted to asses the contributions coming from the three components, GMM/UBM, SCHMM and the constrained Viterbi. The GMM/UBM is regarded as the baseline. Benefits of the SCHMM extension are predicted to come from the TSI and the constrained decoding. The experimental results are presented in Table 1. The first column of Table 1 shows the results of the GMM baseline. Error rates fall from 4.00% to 3.22% and finally 2.46%for the KNOWN, ALL and UNKNOWN conditions respectively. These results show that the text-dependency has a real influence on the GMM/UBM performances. Decreasing the variability between the phonetic contents of the training and the test material allows successive improvements in reducing EER.

Configuration

The main expected advantage of the EBD system using SCHMM compared to a classical GMM/UBM is to incorporate the passwordbased information like the password itself and the relative TSI. This point is evaluated by the experiments presented in the second column of Table 1. A classical Viterbi algorithm is used in this experiment. Error rates fall from 2.46% to 1.11% by using SCHMM when impostors do not know the client passwords. This result is not confirmed when the impostors know the client utterances (KNOWN) and the performance of the EBD and GMM/UBM systems are equivalent.

In order to evaluate the effect of an external synchronisation, a new experiment is performed. Results of this experiment are presented in the third column of Table 1. As expected, performance of the EBD in the UNKNOWN condition improves when constraining the Viterbi decoding with an external segmentation. In this condition, impostors do not know the client password and pronounce a different utterance whose temporal structure is penalised by the synchronisation constraint.

A further analyse of the evolution of the client and impostor scores is presented in the following paragraphs.

Figure 5 shows the evolution of client score distributions with and without constraining the Viterbi decoding. Distributions of client scores remain equivalent with or without the external information. This additional information seems not to the quality of SCHMM models in terms of likelihood with test data. This observation could be explained by the nature of the additional information coming from a phonetical alignment. This information could be over correlated with the acoustic information already exploited by the SCHMM extension. At the same time, these results show that the Viterbi algorithm is still optimal when dealing with The integration of an external source of information in the Viterbi decoding is predicted to degrade the impostor scores. Figure 6 confirms that impostor scores decrease significantly when adding an external source of information. Furthermore, Figure 7 shows that the effect of the Viterbi constraint affects a huge majority of impostor tests. This observation shows that the synchronisation constraint allows the Viterbi algorithm to provide sub-optimal alignment when the temporal structure of the test utterance is different from the training utterance structure. The same experiment performed with impostors knowing the client password (Fig. 8) shows that the effect of the external information is less significant in this configuration due to the similar temporal structure of the utterance. This effect is not sufficient to discriminate speakers by the way they pronounce a lexical content.

Impostor score differences

Number of tests

CONCLUSIONS AND FUTURE WORKS

The approach proposed in this paper is designed for embedded applications. It takes advantages from a GMM/UBM textindependent approach and the HMM/Viterbi speech-recognition power. In addition we propose a version of the Viterbi algorithm constrained by an external source synchronisation. The use of this temporal information in a speaker recognition system leads to improved performance by reducing impostor scores. Performances of our approach is equivalent to the GMM/UBM baseline system when not considering the linguistic content (example of EER in KNOWN condition, GMM: 4.00, EBD 4.07) whereas the proposed approach outperforms the GMM/UBM when impostors do not know the client utterance (EER in UNKNOWN condition, GMM: 2.46, EBD: 1.11). Furthermore, the external synchronisation allows an additional gain when impostors do not know the client utterance (EER in UNKNOWN condition, EBD without constraint: 1.11, constrained-EBD: 0.89).

Future work will focus on the multi-modality by substituting the phonetic segmentation with temporal information extracted from the video stream. By incorporating this strong constraint in the training and testing phases we aim at increasing the performance and to thwart replay attacks. The EBD approach will be tuned to better balance the speaker and utterance specific information in order to outperform the baseline GMM in every condition. Moreover, given that the first results show the ability of the EBD approach to take advantage of the linguistic content of customised utterances, more tests are to be performed to evaluate the performance of the EBD system with more utterance-variability, for example considering the utterance duration.

Figure 1 :

 1 Figure 1: General view of the EBD model architecture.

Figure 2 :

 2 Figure 2: Use of an External Segmentation in the bottom layer of the EBD system to constraint the Viterbi decoding and increase the discriminative power of the EBD approach.

Figure 3 :

 3 Figure 3: Alignment for client training and test utterances with an external synchronisation.

Figure 4 :

 4 Figure 4: Alignment for impostor utterance with and without external synchronisation.

  The short sentences not used for utterance training are compared to the client model. 1,800 client tests are performed (2 test occurrences for each of the 900 utterances). Three configurations of impostor tests are proposed. The speaker and utterance models are compared to the 14 impostors who are the remaining speakers of the same group. UNKNOWN configuration the linguistic content of the impostor test occurrences is different from the training material of client models. Each speaker model is compared to three randomly selected short sentences (one per session) out of the 9 remaining sentences of each of the 14 impostor speakers. KNOWN configuration the linguistic content of the impostor test sequences is the same as the occurrences used to train the client models. Each utterance model is compared to three randomly selected sentences from each of the 14 other speakers of the Client-set.

Figure 5 :Figure 6 :

 56 Figure 5: Evolution of the client text-dependent score distributions with or without external synchronisation constraint.

Figure 7 :

 7 Figure 7: Evolution of the impostor text-dependent scores distributions with or without external synchronisation constraint.

Figure 8 :

 8 Figure 8: Evolution of the of impostors text-dependent scores distributions with or without external synchronisation constraint.

Universal Background Model Speaker X PassWord 1 State 1 State 2 State n

	Acoustic
	Space Model
	Text-independent
	Speaker Models
	Text-dependent
	Speaker Models

Table 1 :

 1 EER of GMM compared to the EBD system with 20 states per SCHMM, when constraining or not the Viterbi decoding

		GMM	EBD system
		baseline	Free	Constrained
	UNKNOWN	2.46	1.11	0.89
	KNOWN	4.00	4.06	4.07
	ALL	3.22	2.83	2.83

ACKNOWLEDGMENTS


Anthonylarcher

Jean-François Bonastre
email: jean-francois.bonastre@univ-avignon.frj.s.d.mason@swansea.ac.uk

Johns D M Ason

FROM GMM TO HMM FOR EMBEDDED PASSWORD-BASED SPEAKER RECOGNITION

Embedded speaker recognition in mobile devices involves a limited amount of computing resource but also is linked with several ergonomic constraints. For example both the enrolment and the test have to be done using short audio sequences. Even if they proved their efficiency in more classical situations, GMM/UBM based systems show their limits in this context. This paper deals with this problem and proposes to take into account the linguistic nature of the speech material inside the GMM/UBM framework. The proposed solution mixes the text-independent aspects of the GMM/UBM with a semi-continuous like approach in order to deal with the text-dependent information. This system respects both the resource and the ergonomic constraints of the considered application field.

The preliminary experiments are done on the publicly available database ValidDB and show the potential of the proposed approach. Particularly, when compared to the GMM/UBM, our approach decreases drastically both the computational cost and the equal error rates when impostors don't know the user passwords. For other situations the performance remains comparable between both approaches.

INTRODUCTION

An embedded speaker recognition system, in cellphones for example, has to respect several constraints. Computational and memory resources are limited and the ergonomic aspects of a realistic application impose to train the models with few data and to perform on short test audio sequences. Classical speaker recognition engines work on text-independent inputs and follow usually the GMM/UBM (Gaussian Mixture Model/ Universal Background Model) paradigm [START_REF] Bimbot | A tutorial on text-independent speaker verification[END_REF]. This solution allows a high level of performance as shown during NIST evaluations [START_REF] Przybocki | NIST speaker recognition evaluations utilizing the mixer corpora -2004[END_REF]. Unfortunately, the UBM/GMM performance depends strongly on the quantity of training data available to enrol a speaker when the embedded context involves short duration speech material. A solution to this problem is to increase the amount of information taken into account by the system by including text dependencies, like in user-customized password scenario [START_REF] Faouzi | User-customized password speaker verification using multiple reference and background models[END_REF]. In this case, the Temporal Structure Information (TSI) gathered from the password helps to compensate the short duration of the audio sequences. In order to model the TSI of speech while achieving statistical modelling, a word recognition system could be combined with a speaker recognition system [START_REF] Navratil | A speech biometrics system with multigrained speaker modeling[END_REF]. To satisfy the targeted application constraint, the system should accept all kinds of passwords and also be language-independent. Adding language options when using a phoneme-based word recognition system would seem viable with a good set of phonemes covering languages. However this solution would be expensive in terms of storage. A cellphone-embedded system is confronted with strongly variable environments. Due to this constraint the acoustic modelling used in the recognition system has to be adapted to the environment and the computational cost of the adaptation has to follow the targeted context resource constraints. HMM modelling doesn't seem well suited as it requires a large amount of training data and a significant resource consumption. The solution proposed in this paper tries to associate the well known advantages of a GMM based statistical acoustic modelling with an original architecture able to deal with the application context constraints. It uses the GMM/UBM paradigm for the general acoustic space modelling and the text-independent speaker recognition abilities. It also involves an HMM/Viterbi approach in order to incorporate the text-dependent and TSI aspects. Such a combined system was originally proposed in [START_REF] Bonastre | Gaussian dynamic warping (gdw) method applied to text-dependent speaker detection and verification[END_REF] for speaker recognition and extended to word recognition in [START_REF] Lévy | Mobile Phone Embedded Digit-Recognition[END_REF]. The specific three stage architecture is described in Section 2. The method using this structure to reduce memory and computational costs and the enrolment algorithms are also described in this section. The experimental protocol and preliminary results are described in Section 3 as well as the Valid database [START_REF] Fox | The realistic multi-modal valid database and visual speaker identification comparison experiments[END_REF]. Section 4 summarises the benefit of this approach and presents different future work directions.

DESCRIPTION OF THE APPROACH

The proposed system combines a statistical representation of the acoustic space and a precise modelling of the TSI. Based on a semi-continuous hidden Markov model (SCHMM) [START_REF] Young | The general use of tying in phonemebased hmm speech recognisers[END_REF], it operates a three stage acoustic modelling architecture. In order to involve the TSI with the respect of training data and resources limits, a GMM which represents the acoustic space is derived to obtain the SCHMM state probability functions.

EBD Hierarchical Architecture

Figure 1 illustrates the proposed three stage hierarchical architecture, denoted EBD, for Embedded LIA_SpkDET [START_REF] Bonastre | Alize/spkdet: a state-ofthe-art open source software for speaker recognition[END_REF] system in this paper. All the nodes in this architecture are a GMM. The upper layer is the least specialised one and is a classical UBM. It aims to model the general acoustic space. The middle layer contains the text-independent specific characteristics of each speaker. These text-independent speaker models are obtained by a classical GMM/UBM adaptation method: each speaker model is derived from the UBM following the Maximum A Posteriori (MAP) criterion [START_REF] Gauvain | Maximum a posteriori estimation for multivariate gaussian mixture observations of markov chains[END_REF] and using the EM algorithm. Only the mean parameters are adapted and the other parameters are taken from the UBM. The bottom layer uses the ability of a left-right SCHMM in order to capture the text-dependent information. Particularly the SCHMM takes into account the TSI of the user-customised passwords. Each of the SCHMM states is a GMM derived from the corresponding middle level model. The transformation function works only on the weights of the GMMs, the other parameters are directly taken from the middle level model.

Training Step

The training algorithm is decomposed into three steps, corresponding to each level of the EBD architecture:

• The upper level UBM is firstly trained to model the largest part of the acoustic space. It is built off-line using a large amount of data. The UBM is trained with a classical EM/ML algorithm as described in [START_REF] Bimbot | A tutorial on text-independent speaker verification[END_REF], [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF].

• The text-independent speaker model training consists in adapting the UBM/GMM with the whole available data pronounced by this client. An energy labelling is performed on the signal and only the speech frames are kept.

Using the selected frames, the final model is obtained by adapting the means of the UBM thanks to an EM algorithm with the MAP criterion. • In order to catch the TSI of the password, a SCHMM is learnt. Contrary to the speaker dependent GMMs of the second stage, all frames of the password -and not only speech frames -are used to adapt the S states of the SCHMMs. The password sequence is cut in S segments {segi} of the same length. Each state i of the SCHMM is adapted from the speaker text-independent model using segi to modellise a subpart of the password. An EM/MAP algorithm is applied on only the weight parameters. Then the SCHMM is optimised thanks to a classical Viterbi algorithm (a new segmentation is achieved by Viterbi and is used to adapt the state models). The number of states, S, of the SCHMM is experimentally determined. Finally, the transitions of the SCHMM are computed using the relative length of each segment.

Testing Step

During a test phase, the score between the input signal and a password SCHMM model is issued from the corresponding Viterbi path. All the input frames are used during this Viterbi decoding phase. The log-likelihood of a frame sequence could be expressed as a sum of two log-likelihood accumulations, one using speech-labelled frames and the other using non-speech-labelled frames, as shown in expression 1.

log p(X|λ) = log p(X speech |λ) + log p(X non-Speech |λ) (1)
The final speaker matching score corresponds to the loglikelihood computed with only the log p(X speech |λ). This score is normalised by the log-likelihood computed using the UBM model on the same speech-labelled frames.

Respect of the Resource Constraints

In order to evaluate our approach in terms of memory constraints and computational cost, the EBD system is compared to baseline systems. The memory occupation of the EBD system is evaluated by considering the number of stored parameters. Indeed this paper does not consider data compression and coding which are both mandatory for a such embedded application. In this context the memory estimation in terms of megabytes does not make sense. The computational cost is estimated from the number of operations associated with the function f (y, G), where f (y, G) is the log-likelihood computation function for an acoustic feature y and a monovariate Gaussian G.

The EBD system lies at the limit between speaker and speech recognition systems and could be compared with two systems working in parallel. We use this decomposition and we compare our system to both a classical speech recognition engine and a classical speaker recognition system.

Speaker Recognition

As described in 2.1 the EBD system has a hierarchical architecture. The first and the second layers of this architecture present the same structure as a classical text-independent speaker recognition system. This part of the EBD system has then the same characteristics in terms of computational cost and memory resources as this standard GMM/UBM system. As in classical approaches the memory and computational resources are already minimized by tying parameters between the UBM and the speaker models and by computing only the n-top Gaussian. Considering a system with 128 Gaussian components per model, 32 acoustic coefficients and 5 speaker models, the number of stored parameters is 28 800 and for each input frame, 24 576 log-likelihood computations are done. We adapt all the mean parameters in this work, an additional gain is possible in both memory and computing by adapting only a subpart of the parameters or with a Gaussian selection [START_REF] Auckenthaler | Gaussian selection applied to text-independent speaker verification[END_REF].

Speech Recognition

The main benefit comes from the speech recognition part of the EBD system. As the two first layers are equivalent to a UBM/GMM speaker recognition system, the third layer should be compared to a speech recognition system. The main constraint of this application comes from the possibility of the users to choose their own password without any limitation. The system used should also be vocabularyunconstrained. State-of-the-art speech recognition systems are mainly related to statistical methods like Hidden Markov Model and phonemic models [START_REF] Lévy | Mobile Phone Embedded Digit-Recognition[END_REF]. In this sense, two approaches are compared with the EBD system. The first one is a phonemic based approach using noncontextual models which requires significant memory resources.

For an acoustic model with non-contextual phonemes there are 128 distributions per state, each with 32 acoustic coefficients. The acoustic model contains 108 emitting states. The number of trained passwords has a negligible effect and the memory occupation in terms of parameter numbers could be approximated by only the acoustic model size:

nbes

× nbg × (2 × vectsize + 1) oneGaussian ( 2
)
where nbes is the number of emitting states, nbg is the number of Gaussian components and vectsize is the number of acoustic features. The computational cost of this part is a supplement to the speaker recognition task. The likelihood of each distribution of the acoustic space model is computed for each acoustic feature of the test data. For the system previously described, the computational supplement consist in computing 13, 824 log-likelihood per acoustic feature.

The second approach is a global HMM for each password.

For one password trained by one speaker, we estimate the acoustic model size (in terms of number of parameters) as follows:

stnb × nbg × (2 × vectsize + 1) oneGaussian ( 3
)
where stnb is the number of states of the HMM. The computational cost (in terms of log-likelihood number) is estimated as follows:

stnb × nbg × vectsize (4)
When linking speaker recognition and password recognition with SCHMM in the EBD system, only the weight parameters are needed for a given state, the other parameters are tied to the upper level GMM. For one password trained by one speaker, this system only requires the storage of very few parameters:

stnb × nbg × 1 (5)
The computation of the Gaussian component log-likelihoods is done in the speaker recognition part of the architecture. By adapting only the weights of the Gaussian distributions, the speech recognition task only requires weighted combinations which makes the computational cost of this system negligible compared to the baseline systems. The memory occupation for the EBD system is proportional to the number of passwords but would be very small compared to the baseline systems for the targeted application.

Table 1 shows the required resources of the 4 systems for 5 speakers, 2 passwords per speaker, 15 states per password (for the HMM and EBD system), 128 Gaussian distributions and 32 acoustic parameters. (It is also possible to tie some parameters between models for each system; this optimization is not taken into account here.) The cost of the weighted means is neglected.

Table 1: Evaluation of the memory (in terms of parameters) and computational resources (in terms of log-likelihood computation) of EBD and two baseline systems per input frame

EXPERIMENTS ON VALID-DB

Valid Database

The experiments are performed on the Valid Database [START_REF] Fox | The realistic multi-modal valid database and visual speaker identification comparison experiments[END_REF]. This database contains audio-video records from 106 speakers: 77 men and 29 women. Based on the XM2VTS linguistic content, two occurrences are recorded in 5 sessions for each of these speakers. The first one -denoted in this paper DIGIT -is a sequence of digits : "5 0 6 9 2 8 1 3 7 4" and the second -denoted in this paper SENTENCE -is the same phonetically well-balanced sentence as in M2VTS "Joe took father's green shoe bench out".

The five sessions are recorded over a period of one month. The first session is a clean one (this session is recorded under controlled acoustic and illumination conditions). The other four sessions are recorded in a real world scenario without any control on illumination or acoustic noise. The Valid database was chosen as we aim to include the video stream in the system, despite its small size. However Valid presents several drawbacks for our work. The sentences are too long for a real password dedicated system and only two different sequences are recorded. Moreover, these sequences are too different (digits vs. sentence) to represent realistic passwords. Finally, the number of speakers is small for a speaker verification experiment. In order to withdraw the gender mismatch problem, only male sessions have been used in this work.

Protocol

The 77 male speakers are separated into three sets: the UBM-set with 25 speakers, the CLIENT-set with 25 other speakers, and the IMPOSTOR-set with the 27 remaining males.

As indicated by the set names, the UBM-set is used in order to train the UBM model, the Client-set is used to enrol the client speakers and for the true access and the last set, the impostor-set is only used for the impostor trials. Three background models are computed using the UBM-set of speakers. The UBM-DIGIT is learned on the 5 DIGIT sessions of the 25 ubm-speakers, UBM-SENTENCE is learned on the 5 SENTENCE sessions of the UBM-set of speakers. Finally, a third UBM model, UBM-ALL is learned with the 5 sessions of both occurrences.

The UBM models are learned using the noisy and the clean sessions but the clean sessions are no longer used in the rest of the protocol as only one clean session is available per speaker.

Starting with the three UBMs, eight conditions are defined:

• 1occ-DIGIT : in this condition, each client model is derived from UBM-Digit using one noisy DIGIT occurrence of this speaker. Due to the small number of speakers available in the database, a jacknifing process is used by learning a client model on each available noisy DIGIT occurrence. Of course, the used training occurrence will not be used for the true speaker access. This process gives 100 speaker models (25 speakers, 4 noisy DIGIT occurrences). • 1occ-SENTENCE : respectively, 100 speaker models are derived from the UBM-SENTENCE using the noisy SENTENCE occurrences. The same jacknifing process is used. • 2occ-DIGIT : each client model is now trained using two DIGIT noisy occurrences of one given speaker and the UBM-DIGIT. The jacknifing process selects all the two from four combinations in order to give six models of the original speaker and a total of 150 speaker models (25 speakers, 6 couples of noisy DIGIT occurrences). • 2occ-SENTENCE : respectively, 150 speaker models are defined using UBM-SENTENCE and the noisy SEN-TENCE occurrences. • 1occ-all-DIGIT : this condition is the same as 1occ-DIGIT but using the UBM-ALL instead of UBM-DIGIT. • 1occ-all-SENTENCE : this condition is the same as 1occ-SENTENCE but using the UBM-ALL instead of UBM-SENTENCE. • 2occ-all-DIGIT : this condition is the same as 2occ-DIGIT but using the UBM-ALL instead of UBM-DIGIT. • 2occ-all-SENTENCE : this condition is the same as 2occ-SENTENCE but using the UBM-ALL instead of UBM-SENTENCE.

The number of target accesses per condition is constant for all the above eight conditions. 300 target accesses are defined. For 1occ-X conditions, 100 target models are compared to the three available test sequences (four noisy occurrences are available for each password, one is used in order to train the speaker model, the three remaining ones for the client accesses). For 2occ-X conditions, 150 target models are compared to the two available test sequences.

The impostor tests are computed using the IMPOSTOR-set of speakers. Two configurations are proposed:

• PASSWORD: in this configuration, the linguistic content of the impostor test sequences is the same as the occurrences used to train the client models. Each speaker model of the Xocc-DIGIT and Xocc-all-DIGIT sets is compared to the four noisy DIGIT occurrences of each of the 27 speakers of the IMPOSTOR-set. Respectively, each speaker model of the Xocc-SENTENCE and Xoccall-SENTENCE sets is compared to the four noisy SEN-TENCE occurrences of each of the 27 speakers of the IMPOSTOR-set. • WRONG: in this condition, the linguistic content of the impostor test occurrences is different to the one of client models training material. Each speaker model of the Xocc-DIGIT and Xocc-all-DIGIT sets is compared to the four noisy SENTENCE occurrences of each impostor speaker. Respectively, each client model of the Xocc-SENTENCE and Xocc-all-SENTENCE sets is compared to the four noisy DIGIT occurrences of each of the 27 impostor speakers. For both configurations, the number of impostor tests is constant for a given client model. However, the global number of impostor tests performed depends on the selected client condition:

• 1occ-X and 1occ-all-X conditions are composed on 100 client models. These conditions give 10, 800 impostors accesses (27 impostor speakers, four noisy occurrences of the chosen linguistic content for each of the 100 client models). • 2occ-X and 2occ-all-X conditions give 150 client models.

It gives 16, 200 impostor accesses (27 impostor speakers, four noisy occurrences of the selected linguistic content and 150 client models). The EBD and the baseline systems are developed using the ALIZE toolkit [START_REF] Bonastre | Alize/spkdet: a state-ofthe-art open source software for speaker recognition[END_REF].

Parametrization

Mel-scaled frequency cepstral coefficients (MFCC) are used, computed every 10ms. An energy labelling is applied to separate the speech frames from the non-speech frames. Acoustic feature frames are 32-dimension vectors, 15 cepstral coefficients, the log-energy and the corresponding ∆ coefficients.

Results

Firstly, we wish to evaluate the influence of the number of components of the GMM models on the EBD system overall performance. Figure 2 shows the performance of the EBD system with five states against the number of components in the GMM, from 8 to 1, 024. The performance is evaluated in terms of EER using UBM-ALL, SENTENCE and both PASSWORD and WRONG conditions (1occ-all-SENTENCE plus PASSWORD and WRONG impostor conditions). The performance of the baseline UBM/GMM system is given for comparison. The experiment shows that a sufficient number of components is 64 to achieve a good level of performance for both systems. Due to the small amount of available training data, the performance reaches a maximum when the number of components is about 256, which is similar to the observations made by [START_REF] John | Data-model relationship in text-independent speaker recognition[END_REF]. In the configuration retained for the rest of the experiments, the other fixed parameters are:

• dimension of GMMs is fixed to 256 and all mean parameters of the text-independent speaker models are adapted. • 32 weight parameters are adapted for each SCHMM state The main expected advantage of the EBD system compared to a classical UBM/GMM is to incorporate the passwordbased information like the password itself and the relative TSI. This point is evaluated by the experiments presented in the Tables 2 and3 of the EBD system depending on the number of states and the nature of the impostor tests. The performance of the baseline GMM in the same conditions is provided for comparison. It is important to note that the GMM system is equivalent to an EBD system with only one state. Table 2 (WRONG condition) shows a clear advantage for the EBD system when the impostors do not know (pronounce) the speaker passwords. Increasing the number of states from one to 15 allows a continuous gain in terms of EER. This result is not confirmed by Table 3 where the impostors know the target speaker passwords. A loss of performance is observed when the number of states increases. It seems that, in the speaker matching score, the TSI shadows the speaker specific information, i.e. the password is recognised and not the speaker. A solution to this problem is to increase the amount of speaker specific information in the EBD model, for example by adding several training occurrences during the speaker enrolment. In order to investigate this possibility we propose a similar experiment to the previous one but with two occurrences of the speaker password during the training phase. Table 4 presents the results using the PASS-WORD condition, depending on the number of training occurrences (one or two) and on the number of EBD states. As expected, the performance improves when the amount of data increases, for all conditions. More interesting is the comparative behaviour of the GMM system and EBD system. The EBD system gains more from the increase in training data than the GMM, even if the GMM performs generally better than the EBD. This result indicates clearly that the balanced problem between text-content information and speaker specific information could be solved by adding more training occurrences during the speaker enrolment. For the WRONG conditions, the same behaviour is noticed.

CONCLUSIONS AND FUTURE WORKS

Figure 1 :

 1 Figure 1: General view of the EBD model architecture.

Figure 2 :

 2 Figure 2: EER for GMM and EBD 5-state systems against GMM size (1-occ-Sentence, both WRONG/PASSWORD impostor tests)

Universal Background Model Speaker X Speaker Y PassWord 2 State 1 State 2 State n PassWord 1 State 1 State 2 State n

	Space Model	Acoustic
	Models	Text-independent
	Models	Text-depedent

Table 2 :

 2 . These tables present the performance EER of GMM and EBD systems (with different number of states) -WRONG test configuration

	Client Set	GMM baseline	Number of states of EBD 5 10 15
	1occ-Digit	2.33	2.67	2.35	2.33
	1occ-Sentence	1.67	1.33	1.27	1.00
	1occ-all-Digit	1.30	1.67	1.32	1.00
	1occ-all-Sentence	0.38	0.08	0.02	0.03
	Client Set	GMM baseline	Number of states of EBD 5 10 15
	1occ-Digit	1.96	2.02	2.64	2.67
	1occ-Sentence	2.00	1.73	2.93	3.27
	1occ-all-Digit	1.74	3.02	3.59	3.35
	1occ-all-Sentence	2.00	2.33	3.67	4.41

Table 3 :

 3 EER of GMM and EBD systems (with different number of states) -PASSWORD test configuration

	Client Set	GMM baseline	Number of states of EBD 5 10 15
	1occ-Digit	1.96	2.02	2.64	2.67
	2occ-Digit	1.00	1.00	1.00	1.27
	1occ-Sentence	2.00	1.73	2.93	3.27
	2occ-Sentence	0.73	0.42	1.00	1.28
	1occ-all-Digit	1.74	3.02	3.59	3.35
	2occ-all-Digit	1.05	1.67	1.95	1.68
	1occ-all-Sentence	2.00	2.33	3.67	4.41
	2occ-all-Sentence	0.92	0.73	1.34	1.99

Table 4 :

 4

Comparison of EER with one and two training occurrences -WRONG test condition

We present a new acoustic architecture for a passworddependent speaker recognition dedicated to embedded applications. The proposed approach associates the advantages of a text-independent GMM/UBM system with an HMM/Viterbi-based text-dependent system. It follows both the ergonomic constraints like the small amount of training data and the computing resource constraints. The preliminary experiments demonstrate the password recognition abilities of EBD, even if a very small amount of training data is provided. Even if a large decrease in EER was observed for several experimental conditions compared to a GMM, the results have to be validated on a larger database as Valid is limited for a speaker recognition task. Future work will focus on the TSI by exploring the ratio between passwords and the number of states of the EBD model. We aim also to incorporate TSI from a second modality like the video stream in order both to increase the performance and in an attempt to thwart replay attacks.


Jean-Franc ¸ois Bonastre

Nicolas Scheffer

Driss Matrouf

Corinne Fredouille

Anthony Larcher

Alexandre Preti

Gilles Pouchoulin

Nicholas Evans

Benoît Fauve

John Mason

ALIZE/SpkDet: a state-of-the-art open source software for speaker recognition

This paper presents the ALIZE/SpkDet open source software for text independent speaker recognition. This software is based on the well-known UBM/GMM approach. It includes also the latest speaker recognition developments such as latent factor analysis and unsupervised adaptation. Discriminant classifiers such as SVM supervectors are also provided, linked with the nuisance attribute projection. The software performance is demonstrated within the framework of the NIST'06 SRE evaluation campaign. Several other applications like speaker diarization, embedded speaker recognition, password dependent speaker recognition and pathological voice assessment are also presented.

Introduction

Indicated by the growing number of participants in the international NIST speaker recognition evaluations (SRE) [START_REF] Martin | The NIST speaker recognition evaluation series[END_REF], textindependent automatic speaker verification (ASV) has experienced an increasing interest in recent years. At the same time meaningful improvements in performance have been achieved with error rates roughly halving over the last three NIST SRE campaigns. These two phenomena are not directly linked since the best performances invariably come from a limited number of sites, mostly with previous experience in such campaigns. And any exceptions to this observation tend to relate to site combinations where perhaps one or more of the partners have previous SRE experience. The key point here is the high level of research and technology investment required to reach and remain close to the ever moving state-of-the-art. In this context, combined effort across sites can clearly help; the simplest and most common example of this is the sharing of system scores with score-level fusion. At the other extreme of such cooperation is open source software, the potential of which is far more profound.

This paper presents an open source software: the AL-IZE/SpkDet packages. The realization has taken place in the context of the open source ALIZE toolkit [START_REF]ALIZE: open tool for speaker recognition[END_REF][START_REF] Bonastre | Alize, a free toolkit for speaker recognition[END_REF] developed by the LIA in the framework of the French Research Ministry Technolangue 1 programme. ALIZE comes from the collaboration done around the ELISA consortium [START_REF] The | The ELISA consortium. the ELISA systems for the NIST'99 evaluation in speaker detection and tracking[END_REF], which grouped the efforts of several European laboratories in order to participate in NIST evaluation campaigns (mainly NIST-SRE and NIST Rich Transcription evaluation) from 1998 to 2004. ALIZE/SpkDet is a package within ALIZE tailored specifically to ASV; AL-IZE/SpkDet was one of the chosen reference systems in the framework of the BioSecure Network of Excellence 2 ; it was 1 http://www.technolangue.net/ 2 http://biosecure.info also used by several institutions for NIST'06 SRE. Feature extraction comes from SPro [START_REF] Gravier | SPro: speech signal processing toolkit[END_REF]. All these systems are distributed through open source licences.

Recent key developments include support vector machines (SVMs) [START_REF] Campbell | SVM based speaker verification using a GMM supervector kernel and NAP variability compensation[END_REF], the associated nuisance attribute projection compensation (NAP) [START_REF] Solomonoff | Advances in channel compensation for SVM speaker recognition[END_REF], and factor analysis (FA) [START_REF] Kenny | Eigenvoices modeling with sparse training data[END_REF][START_REF] Kenny | Factor analysis simplified[END_REF]. These developments stem mainly from a collaboration between the LIA and the University of Wales.

This paper presents an overview of the ALIZE/SpkDet software. After a short technical description in Section 2, Section 3 presents a demonstration of the toolkit performance, using the latest NIST SRE database. In Section 4, some work concerning unsupervised adaptation of the client models are described and evaluated using the NIST'05 and '06 SRE framework. Even if the software is mainly designed for text-independent speaker recognition, several other applications use ALIZE/SpkDet. Section 5 presents some examples of these developments. Finally, Section 6 concludes with ideas on the immediate future of the project.

Overview of ALIZE/SpkDet

The ALIZE project was initiated by the ELISA consortium under the responsability of the LIA and initially funded by the French Research Ministry Technolangue program during 2003/2004. The project was continued using LIA and BioSecure funds. ALIZE was also retained for two French National Research Agency (ANR) projects, BIOBIMO and MISTRAL, allowing support until the end of 2008.

ALIZE main objectives

The main objectives of ALIZE are:

• to propose a toolkit facilitating the development of new ideas, with a (proven) state-of-the-art level of performance;

• to encourage laboratories to evaluate new proposals using the toolkit on both standard databases and protocols such as the NIST SRE evaluations;

• to help the understanding of speaker recognition algorithms (like EM training, MAP adaptation or Viterbi algorithm), parameters and limits;

• to test new commercial applications;

• to facilitate the exchanges and knowledge transfer between the academic laboratories and between academic laboratories and companies.

Thought the MISTRAL and BIOBIMO projects, some new functionalities are currently developed, like client/server or embedded architectures, text dependent speaker recognition systems and multimodal user authentication systems.

An illustration of ALIZE specificities

ALIZE is open source software developed in C++ following an object oriented UML method. The general architecture of the toolkit is based on a split of the functionalities between several software servers. The main servers are the feature server, which manages the acoustic data, the mixture server which deals with the models (storage, modification, tying of components, saving/reading...), and the statistics server which implements all the statistical computations (EM-based statistic estimations, likelihood computation, viterbi alignment, etc.) This architecture presents several advantages:

• Each server is accessible thanks to a very short list of high level functions and the low level functions such as memory management are usually hidden from the user;

• Each server is optimized and updated separately (thanks to this relative independence, developing new functionalities inside a new server is also very easy);

• Several instances of the same server could be launched at the same time (particularly useful for multi-stream or multimedia systems);

• The user-code presents the same structure, organized between the main servers, helping the source code development and understanding;

• Finally, the software architecture allows easily to distribute the servers on different threads or computers.

Data reading and synchronization

All the acoustic data management is delegated to the feature server. It relies on a two step procedure: feature server initialization and a reading call for accessing to each feature vector.

The reading of an acoustic stream is performed frame by frame following a feature loop as illustrated in figure 1. The user synchronizes the reading process thanks to the reading calls. Therefore, the same source code is employed for off-line (file based) processing and for on-line (open microphone) processing (the only difference occurs in the server configuration). Accessing at different time instants in the audio stream (go backward or forward, go to frame x) is also allowed. Memory management is achieved according to two simple rules:

• the user defines the size of the data buffer, in time, from unlimited to one frame. If the buffer shows a limited size, while the user requests a frame out of the buffer, the server will return a 'frame non available message' (the same message is sent if no frame is available during in open microphone mode).

• the user defines the memory footprint for the data server. An integrated buffering service is provided for accessing very large datasets (several hundred hours of speech).

EM/ML world model training

For training a GMM world model, the user has to initialize three servers, the feature server for the data, the mixture server for managing the Gaussian components (and mixture models) and the statistics server for estimating the statistics. Figure 2 shows

EM/MAP speaker model estimation

For deriving a speaker model from the world model using a MAP adaptation algorithm, the user builds exactly the same program as the previous one. Only two differences have to be highlighted: the client model is initialized as a copy of the world model and the final model (at each iteration) is the result of MAP(), a function involving both the world model (the a priori knowledge) and the statistics estimated on the client training data. Implementing some variants of the MAP algorithm will only take place in this MAP() function. This process is illustrated Figure 3.

Score computation

The score computation follows the same structure as the two previous programs. Figure 4 shows an example of the (log) likelihood computation for several client models, using a n-top Gaussian computing. For top Gaussian computing, the user needs only to set an optional flag (DE-TERMINE TOP DISTRIBS) during the corresponding call for memorizing the winning components and to set the flag to a different value (USES TOP DISTRIBS) for using it for the other calls. An implicit component tying is also implemented and helps to save both computational time and memory.

Discriminant classifiers

Discriminant classifiers like the SVM were proposed during the past years in several works as in [START_REF] Mariéthoz | A kernel trick for sequences applied to text-independent speaker verification systems[END_REF][START_REF] Wan | Speaker Verification Using Support Vector Machines[END_REF][START_REF] Campbell | Support vector machines for speaker and language recognition[END_REF][START_REF] William | Support vector machines using GMM supervectors for speaker verification[END_REF]. These classifiers are usually applied to GMM supervectors. A GMM supervector is composed of the means of a classical GMM system, as initially proposed by [START_REF] William | Support vector machines using GMM supervectors for speaker verification[END_REF]. Libsvm library3 is used for the basic SVM functionalities.

Session variability modeling

Some of the most important developments in ASV over recent years relate to strategies that address the "mismatch factor". ALIZE/SpkDet includes a set of functionalities following the Factor Analysis proposed by [START_REF] Kenny | Eigenvoices modeling with sparse training data[END_REF] and the Nuisance Attribute Projection proposed by [START_REF] Solomonoff | Advances in channel compensation for SVM speaker recognition[END_REF]. In these approaches the goal is to directly model the mismatch rather than to compensate for their effects as it was done with H-norm and T-norm. This involves estimating the variabilities from a large database in which each speaker is recorded in multiple sessions. The underlying hypothesis is that a low dimensional "session variability" subspace exists with only limited overlap on speaker specific information. Both Factor Analysis and NAP were developed inside AL-IZE/SpkDet, using the SVDLIBC4 toolkit for the singular value decomposition.

Performance of the software

Experimental protocol

The male part of the NIST'05 primary task (1conv4w-1conv4w) is used for development (DevSet). For this condition, one side of a 5-minute long conversation is available for testing and the same amount for training. All background training data, for the universal background model (UBM), T-norm [START_REF] Auckenthaler | Score normalization for text-independent speaker verification systems[END_REF], NAP, and FA come from the NIST'04 database. This procedure leaves the NIST'06 free for validation. The final comparisons are made on the NIST'06 core (required) condition which includes multiple languages (rather than the English only common condition). Performances are assessed using DET plots and measured in terms of equal error rate (EER) and the minimum of the decision cost (minDCF). The cost function is calculated according 4) For a given frame and using the world model, determine the top gaussian, memorize the individual component likelihood and compute/accumulate the world likelihood, (5) For the same frame, using the top components and the memorized values, compute and accumulate the likelihood for each client, ( 6) and ( 7) Get the mean log likelihood and compute the LLR per client.

to NIST criteria [START_REF] Martin | NIST speaker recognition evaluation chronicles[END_REF].

Description of the systems

This paragraph presents several systems in order to illustrate the large scale of techniques embedded in the software as well as the level of performance achieved using ALIZE/SpkDet (more details on the different systems are in [START_REF] Fauve | State-of-the-art performance in text independent speaker verification through open source software[END_REF]):

• GMM baseline (GMM). A classical GMM system is developed using ALIZE/SpeakerDet. The features are based on 19 linear filter-bank derived cepstra (computed using the open source SPro toolkit [START_REF] Gravier | SPro: speech signal processing toolkit[END_REF]). The feature vector is composed of 50 coefficients, 19 static coefficients, 19 delta and 11 delta-delta and the delta energy. A classical energy-based frame pruning system is applied before to normalize the recordings, file-by-file (cepstral mean subtraction and variance normalization). Feature mapping is also applied. The UBM model size is 512 (with diagonal covariance matrices) with the speaker models adapted from this UBM via a MAP procedure. This GMM baseline system is also used for the other systems.

• Gmm Supervector Linear kernel (GSL). The GSL system uses a SVM classifier applied on GMM supervectors. The supervectors are taken directly from the GMM baseline system, giving in our case a vector size of 512*50.

• Gmm Supervector Linear kernel + Nuisance Attribute Projection (GSL-NAP). GSL-NAP corresponds to a GSL system using the nuisance attribute projection technique [START_REF] Solomonoff | Advances in channel compensation for SVM speaker recognition[END_REF] in order to deal with intersession variabilities

• Symmetrical latent Factor Analysis (SFA). Recently new approaches have been proposed by Kenny [START_REF] Kenny | Eigenvoices modeling with sparse training data[END_REF] with factor analysis (FA) in a generative framework. This approach was implemented into our software using an original symmetrical approach presented in [START_REF] Matrouf | A straightforward and efficient implementation of the factor analysis model for speaker verification[END_REF] 0.1 0.2 0. 1 for all the independent systems, for the development set and for the differrent validation sets (issued from NIST-SRE 2006). The results confirm that NAP and SFA techniques bring a significant improvement compared to the classical GMM system. Compared to the official results of NIST SRE 2006, available in [START_REF] Przybocki | NIST speaker recognition evaluation chronicles -part 2[END_REF], it is clear that ALIZE/SpkDet are among the few systems able to achieve an EER in the region of 5%.

Figure 5 shows the results of an unweighted linear fusion between the 2 best systems namely SFA and GSL-NAP 5 . The improvement shows their complementarity, even if the two systems share the same acoustical representation, the same GMM parameters (UBM) and the same session variability parameters.

Unsupervised adaptation

A classical solution in order to improve the performance of a speaker recognition sysstem is to increase the amount of information used to train the client model. However, this solution depends on the availability of such training data. Unsupervised adaptation of the client models is a good solution to this data availability problem. In unsupervised adaptation, the client model is updated online, using data gathered from the test trials. The different approaches to unsupervised adaptation proposed in the literature rely mainly on a decision step to decide if a test trial belongs to the targeted speaker 5 Tnorm is applied on both system scores before the fusion [START_REF] Barras | Unsupervised online adaptation for speaker verification over the telephone[END_REF][START_REF] Van Leeuwen | Speaker adaptation in the nist speaker recognition evaluation 2004[END_REF][START_REF] Hansen | Supervised and unsupervised speaker adaptation in the nist 2005 speaker recognition evaluation[END_REF][START_REF] Preti | A continuous unsupervised adaptation method for speaker verification[END_REF][START_REF] Heck | Unsupervised on-line adaptation in speaker verification: Confidence-based updates and improved parameter estimation[END_REF]. If the test trial is considered as client, it is used to either retrain the corresponding client model or to adapt it. The main drawback of such techniques remains the difficulty to set a decision threshold for selecting the trials: the performance gain relies on the number of client test trials detected, a false acceptance (an impostor trial is accepted as a client one) degrades the speaker model. The work presented in this Section was done in collaboration with Thales Communication and is presented in detail in [START_REF] Preti | Confidence measure based unsupervised target model adaptation for speaker verification[END_REF]. It aims to avoid the decision threshold problem by negativing the need for this early decision step. In this approach, the speaker model adaptation is applied to each test trial, even if it does not belong to the client but to an impostor. This "continuous adaptation" method relies on a confidence measure on the tying between the test trial and the target speaker, used to weight each test information during the speaker model adaptation process.

Confidence measure estimation

The confidence measure is the estimation of the a posteriori probability of a test trial belonging to a given target speaker. This a posteriori probability is computed using two score models, one for the client scores and one for the impostor scores. Each score distribution is modelled by a 12 component GMM learned on a development set. The confidence measure is then computed using the WMAP approach [START_REF] Preti | A continuous unsupervised adaptation method for speaker verification[END_REF][START_REF] Fredouille | Bayesian approach based-decision in speaker verification[END_REF]. To avoid the problem of the re-estimation of the WMAP function after each adaptation step, we use only the initial target model, learned on a single session recording, to compute the score of the test trials.

Proposed adaptation function

The proposed adaptation function relies on the classical MAP algorithm [START_REF] Bimbot | A tutorial on text-independent speaker verification[END_REF], where only the mean parameters are updated. The empirical statistics are gathered from all the available data using the EM algorithm (initialized with the background model and maximizing the ML criterion). The statistics are then combined using the following rules:

• The statistics gathered from the initial voice excerpt used to train the target speaker model is associated to a confidence measure equal to 1;

• The statistics gathered from the different test trials are associated with the corresponding confidence measure;

• The empirical means and the corresponding occupancies are computed for each Gaussian component of the GMM, using all the EM statistics weighted by the corresponding confidence measures.

Finally, the adapted means (µ i map ) for each Gaussian component (i) are computed using the background means (µ i ubm ), the empirical means(µ i emp ) and the occupancy values (ni) using the classical MAP formula.

Experiments and results

All the experiments presented here are performed based upon the NIST'05 and '06 databases, all trials (det 1), 1conv-4w 1conv-4w, restricted to male speakers only. The baseline GMM system is the one described in Section 3. In addition, a version of the latent factor analysis is used at the feature level: in this case, the channel compensation is applied to each feature sets instead of the feature mapping. Figure 6 presents the results for the adapted system and the baseline on the NIST'05 database, for feature mapping and the LFA channel compensation techniques. The results demonstrate the potential of the proposed method as it reaches a significant 27% DCF relative gain (and 37% in terms of EER) with the feature mapping (FM). When the LFA-normalized features are used, the DCF gain is about 20% (and 12.5% for the EER). The gain is smaller in this case, as expected because LFA is known to perform better channel compensation than FM, reducing the influence of the unsupervised adaptation on the channel effects.

In Figure 7, we attempt to analyze more precisely the behavior of our method. This figure shows the performance in terms of min DCF of the adapted system for each newly added test trial (n denotes the number of test trials added to the system) 6 .

Results for the adapted system and the baseline are provided in Figure 8, for the NIST'06 database and using both FM and LFA. The results are disappointing compared to those of NIST'05. On this database, the unsupervised adaptation method introduces a significant loss. Different factors could explain this • When looking at the target and impostor score distributions of the baseline system we observe that a large part of the errors comes from a small percentage of the impostor trials which obtained a very high score. This phenomena is significantly higher for the NIST'06 than for the NIST'05 database. It shows the obvious dependence between the baseline system and the adaptation process behavior.

Other applications 5.1. Embedded password dependent speaker verification

For embedded speaker recognition, the main issues are usually the memory and computational constraints. But realistic embedded applications generally require short utterances, few training data and very noisy environments, giving degraded speech quality. In order to deal with these application constraints, we moved from a text-independent speaker recognition scenario to a text-dependent authentication using passwords that were chosen by the clients. This section presents the LIA EBD package based on ALIZE and dedicated to such embedded applications.

Description

Our approach merges a state of the art GMM/UBM system and a semi-continuous HMM architecture, following the ideas developed in [START_REF] Bonastre | Gaussian dynamic warping (gdw) method applied to text-dependent speaker detection and verification[END_REF] with a DTW algorithm. Hierarchical three-step models are created where a lower level model takes some of the parameters from the immediate upper level model. The top model is a classical UBM, the second one is a quite classical text-independent speaker model. It is adapted from the UBM, updating only the mean parameters using a MAP procedure. In LIA EBD, only few UBM components are adapted and saved, in order to save both memory and computation time (other mean parameters, weight parameters and variance parameters are taken from the UBM model). In the lower level, a semi-continuous HMM (SCHMM) is built in order to take into account the phonetic information and the duration information. Each state of the SCHMM is linked to a GMM distribution.

The SCHMM state-dependent distributions are adapted from the corresponding speaker model, moving only the weights. As for the second level model, only a few weights are adapted, the other parameters are taken from the upper level models (the corresponding speaker model and the UBM).

Database and Experimentations

The system, currently in development, is evaluated using the Valid database [START_REF] Fox | The realistic multi-modal valid database and visual speaker identification comparison experiments[END_REF] (as the final objective is to build a bimodal system: audio+video). This audio-video Database consists of five recording sessions composed of two sentences (the phonetic content is identical to that of the XM2VTS database 7 ). Four of these sessions were recorded in variable and uncontrolled environments, the latter in a clean environment. 76 male speakers are used for the experiments. The results are promising as the system seems able to exploit both speaker dependent information and password dependent information (the EER decreases from 3.09% to 0.19% when the impostors pronounce a wrong password for the embedded system. The GMM system stays at 1.17% with the wrong sentences). The level of resources (memory and computation) remains very small for the LIA EBD system. Of course, the presented results are preliminary as the database is very small (76 speakers, two sentences) but only a few system parameters have been optimized and much room remains for improvements.

GMM

Speaker diarization

The design of efficient indexing algorithms to facilitate the retrieval of relevant information is vital to provide easy access to multimedia documents. and grouping together all the segments that correspond to the same speaker. The first part of the process is also called speaker change detection while the second one is known as the clustering process. Generally, no prior information is available regarding the number of speakers involved or their identities. Estimating the number of speakers is one of the main difficulties for the speaker diarization task. An original approach, denoted the integrated approach, in which all the steps involved in speaker diarization are performed simultaneously, was developed by the LIA, using ALIZE/SpkDet software. This work was done in the framework of the ELISA consortium and both classical step-bystep and integrated diarization approaches are proposed in the software, with a lot of functionalities in order to deal with differrent natures of audio files (conversational recordings, broadcast news or meeting information). This work is detailled in [START_REF] Meignier | Step-by-step and integrated approaches in broadcast news speaker diarization[END_REF]. Table 3 presents a summary of the results obtained between 2002 and 2004 on different types of data, in the framework of the NIST evaluations (mainly on Rich Transcription).

More recent results on meeting data were reported in [START_REF] Fredouille | Technical improvements of the e-hmm based speaker diarization system for meeting records[END_REF][31].

Pathological voice assessment

In the medical domain, assessment of the pathological voice quality is a sensitive topic, involving multi-disciplinary domains. This task is usually realized by a jury composed of experts. The work presented in this paragraph proposes to rely on an automatic system for pathological voice assessment, more precisely for dysphonic voices assessment. The proposed method relies on an automatic system derived from the ALIZE/SpkDet automatic speaker recognition technology [START_REF] Fredouille | Application of automatic speaker recognition techniques to pathological voice assessment (dysphonia)[END_REF][START_REF] Pouchoulin | Characterization of pathological voices (dysphonia) in the frequency space[END_REF][START_REF] Pouchoulin | Frequency study for the characterization of the dysphonic voices[END_REF].

Basically, the task consists in labelling a recording or a part of a recording with a label between 0 and 3 (the G parameter of the Hirano's GRBAS scale [START_REF] Hirano | Psycho-acoustic evaluation of voice: Grbas scale for evaluating the hoarse voice[END_REF]). Table 4 presents the general recognition results (given that a strict classification protocol was used, with a clear separation between training and testing set). The obtained results demonstrate the possibilities of this approach, even if the task is very difficult, due to its medical nature, the variability of the data (and the small number of data available) and the difficulty in obtaining a ground truth (done here by an expert jury). the voice depending on the G value given by the jury of experts; the columns represent the G values determined by the system recognition group. The basics of the project (open source, easy to understand, assessment during the NIST international evaluation campaigns) remain unchanged once the years.

Conclusion and future

The latest discrimant approaches and channel compensation functionalities were added to the toolkit during the past year leading to state-of-the-art speaker recognition performance.

The software is now regularly used by about 20 laboratories in the world, and support is provided to all the developers. The software is dedicated to text-independent speaker recognition but it can also be used for a large range of applications.

In addition to the applications presented in this paper, AL-IZE/SpkDet software is also used for several other areas, within LIA (face and fingerprint recognition, emotion detection, topic detection for text processing, etc.) or by other users (for example signature/multimodal recognition [START_REF] Kryszczuk | Reliability-based decision fusion in multimodal biometric verification systems[END_REF]). An important effort will also be dedicated to the documentation.

The main challenge to the ALIZE project is not at the technical level but is to propose a new concept inside the evaluation paradigm: the participation to the international evaluation campaign is viewed as a competition and quite in opposition to the open source idea. Regarding this challenge, it seems that the ALIZE project is a success. In order to emphasize this aspect, we wish to propose inside the ALIZE project some physical or virtual evaluation campaign specific tutorials, brainstorming and practical sessions.

Figure 1 :

 1 Figure 1: Acoustic data management in ALIZE. (1) Server init., (2) Server reset, (3) Reading call in the reading loop, (4) Verify if a frame is available.

Figure 2 :

 2 Figure 2: EM based world model training. (1) Servers init., (2) Init. the world model, (3) EM it. loop, (4) Reset the stat. accumulator, (5) Feature reading loop, (6) Accumulate the stat., (7) Get the stat and copy it into the world model, (8) Save the model.

Figure 3 :

 3 Figure 3: Client model estimation by MAP algorithm. (1) Load the world model, (2) Create the client model by world duplication, (3) Get the stat on the client training data, (4) Estimate the resulting model as a function between a priori knowledge (world) and the current stat, copy it into client model.

Figure 4 :

 4 Figure 4: LLR score computation with n top gaussian computing. (1) Load the models, (2) and (3) Reset the LLK accumulators for the world model and for each client model, (4) For a given frame and using the world model, determine the top gaussian, memorize the individual component likelihood and compute/accumulate the world likelihood, (5) For the same frame, using the top components and the memorized values, compute and accumulate the likelihood for each client, (6) and (7) Get the mean log likelihood and compute the LLR per client.

Figure 5 :

 5 Figure 5: Det plots for GSL-NAP, SFA and their unweighted linear fusion, rank=40, T-norm (devSet)

False

  Alarms probability (in %) LIA: NIST 2005 det1 1conv4w-1conv4w (dcf:2.22,eer:5.12) adapted.classical.features (dcf:1.71,eer:3.57) adapted.LFA.features (dcf:3.04,eer:8.11) baseline.classical.Features (dcf:2.13,eer:3.98) baseline.LFA.Features

Figure 6 :

 6 Figure 6: Results for the adapted/baseline systems, NIST'05

Figure 7 :

 7 Figure 7: Step by step DCF for the two different sets of features on NIST'05

Figure 8 :

 8 Figure 8: Results for the adapted/baseline systems, NIST'06

  The main part of the development was done by the LIA team (thanks to several French Research Ministry funds) until 2005/2006. Recently, several external (not from the LIA) contributions have been proposed. In order to respond to the requests of the ALIZE/SpkDet developer community, the project has been migrated from a quite closed software engineering platform to a classical open-source open architecture.

3.3. Performance on NIST SRE 2006 database

		male 05	male 06	all 06
		DCF EER	DCF EER	DCF EER
	GMM baseline	3.37	8.67	3.94	8.47	4.04	9.14
	GSL	2.79	8.02	3.37	6.88	3.35	7.20
	GSL-NAP	1.62	5.28	2.07	4.33	2.26	5.02
	SFA	1.94	4.38	2.17	4.78	-	-

Table 1 :

 1

Performance in EER(%) and minDCF(x100) for the GMM, GSL, GSL-NAP and latent factor analysis GMM (SFA) on 05 development set, male'06 and male and female combined '06 validation sets Performance values are presented in Table

Table 2 :

 2 Performance in EER (%) for the GMM and LIA EBD on the Valid database with different test sets, male only.

			LIA EBD
	Passwords	1,81	3,09
	Passwords &	2,50	3,00
	Wrong Sentences
	Wrong Sentences	1,17	0,19
			The
	experiments are performed by computing 100 client tests, 3275
	impostor tests with the same password pronounced by the im-
	postor cohort (the tests are referred to as Passwords) and 3275
	tests with the second sentence pronounced by the impostor co-
	hort (referred to as Wrong Sentences)
	5.1.3. Comments

Table 4 :

 4 This paper presents the ALIZE/SpkDet open-source software. The ALIZE project began in 2003 inside the ELISA consortium and under the supervision of LIA laboratory speaker Pathological voice assessment results. The rows separate

		G=0 G=1 G=2 G=3
	Speakers with a G=0	19	1	0	0
	Speakers with a G=1	2	12	4	2
	Speakers with a G=2	2	5	11	2
	Speakers with a G=3	0	1	4	15

http://www.csie.ntu.edu.tw/∼cjlin/libsvm

http://tedlab.mit.edu/∼dr/SVDLIBC/

When the target models are updated using a new test trial, the entire test is recomputed, which differs from the NIST protocol where only the current trial and the next trials scores are computed using the new models.

www.ee.surrey.ac.uk/Research/VSSP/xm2vtsdb/