Armando Muscariello

Guillaume Gravier
email: ggravier@irisa.fr

Frédéric Bimbot
email: bimbot@irisa.fr

Zero-resource audio-only spoken term detection based on a combination of template matching techniques

Keywords: spoken term detection, template matching, unsupervised learning, posterior features

Spoken term detection is a well-known information retrieval task that seeks to extract contentful information from audio by locating occurrences of known query words of interest. This paper describes a zero-resource approach to such task based on pattern matching of spoken term queries at the acoustic level. The template matching module comprises the cascade of a segmental variant of dynamic time warping and a self-similarity matrix comparison to further improve robustness to speech variability. This solution notably differs from more traditional train and test methods that, while shown to be very accurate, rely upon the availability of large amounts of linguistic resources. We evaluate our framework on different parameterizations of the speech templates: raw MFCC features and Gaussian posteriorgrams, French and English phonetic posteriorgrams output by two different state of the art phoneme recognizers.

Introduction

A popular strategy for mining useful information from speech data sets consists in localizing occurrences of known words of interest within the data set, a task known as spoken term detection or keyword spotting. In an era dominated by growing rates of digital media creation and diffusion, keyword spotting plays a prominent role as an indexing tool operating on the audio part of a multimedia database. State of the art approaches rely on automatic speech recognition (ASR) paradigms, whose founding principles are deeply rooted into statistical methods for training complex acoustic and language models. These techniques often require large amounts of annotated speech data, even language and topic specific, and long off-line procedures for performing reliable training estimates. On the one hand, the increase in computational power and the availability of quality training data have largely justified these supervised approaches, that have indeed proven to be very effective and accurate [START_REF] Miller | Rapid and accurate spoken term detection[END_REF]. On the other hand, the drawbacks limiting their attractiveness are also notable. First, performance of supervised systems are all tightly related to the quality and quantity of training data. And the annotation of such data often implies the direct intervention of human experts, a process that is expensive, time consuming and error prone. If the ultimate goal is to automatize the archiving and management of large data sets, conceiving solutions demanding in turn high levels of supervision might seem counterintuitive. Moreover, it is well known that for many languages, indeed for most of the about 7,000 languages and dialects spoken in the world, rich and valuable linguistic resources are scarce or absent.

In the recent years, a few solutions have been proposed that limit as much as possible resorting to prior knowledge in the form of acoustic and linguistic resources. In [START_REF] Zhang | Unsupervised spoken keyword spotting via Segmental DTW on Gaussian Posteriorgrams[END_REF], a keyword spotting framework is proposed, that relies on dynamic time warping (DTW) distances among speech templates obtained by modelling speech through a Gaussian mixture model (GMM), trained without supervision. The same task is carried out in a very similar way in [START_REF] Hazen | Query-by-example spoken term detection using phonetic posteriorgram templates[END_REF], although speech is modeled by posterior features produced by an independently trained phoneme recognizer.

In this paper, we describe one such zero resource approach, where a query sample of a given keyword is directly searched on the acoustic data set by template matching techniques, without any prior knowledge, or preliminary, off-line training of model parameters. We claim two main contributions of the present work: first, the computational architecture, that combines the scores of a segmental variant of DTW and self-similarity matrix (SSM) comparison between speech sequences, previously applied in word discovery tasks [START_REF] Muscariello | Towards robust word discovery by self-similarity matrix comparison[END_REF]. Second, a thorough evaluation of the performance on a 4h subset of the French ESTER corpus with respect to different types of speech features: melfrequency cepstral coefficients (MFCCs), Gaussian posterior features trained without any supervision [START_REF] Zhang | Unsupervised spoken keyword spotting via Segmental DTW on Gaussian Posteriorgrams[END_REF], English HMMbased phone posterior features, and French phonetic posteriorgrams. We show how the additional use of the SSM technique benefits performance for all the feature types with respect to the DTW-based system alone, except for the French posteriorgram case. For this last case, the high level of phonetic knowledge incorporated in the training procedure, performed on the same ESTER corpus, successfully captures the speech variability of the database, making SSM comparison superfluous, if not detrimental to performance. While phonetic posteriorgrams require prior linguistic knowledge to train acoustic models, we justify their use to evaluate the system against the effectively unsupervised (zero-resource) approach implied by the use of MFCCs or Gaussian posteriorgrams.

The remainder of this paper is structured as follows: in Section 2 the architecture of the system is described, by presenting the pattern matching techniques employed and the speech features examined, while in Section 3 the experimental evaluation is thoroughly detailed.

The keyword spotting architecture

Template-based spoken term detection relies on two crucial components: the feature extraction and the pattern matching step used to compare speech segments. We tackle these two complementary aspects in this section, by first describing the features and the metric used to measure the pairwise distance of feature vectors.

Feature extraction

Two main type of features are evaluated in our experiments, namely a) posteriorgrams [START_REF] Aradilla | Posterior features applied to speech recognition tasks with user-defined vocabulary[END_REF] each class represents the state of an HMM modelling a language specific phone. Each posterior vector is a probability vector, i.e., its entries sum up to 1. The probability of two vectors p and q drawing from the same distribution is defined by their dot product p • q, that can be assumed as a measure of closeness. To map these scores into the distance-like range [0, ∞[, the distance between two posterior vectors is computed as -log(p • q). To avoid zeros while computing the log-probability, each vector is smoothed by zeroing all entries pi below a threshold Pmin, then distributing a small portion of the mass probability from the non-zero entries to the zeroed ones (see [START_REF] Zhang | Unsupervised spoken keyword spotting via Segmental DTW on Gaussian Posteriorgrams[END_REF]).

To score the dissimilarity of MFCC vectors, instead, we make use of the Euclidean distance.

Given account of the parameterization of the signal, we are ready to describe the pattern matching techniques employed.

Template matching techniques

The search for a match of the given query within the test utterance, raises two issues: the primary issue consists in locating the subsegment of the utterance most likely to represent a repetition of the query. The second problem amounts to providing a quantitative measure (i.e., a score) of the (dis)similarity between the query and the extracted segment.

In the system proposed, the first duty is assigned to a segmental variant of DTW, called segmental locally normalized DTW (SLNDTW) [START_REF] Muscariello | Audio keyword extraction by unsupervised word discovery[END_REF]. This technique also provides a score that can be used to directly decide on the similarity of the templates, or that can be further combined with additional measures that might better model speech variability and improve scoring.

DTW-based matching

Given the query u M i=1 and the test utterance v N j=1 , the goal is to detect the segment m = v je js most similar to u. SLNDTW solves the task by a three-stage procedure operating on the distance matrix between u and v, which is a structure gathering the dissimilarity score d between any pair of feature vectors from u and v. The three stages are respectively:

1. a starting point selection heuristic, responsible of selecting the starting points of likely matching subsegments along the first row of the distance matrix. the average distance over the L(P * ) entries absorbed in the path P * (see [START_REF] Muscariello | Audio keyword extraction by unsupervised word discovery[END_REF] for more details).

The DTW dissimilarity score DDTW between u and v is defined as DDTW = W (M, je). This same score can either be assumed as the final score S(u, v) to deem the similarity of u and v, or be further refined by the SSM-based scores, to better account for speech variability.

SSM-based matching

The SSM of a sequence u is the square symmetric matrix Φ(u) defined as

Φ (i, j) = d (u(i), u(j)).
The structure of these matrices is strictly dependent on the acoustic-phonetic content of the underlying sequences, and unlike DTW, captures the interaction between all parts of the compared patterns, encoding a richer information on their (dis)similarity, hence accounting for more variability. Indeed, many empirical observations have consistently confirmed the visual resemblance of these matrices across different conditions, that is when instances of a same word are uttered by different speakers, or undergo different channels or are imposed on a noisy background. The main argument advising against the use of SSM comparison is that words that are different at the lexical level might indeed exhibit similar SSMs. As a countermeasure, in our experiments the SSM score is combined with the DTW one, to balance the effect of the SSM score with the more reliable DTW one. The cascade of these two pattern matching methods has been proven to better deal with speech variability in word discovery tasks [START_REF] Muscariello | Towards robust word discovery by self-similarity matrix comparison[END_REF]. We describe in the following how to effectively compare the SSMs. SSM scores. To ease the comparison of the SSMs, the templates u and m are warped into u ′ and m ′ according to the path length L(P * ), to obtain SSMs of equal size. Then two different dissimilarity scores are computed:

1. D ′ SSM = Φ(u ′ ) -Φ(m ′ ) 1/L(P * ) 2 2. D ′′ SSM = V (Φ(u ′ )) -V (Φ(m ′ )) 1/L(V ) 2
where V denotes a type of visual descriptor of length L(V ) extracted from each SSM. The score D ′ SSM represents the simple entry-by-entry distance between the SSMs, but this measure is dependent on the absolute values of the SSMs and does not encode well their implicit spatial pattern. That is why the visual descriptors V , based on local histograms of oriented gradients [START_REF] Dalal | Histograms of oriented gradients for human detection[END_REF], are computed, assuming that local objects' appearances and shapes can be well characterized by the distribution of local intensity gradients and edge orientations. The score D ′′ SSM does not depend on entries (or pixels) magnitudes but rather on local gradients' magnitudes, as they measure the strength and directions of local edges. Moreover, they do not provide just a punctual information, i.e., confined to each single pixel, but are computed over dense, overlapping grids of pixels, encoding a more complex information on the self-similarity visual patterns. Scores fusion. The different scores are finally combined into the global score S(u, v) according to the following expression:

S(u, v) = αDTW • DDTW thDTW + α ′ SSM • D ′ SSM th ′ SSM + α ′′ SSM • D ′′ SSM th ′′ SSM (1)
That means each score is normalized by a proper threshold value for each pattern matching technique, and weighted by a factor so that αDTW + α ′ SSM + α ′′ SSM = 1. A schematic representation of the combination of the pattern matching techniques is provided by Fig. 1.

Experimental results

Experiments were performed on a 4h subset of the ESTER corpus [START_REF] Galliano | The ESTER evaluation campaign for the rich transcription of French broadcast news[END_REF], comprising four different French broadcast news shows, sampled at 16 KHz. The shows were recorded on the same days, at different channels, to build a data set including several speakers, likely covering similar topics, thus inducing frequent repetitions. The file has been segmented into 2,915 utterances separated by silences. From the same data set, 20 keywords with at least 10 occurrences, listed in Table 1, were considered and a unique sample for each was randomly selected as the actual query (and obviously disregarded during performance evaluation). The average number of speakers per keyword is 22.55.

Evaluation criteria

The scores between each acoustic query and each utterance are computed according to Eq. 1 and utterances are ranked accordingly in ascending order. The various threshold have been tuned, for each type of feature, on a separated set of word occurrences. The weights have been varied during different runs of the system and the respective performance are to be described in subsection 3.3. We opted for three sets of weight distribution: a) αDTW = 0.34, The evaluation is conducted on a per-utterance basis, i.e., an utterance is deemed a correct hit if containing the desired search term, regardless of its location. Moreover, a match is considered correct even if the acoustic query appears as the subword of a longer word at the lexical level.

α ′ SSM = 0.33, α ′′ SSM = 0.33, b) αDTW = 0.4, α ′ SSM = 0.2, α ′′ SSM = 0.
As for the performance indicators, we have resorted to 1) the average P@10, that is the precision over the first 10 ranked utterances, 2) the average P@N, the precision over the best N utterances (N being the occurrence count for each keyword), 3) the average MAP, the mean average precision, the mean of precision scores after each keyword occurrence is retrieved and 4) the average EER, the equal error rate, where the false acceptance and false rejection rates are equivalent.

Details on the features

All the features are extracted at a 100 Hz frame rate. The MFCCs are 39 dimensional vectors, comprising 12 MFCCs, the log energy coefficient and their first and second order derivatives, refined by mean subtraction and variance normalization. As for the posteriorgrams:

1. the Gaussian posterior features are 50 dimensional vectors obtained by a GMM with 50 components trained on the same MFCCs extracted from the data set. The number of components was decided upon the evaluation conducted in [START_REF] Zhang | Unsupervised spoken keyword spotting via Segmental DTW on Gaussian Posteriorgrams[END_REF].

2. The French state posteriorgram representation comprises 115 dimensional vectors corresponding to 38 phones, each modelled by a three-state HMM with 16 component GMM emission distributions, independently trained on the 150h ESTER corpus. Each posterior frame has been computed from the same MFCCs used for the experiments.

3. The English state posteriorgrams are output by the BUT phoneme recognizer [START_REF] Schwarz | Towards lower error rates in phoneme recognition[END_REF], with 120 classes corresponding to the states of 40 three-state tandem HMM English phone models, discriminatively trained on the TIMIT database. In this case, the posteriorgrams are computed over TRAP-based features extracted from the signal.

Results and discussion

The results of the experiments are concisely summarized in Table 2, respectively accounting for the DTW-based system (αDTW = 1) and the SSM-driven one, for the three different weight distributions examined. According to the evaluation framework described, the English phonetic posteriorgram is outperformed by all the other ones, in any configuration setting. For this type of feature, P@10, P@N and MAP barely fall above the 50% and 30% respectively in the best cases, figures significantly improved by all the other feature types in any setting scenario. The EER, instead, is comparable with the MFCC case, and, in the DTWonly system, significantly better (18.9% against 21.6%). Such disappointing results might look surprising at first, as this representation of the speech data relies on probability estimates output by a state of the art phoneme recognizer, based on long temporal context features and neural network classifiers, proven to successfully capture the speech variability over a data set by the training procedure. We suspect that the likely reason for this relatively poor performance is the application of English specific phone models and parameters to a French data set. This interpretation seems confirmed by the impressive numbers (about 90% for P@10, about 65-75% for P@N and MAP, 6-7% for the EER) yielded by the posterior features output by the French GMM-HMM recognizer, which not only is based on French specific phone models, but has also been trained on the same ESTER corpus including our test data 1 . While the English phonetic posteriorgram still produces valuable results, it does not even perform as well as the raw MFCC features (from 55% to 67% depending on score combination for P@10, 30-40% for P@N and MAP), except for slightly better EERs (around 20%), or as well as the Gaussian posteriorgrams (about 70% for P@10, 40-45% for P@N and MAP, 15% for EER).

One very interesting conclusion that can be drawn is the beneficial effect induced by the use of the SSM comparison stage, for all features but for the French phone-state posteriorgram. The aim of using this technique in addition to the DTW-based one is to enhance the robustness of the system to speech variability. Its capability to better account for variability is demonstrated by an improvement of about 12% for P@10, of 6-9% (depending on score weighting) for P@N and MAP in the MFCC case (while failing to substantially impact the EER), and a minor one for the Gaussian and English phonetic posteriorgram case (in these two cases, it is mostly the P@10 that is improved). The addition of the SSM scores appears, instead, slightly detrimental to performance in the French posteriorgram case (even if, for at least one weight distribution, this worsening might be negligible, amounting to about 2% for P@10, P@N and MAP, with the EER unchanged). The reason for this behaviour resides evidently in the modelling capability of the posteriorgram representation; the SSM matching technique does not specifically account for a particular type of speech variability, for example the inter-speaker one, while phonetic posteriorgrams are speaker-independent by construction. While this is also true for the Gaussian and English phone posteriorgrams, the far more precise prior knowledge available for the French recognizer during training clearly improves results. This underlines how SSM might be of significant value in many situations where language-specific phonetic knowledge is absent (indeed the zero-resource scenario depicted while using MFCCs or Gaussian posteriorgrams, or even when languagespecific phone recognizers are available, even if not matching the target language).

Finally, the analysis of the results for the three different distributions of weights shows that the uniform weight setting is the least performing. This is an expected outcome since, like previously remarked, the DTW score is indeed the most reliable and should be given more relevance in the global score computation. As far as the alternative weights is concerned, a revealing trend is observed that makes the distribution αDTW = 0.50, α ′ SSM = 0.20, α ′′ SSM = 0.30 the more convenient over the choice αDTW = 0.40, α ′ SSM = 0.20, α ′′ SSM = 0.40 (except for the Gaussian posteriorgram case and the P@10 measure).

Conclusion and future work

In this paper a computational framework has been described for audio-only keyword spotting by combining template matching techniques, based on DTW and SSM comparison. The system is easily applicable and well performing in a zero-resource approach, where no prior linguistic or acoustic knowledge is needed to model variability. For evaluation purposes, a comparison of the performance has been carried out against more supervised frameworks based on language-specific features. The evaluation has shown that a) the benefit of SSM comparison in a zero-resource approach and b) the crucial impact of language-1 the system, though, can hardly be considered biased towards the 4h file, indeed a very small sample of the 150h ESTER corpus In the near future we plan to extend this evaluation to word discovery experiments, and investigate the use of the described pattern matching techniques to alternative signal processing tasks.

  and b) MFCCs. Posteriorgrams are a time-vs.-class matrix representing the posterior probability of each class for each feature frame (for example, an MFCC frame). The type of class defines the type of posteriorgram. Here we evaluate the following posteriorgrams: 1. Gaussian posteriorgram: each class represents a component of a Gaussian mixture model (GMM) trained in an unsupervised way from a training data set, or, possibly, from the test data itself. 2. Hidden Markov model (HMM) state posteriorgram:

2 .Figure 1 :

 21 Figure 1: Example of combined use of DTW and SSM-based comparisons for similarity scoring of templates.

4

 4 and c) αDTW = 0.5, α ′ SSM = 0.2, α ′′ SSM = 0.3. The logic behind this choice was to check the behaviour a) with all the scores equally weighted, and b) and c) with the DTW score privileged over the others because of the more reliable similarity information provided, and with D ′′ SSM preferred over D ′ SSM , as showing better results in previous word discovery experiments.

Table 1 :

 1 The 4h dataset keyword collection with the occurrence counts.

	mobilisation (19)	Jean Marie Le Pen (35)	final (25)
	vingt-et-un avril (47)	Lionel Jospin (13)	syndicat (48)
	Saddam Hussein (36)	important (14)	Iraq (127)
	personne (45)	gauche (48)	general (29)
	week end (48)	president (77)	droite (26)
	politique (67)	journal (35)	france (284)
	responsable (21)	americain (58)

Table 2 :

 2 Results for the DTW-only and the SSM-based system, for different weight distributions: a) αDTW = 0.34, α ′ SSM = 0.33, α ′′ SSM = 0.33, b) αDTW = 0.40, α ′ SSM = 0.20, α ′′ SSM = 0.40 and c) αDTW = 0.50, α ′ SSM = 0.20, α ′′ SSM = 0.30.

	DTW	MFCC(%) Gauss(%) ENG(%) FRA(%)
	P@10	55.5	67.5	48.5	90
	P@N	30.8	46.6	30.8	70.5
	MAP	29.2	47.9	29.2	74.8
	EER	21.6	15.1	18.5	6.4
	SSM a)	MFCC(%) Gauss(%) ENG(%) FRA(%)
	P@10	64	67.5	50	86
	P@N	36.1	46.6	29.7	63.9
	MAP	34.6	38.0	27.9	66.9
	EER	22.9	16.1	21.8	7.8
	SSM b)	MFCC(%) Gauss(%) ENG(%) FRA(%)
	P@10	67.5	68.5	53.3	87
	P@N	38.5	44.1	32.6	67.3
	MAP	37.1	45.6	30.8	71.1
	EER	20.9	15.1	21.0	6.5
	SSM c)	MFCC(%) Gauss(%) ENG(%) FRA(%)
	P@10	67.5	70	51.1	88
	P@N	39.8	45.8	33.1	68.5
	MAP	38.0	47.0	31.6	72.5
	EER	20.8	15.0	19.9	6.5
	specific posteriorgrams on keyword spotting tasks based on
	template matching.

The present work was partially funded by the Quaero project: http://www.quaero.org/


Vahid Salari

Christian Brouder

Y A Popp

Phys Yan

Lett

Comment on "Delayed luminescence of biological systems in terms

Keywords: Delayed luminescence, Hyperbolic decay, Coherent states, Biophotonics PACS: 42.50.-p, 78.55.-m

Popp and Yan [F

Introduction

Delayed luminescence is the phenomenon in which light-irradiated plants reemit photons during a long period of time, up to several minutes after the end of irradiation [START_REF] Goltsev | Delayed fluorescence in photosynthesis[END_REF]. Delayed luminescence exhibits intriguing characteristics, such as hyperbolic (instead of exponential) decay and sometimes oscillations known as afterglow [START_REF] Goltsev | Delayed fluorescence in photosynthesis[END_REF].

Two kinds of interpretation of delayed luminescence can be found in the literature (see Refs. [START_REF] Goltsev | Delayed fluorescence in photosynthesis[END_REF][START_REF] Katsumata | New feature of delayed luminescence: preilluminationinduced concavity and convexity in delayed luminescence decay curve in the green alga Pseudokirchneriella subcapitata[END_REF][START_REF] Lanzanò | Time-resolved spectral measurements of delayed luminescence from a single soybean seed: effects of thermal damage and correlation with germination performance[END_REF][START_REF] Winkler | Ultraweak and induced photon emission after wounding of plants[END_REF][START_REF] Kun | Biophoton emission of biological systems in terms of odd and even coherent states[END_REF] for recent references). The first one, which is usually not quantitative, uses the details of the photosynthetic process, in particular the dynamics of Photosystem II reaction centers. The second one, proposed by Popp and Yan [START_REF] Popp | Delayed luminescence of biological systems in terms of coherent states[END_REF], is based on the idea that a coherent state of the radiation field is present in living cells. The model seems to reproduce the hyperbolic decay of delayed luminescence as well as its Email address: vahid.salari@impmc.upmc.fr (Vahid Salari).

oscillations.

In this comment, we extend the Popp and Yan model by diagonalizing the most general homeostatic time-dependent Hamiltonian compatible with single-mode coherent states. We show that this model cannot agree with experiment. In other words, the apparent agreement of the model with the hyperbolic decay of delayed luminescence is due to a calculation error in ref. [START_REF] Popp | Delayed luminescence of biological systems in terms of coherent states[END_REF].

Popp and Yan also consider hyperbolic decay to be the solution of a classical equation that oscillates with a constant frequency [START_REF] Popp | Hyperbolic relaxation as a sufficient condition of a fully coherent ergodic field[END_REF]. We show that a more general solution exists with a different asymptotic decay and that the hyperbolic solution is unstable. Moreover, we show that the oscillatory behaviour of delayed luminescence is not described by their model.

Finally, we stress the consequences of our findings for the modelling of delayed luminescence.

Hyperbolic relaxation

The most general time-dependent Hamiltonian H(t) which is solved by a single-mode (timeindependent) coherent state |v is [START_REF] Mehta | Time evolution of coherent states[END_REF]

H(t) = ω(t)a † (t)a(t) + f * (t)a(t) + f (t)a † (t) + β(t),
where a(t) is the annihilation operator (in the Heisenberg picture), ω and β are real functions of time and f a complex function of time. As a coherent state, |v satisfies a(t)|v = α(t)|v for some complex function α. The general form of α(t) is given by Mehta and Sudarshan [START_REF] Mehta | Time evolution of coherent states[END_REF] and reproduced (with two misprints) by Popp and Yan:

α(t) = e -iψ(t) α(0) -i t 0 f (t ′ )e iψ(t ′ ) dt ′ , (1)
where

ψ(t) = t 0 ω(t ′ )dt ′ .
The mean number of photons as a function of time is given by n

(t) = v|a + (t)a(t)|v = |α(t)| 2 . Thus, n(t) = α(0) -i t 0 f (t ′ )e iψ(t ′ ) dt ′ 2 . (2)
By assuming "homeostasis" and the fact that "α should not be influenced by external classical energy sources" (we do not comment here on the validity of these assumptions), Popp and Yan obtain a condition on f described by the equation: ḟ + iωf = 0, with general solution f (t) = f (0)e -iψ (t) .

By introducing this value of f (t) into eq. ( 2), we obtain

n(t) = |α(0) -if (0)t| 2 .
(

Therefore, n(t) is a quadratic function of t, independent of ω(t).

The relation between the intensity of light I(t) and n(t) is not given explicitly in Ref. [START_REF] Popp | Delayed luminescence of biological systems in terms of coherent states[END_REF], but the caption of Fig. 3 shows that the authors use the relation I(t) ∝ ṅ(t). It follows from eq. ( 3) that I(t) is a linear function of time. This does not agree with any measurement of delayed luminescence.

Therefore, the experimental hyperbolic relaxation is not accounted for by the coherent-state model of the paper.

The error in the paper by Popp and Yan is readily identified. The homeostasis hypothesis implies also the second equation ω ṅ+ ωn+ β = 0. Popp and Yan then choose ω h (t) = λ/(1 + λt) and β to be time independent and they get a hyperbolic decay (up to an additive constant). The problem is that the second equation is in fact an equation for β (with obvious solution β(t) = β(0) + ω(0)n(0) -ω(t)n(t)), because n already determined and ω is given. Note also that we worked with a general ω and not with the hyperbolic ω h used by Popp and Yan.

It might be interesting to discuss the origin of this particular ω h .

Origin of hyperbolic relaxation

Popp and Li obtained the hyperbolic solution ω h as follows [START_REF] Popp | Hyperbolic relaxation as a sufficient condition of a fully coherent ergodic field[END_REF]. They start from the differential equation ẍ(t) + 2µ(t) ẋ(t) + ω 2 0 x(t) = 0 and they use the classical method to remove the term in ẋ by writing x(t) = exp -t 0 µ(τ )dτ y(t). The equation for y is ÿ + (ω 2 0 -µ 2 -μ)y = 0. They argue that the oscillating part y(t) should have a constant frequency. For this, they solve µ 2 + μ = 0 and get µ(t) = λ/(1 + λt) and

x(t) = y(t)/(1 + λt) = ω h (t) y(t) λ .
However, even if we accept the requirement that the oscillating part has a constant frequency, then the equation that we have to solve is µ 2 + μ = ω 2 , so that y oscillates with frequency ω 2 0 -ω 2 . The equation for µ has the solution µ(t) = ω tanh(ωt + µ 0 ) and x(t) = y(t) cosh µ 0 / cosh(µ 0 + ωt). Of course, we can also consider µ 2 + μ = -ω 2 , for which µ(t) = -ω tan(ωt+µ 0 ) and x(t) = y(t) cos µ 0 / cos(µ 0 +ωt). The important point is that, in both cases, the decay is not hyperbolic and the limit of these solutions for ω → 0 is µ(t) = 0. In other words, the hyperbolic solution is unstable and the slightest perturbation transforms it into a solution with a completely different asymptotic behaviour.

Oscillations

Popp and Yan claim that the oscillatory behavior of delayed luminescence can be explained by a coupling of two coherent states |α 1 and |α 2 described by differential equation (18'), where the primed equation numbers correspond to the equation numbers of ref. [START_REF] Popp | Delayed luminescence of biological systems in terms of coherent states[END_REF]. We shall see that this interpretation meets a rather serious inconsistency. In their calculation, they use eq. (18') to derive eq. (21') and solve eq. (21') with α 1 and α 2 defined by eq. (22'). However, these α 1 and α 2 are not solutions of the starting eq. (18'). In other words, their "solutions" do not solve the coupling equation. Their error comes from the fact that a solution of eq. (21') is generally not a solution of eq. (18'). Indeed, take any differentiable real function y 1 (t) and define y 2 = y + y 1 , with

y(t) = κln(1 + λ 1 t) -κln(1 + λ 2 t) + φ.
Then, α 1 (t) = |a 1 |e -iy1(t) and α 2 (t) = |a 2 |e -iy2(t) define a solution of eq. (21') which is (generally) not a solution of eq. (18').

Conclusion

We showed that the coherent state model proposed by Popp and Yan does not agree with delayed luminescence experiments, even if we generalize their approach: the math is simply wrong.

By fitting the reaction rates of the standard photosynthetic model, it was possible to quantitatively reproduce the short-time decay of plant luminescence (see [START_REF] Guo | A kinetic model structure for delayed fluorescence from plants[END_REF] and references therein). However, no calculation of that kind was reported in the literature for long-time delayed luminescence, probably because of the apparent success of the model proposed by Popp and Yan. We hope that the present paper will clear the way for such studies.

To conclude with a positive remark, we want to stress that, even if the oversimplified model of a single coherent state must be dismissed, growing evidence shows that quantum coherence plays a role in photosynthesis [START_REF] Engel | Evidence for wavelike energy transfer through quantum coherence in photosynthetic systems[END_REF][START_REF] Scholes | Green quantum computers[END_REF]. Its influence on delayed luminescence remains to be investigated.


Thierry Bastogne
email: thierry.bastogne@cran.uhp-nancy.fr

Loraine Tirand

Julien Gravier

Denise Bechet

Vincent Morosini

Marlène Pernot

Céline Frochot

Alain Richard

François Guillemin

Muriel Barberi-Heyob

Contributions of experiment designs in photodynamic therapy: photosensitizer design, treatment analysis

Introduction

One of the difficulties in the development of the photodynamic therapy (PDT) is inherent to the multidisciplinary feature of this treatment gathering mainly clinicians, physicists, biologists, and chemists. Another issue is the great number of biophysical and biochemical parameters involved in the design of new photosensitizers as well as in the in vivo application of this treatment. We present a global development approach based on the methodology and tools of experimental design. Three study cases are developed to assess to potential relevance of such an empirical model-based approach for the development of PDT.

Methods & Results

In a first study, an in vitro screening experimental design was carried out. The addressed question dealt with the determination of influent factors on the phototoxicity of a new photosensitizer based on quantum dots. Five factors were examined: the nature of quantum dots, the excitation light wavelength, the incubation time with cells, the photoactivable compound concentration and the fluence level. Relevance of each factor was finally estimated and compared to identify the significant parameters. In comparison with a typical factorial design, the total number of experiments (42 trials) was divided by 5.

In a second study, an in vivo factorial experimental design was applied to detect potential synergic effects between four therapeutic factors: the phenotype of the cancer cell line, the food type, the nature of photosensitizer and the post-injection time, on the in vivo selectivity (cancer/normal tissue) of the tested photosensitizers. Results particularly pointed out the presence of a statistically significant synergic effect between these four factors and provided the optimal modalities to maximize the response in term of tumor-to-normal tissue ratio.

In a third study, a Doehlert experimental design associated with a response surface model was used to determine the in vivo PDT modalities (photosensitizer concentration, irradiance and fluence) to both minimize the post-treatment growth rate of the tumor and maximize its growth delay. Only 13 experimental conditions were tested and the relevance of the optimized condition was corroborated by in vivo validation experiments.

Conclusion

These studies have confirmed the applicability and attractive contributions of experimental design techniques in the development and determination of optimal modalities of new photosensitizers in PDT. Their main advantages are to a priori organize experiments according to specific questions while minimizing the experimental cost and controlling as much as possible the experimental uncertainty.


Rafik Absi
email: r.absi@ebi-edu.com

Caroline Nalpas

Florence Dufour

Denis Huet

Rachid Bennacer

Tahar Absi

Teaching Fluid Mechanics for Undergraduate Students in Applied Industrial Biology: from Theory to Atypical Experiments

Keywords: Atypical experiments, fluid mechanics, teaching assessment, evaluation analysis, semantic analysis, play-based pedagogy, higher education

EBI is a further education establishment which provides education in applied industrial biology at level of MSc engineering degree. Fluid mechanics at EBI was considered by students as difficult who seemed somewhat unmotivated. In order to motivate them, we applied a new play-based pedagogy. Students were asked to draw inspiration from everyday life situations to find applications of fluid mechanics and to do experiments to verify and validate some theoretical results obtained in course. In this paper, we present an innovative teaching/learning pedagogy which includes the concept of learning through play and its implications in fluid mechanics for engineering. Examples of atypical experiments in fluid mechanics made by students are presented. Based on teaching evaluation by students, it is possible to know how students feel the course. The effectiveness of this approach to motivate students is presented through an analysis of students' teaching assessment. Learning through play proved a great success in fluid mechanics where course evaluations increased substantially. Fluid mechanics has been progressively perceived as interesting, useful, pleasant and easy to assimilate. It is shown that this pedagogy which includes educational gaming presents benefits for students. These experiments seem therefore to be a very effective tool for improving teaching/learning activities in higher education.

Introduction

Teaching science and technology has been often related to experiments conducted to confirm or disprove a theory. If the path travelled by the scientists is marked by different experiments, the fact is that the idea of experience is rooted in our educational practices to become one of the top concerns of those seeking to understand or to solve a problem. Universities, institutes and education in general today are based on technical education that prepares students for the expected responsibilities. EBI, which provides education for engineers in applied industrial biology, listed its efforts in this process that commits teachers and students by giving everyone a role to play. EBI provides a 5 years MSc Diploma course to train students to work as engineers in the field of pharmaceutics, cosmetics, food engineering, environment, and others [1]. In the undergraduate cycle, students learn mathematics, physics, biology and chemistry. Among the courses of physics, "fluid mechanics" [START_REF] Absi | La mécanique des fluides pour les étudiants du cycle préparatoire de l'EBI[END_REF][START_REF] Absi | A simple eddy viscosity formulation for turbulent boundary layers near smooth walls[END_REF] in 2nd year was considered by students as difficult and they seemed somewhat unmotivated. In order to encourage them to show more interest to this course, a new pedagogy based on atypical experiments was tested on students of year "P17". Students were asked to draw inspiration from everyday life situations to find applications of course and to do atypical experiments to verify and validate some theoretical results.

To assess the relevance of this new pedagogy, we will analyze course evaluations by students of "P17" and we will compare them with those of three other years (two before the new pedagogy and one after). The following study focuses on students of four years: "P15" (2005/2006), "P16" (2006/2007), "P17" (2007/2008) and "P18" (2008/2009).

Methodology

Our methodology is based on the goals of our institute which aims to provide industry with practical engineers. The education is based on the abilities of the teacher to put into practice the program and give students the opportunity to become more involved. The relationship between teacher and learners determine the learning process to a large part. By empowering the student to enable him to become the cornerstone of the act of learning, the teacher performs half the way of the learning process. Our methodology is therefore based on the idea of motivating students with a play-based pedagogy through atypical experiments to verify and validate some theoretical results found in course by referring to daily life situations.

Example of atypical experiments in fluid mechanics

As examples of atypical experiments in fluid mechanics made by students of "P17", we present experiments of emptying jerry cans. These experiments of emptying jerry cans were made by three students: Lucie Clavel, Maëla Drouin, and Laure-Anne Gillon. Their work was presented in a report [START_REF] Clavel | La vidange, atypical experiments in fluid mechanics[END_REF] with: Goal of experiments, Material and Method, Experiments and Results, and Conclusions.

Presentation of the experiments

The students began by identifying the required material for these experiments, namely: two plastic jerry cans one blue non-transparent of 35 liters and a second white transparent of 20 liters, two plugs of 1 cm diameter, a stopwatch, a meter, a scales of 100g precision and a spirit level (figure 1). They chose to work with water for a practical purpose: its accessibility and physical properties as density.

They made holes of 1cm diameter in the jerry cans to allow a slow flow, easy to measure with a stopwatch, hence a more accurate measured emptying time. The holes were placed in the bottom part of jerry cans in order to allow an adequate visualization of the flow (figure 1). They were also made to obtain a contraction coefficient C c equal to 0.61. When discharging, the jerry cans were placed on a coffee table, which horizontality was controlled with a spirit level. Thus, the height of water level measured in the jerry cans is uniform. Before beginning the experiments, the jerry cans filled with water were weighed to determine the mass of water passed by subtracting the mass of the jerry can at the end of emptying.

Experiments and results

By applying the Bernoulli equation from the water surface to the orifice, the flow is obtained as

          2 2 2 1 1 2 S s C gh Q c (1)
In this equation, g is the gravity acceleration, h level of water, S the horizontal surface of jerry can, s the surface of hole and C c the contraction coefficient. The emptying time is given by [

2] K h T 2  (2)
Where:

          2 2 2 1 1 2 1 S s C g S K c .
For example for the blue jerry can: h = 37 cm C C = 0.61 s= πr 2 = 0.785 cm 2 S=l*L = 837 cm 2 And the initial flow Q = 0.00013 m 3 /s = 0.13 l/s They find K = 0.0025 m 0.5 s -1 and therefore the theoretical emptying time is T theoretical =486.6 s = 8 min 7 s. The experimental emptying time is T experimental = 8 min 40 s. The students found that the theoretical emptying time is lower than the experimental one. Since the result found in course was for a perfect or ideal fluid (µ=0), they explained the difference between theoretical and experimental results by the fact that they neglected the viscosity µ of water in the theoretical time (assumed equal to 0). They concluded that the viscosity of water (µ ≠ 0) will increase the value of the theoretical emptying time and will allow therefore a more accurate value.

Teaching evaluation by students

The impact of our new pedagogy could be assessed from teaching or course evaluation. These evaluations could be considered as a useful tool in order to improve the exchange between teacher and students. Different studies have been conducted on the effectiveness of this tool and its relevance or not [START_REF] Conle | Moments of interpretation in the perception and evaluation of teaching[END_REF][START_REF] Marsh | Students' evaluations of university teaching: Dimensionality, reliability, validity, potential baises, and utility[END_REF][START_REF] Marsh | Validity of students' evaluations of teaching effectiveness: Cognitive and affective criteria[END_REF][START_REF] Sproule | The underdetermination of instructor performance by data from the student evaluation of teaching[END_REF]. Course evaluations by students are performed at EBI on the website of studies.

Process of teaching evaluations by students:

In order to start the online teaching evaluations, students need first to select the course and the year. The evaluation includes two parts (figure 3):

-The first consists on evaluating on a scale of 4: (1) average, (2) satisfactory, (3) good, (4) very good ; the following criteria : organization, required work, clarity of explanations, pedagogy used, interactivity, implication of students, controls. -The second consists on answering two questions related to principal forces and issues to be improved. It is also possible to write additional comments and observations. Figure [START_REF] Conle | Moments of interpretation in the perception and evaluation of teaching[END_REF] presents the average of the different criteria for students of four years (P15 to P18). This figure allows observing a peak for students of year P17 who were concerned by the new pedagogy based on atypical experiments. This peak indicates that all criteria have maximum values and therefore that the teaching was considered as the best on the basis of the evaluation of the different criteria namely: organization, required work, clarity of explanations, pedagogy used, interactivity, implication of students and controls. It is important to note that the number of students who participated in the teaching evaluation has decreased from year P15 to P18 (figure 6). This should have a significant effect on the evaluation results and their analysis.

Evolution of different criteria

Semantic analysis of comments and observations

The students' comments were analyzed. We identified the different "words" hereafter called "quotations", which were written by students in their comments and observations. These quotations have been grouped by semantic fields and groups (Table 1 1: semantic analysis of students' comments and observations Semantic groups: (1) Human qualities, (2) Pedagogy; (+) forces, (-) issues to be improved.

Table (1) presents the occurrence which is the number of quotations "n" associated to each field by year. In order to take into account the effect of the number of students who participate in the course evaluation "N", we divided "n" by "N". Figure (7.a) presents the number of quotations "n" related to each field by year. Figure (7.b) shows the effect of number of responses or number of students who participate in the teaching evaluation "N". The parameter "n/N" for year P17 (solid line) is above the other lines (three other years) for criteria "experience", "clarity" and "interactivity". However, we notice that the criteria "concrete" decreases for P17, while atypical experiments should raise this criterion. We can explain this by the fact that students are not obliged to write comments and therefore some students found enough to rate each specified criterion (on scale of 4). There is some redundancy between the quantitative evaluations related to the specified criteria and free comments.

Impact on students' learning

²Students' participation in improving teaching through atypical experiments and course assessment gives teaching a new dimension. By this free participation in these experiments, the student discovers the value of effort and no longer hesitates to ask questions or seek solutions to encountered problems. Thus learning is no longer a mere reproduction of abstract knowledge or non-practical applications but an anchorage in the environment.

Conclusions

In this paper, we presented an innovative teaching/learning pedagogy based on the concept of atypical experiments and its implications in teaching fluid mechanics. This new pedagogy was tested at EBI on students of year "P17". Students of "P17" were asked to draw inspiration from everyday life situations to find applications of course and to do experiments to verify and validate some theoretical results. The experiment of emptying of jerry cans is presented as an example of these atypical experiments. The impact of this innovative pedagogy has been evaluated from online teaching assessment made by students. We analyzed the course evaluations by students of "P17" and we compared them with those of three other years (two before the application of the new pedagogy and one after). The teaching evaluation concerns: (1) a first part which consists on assessing on a scale of 4 the criteria: organization, required work, clarity of explanations, pedagogy used, interactivity, implication of students and controls; (2) a second part which consists on two questions related to main forces and issues to be improved and free comments and observations. The analysis of teaching assessment shows for part 1 (assessment of criteria on a scale of 4 for years P15 to P18) a peak in all criteria for students of year P17, which was concerned by the application of the new pedagogy based on atypical experiments. This result is very interesting and shows the efficiency of this approach to motivate students. In addition to the quantitative analysis, we analyzed comments and observations of students (part 2 in the teaching assessment). We identified first the main "words" called "quotations" which were grouped into semantic field and groups. We compared first the occurrence which is the number of quotations "n" related to each field for a given year. However, in order to account for "N" the number of students who took part in the teaching assessment, we introduced the parameter "n/N". The parameter "n/N" for year P17 is above the three other years for criteria "experience", "clarity" and "interactivity".

Learning through play proved a great success in fluid mechanics which has been progressively perceived as interesting, useful, pleasant and easy to assimilate. We showed that this pedagogy which includes educational gaming presents benefits for students. These experiments seem therefore to be an effective tool for improving teaching/learning activities in higher education.

We can deduce from this study that trust acquires in the cooperation and learning is based on both knowledge and students involvement. In this exchange, the teacher learns as much as student, because in teaching him to remain curious, he reinforces his skills to seek solutions to encountered problems.

Figure 1 :

 1 Figure 1: Example of atypical experiments in fluid mechanics: emptying jerry cans, preparation of experiments.

Figure 2 :

 2 Figure 2: Example of atypical experiments in fluid mechanics: Emptying of a 35 liters plastic jerry can. (a) Experiments. Evolution of water level (b) and distance of water/soil impact (c) vs time.

Figure 3 : 3 .

 33 Figure 3: Online course evaluations by students: (a) EBI studies web site, (b) evaluation by "(1) average, (2) satisfactory, (3) good or (4) very good" of each criteria, (c) second part of evaluation: two questions and additional comments.

Figure 4 :

 4 Figure 4: Sample of teaching evaluation of fluid mechanics course by P17 students.

Figure 5 :

 5 Figure 5: Evolution of different criteria for each year (from P15 to P18)

Figure 6 :

 6 Figure 6: Percentage of students' participation to teaching evaluation (from P15 to P18).

Figure 7 :

 7 Figure 7: (a) Number of quotations « n » related to each semantic field by year. (b) Effect of the number of students who participated in the teaching evaluation « N », n/N VS semantic fields.

  ).

			Students →		P18		P17		P16		P15
	Semantic	Semantic fields ↓	n	n/N ;	n	n/N ;	N	n/N ;	n	n/N ;
	Groups↓			N=39		N=58		N=73		N=93
	(1)	(+)	Amiability	6	0,1538	4	0,0689	5	0,0684	3	0,0322
			Passionate	2	0,0512	2	0,0344	4	0,0547	1	0,0107
			Listening	2	0,0512	5	0,0862	5	0,0684	4	0,0430
		(-)	Authority	0	0	1	0,0172	4	0,0547	4	0,0430
			Delay	0	0	0	0	0	0	2	0,0215
			Noise	0	0	4	0,0689	7	0,0958	3	0,0322
	(2)	(+)	Experiments, lab	4	0,1025	5	0,0862	3	0,0410	2	0,0215
			work, project
			Clarity	4	0,1025	9	0,1551	11	0,1506	6	0,0645
			Interactivity	2	0,0512	7	0,1206	7	0,0958	6	0,0645
			Explanations	5	0,1282	6	0,1034	10	0,1369	9	0,0967
			Demonstrations	0	0	2	0,0344	3	0,0410	9	0,0967
			Recalls	0	0	3	0,0517	4	0,0547	6	0,0645
			Motivation	1	0,0256	2	0,0344	4	0,0547	5	0,0537
			Concrete	3	0,0769	3	0,0517	7	0,0958	8	0,0860
			PowerPoint	0	0	0	0	0	0	8	0,0860
		(-)	Repetitions	0	0	0	0	1	0,0136	5	0,0537
			Level difference	3	0,0769	0	0	0	0	5	0,0537
			Table


Nicolas Obin
email: nobin@ircam.fr

Anne Lacheret
email: anne.lacheret@u-paris10.fr

Xavier Rodet
email: rodet@ircam.fr

Stylization and Trajectory Modelling of Short and Long Term Speech Prosody Variations

Keywords: speech prosody, stylization, trajectory model, speech synthesis

In this paper, a unified trajectory model based on the stylization and the modelling of f0 variations simultaneously over various temporal domains is proposed 1 . The syllable is used as the minimal temporal domain for the description of speech prosody, and short-term and long-term f0 variations are stylized and modelled simultaneously over various temporal domains. During the training, a context-dependent model is estimated according to the joint stylized f0 contours over the syllable and a set of long-term temporal domains. During the synthesis, f0 variations are determined using the long-term variations as trajectory constraints. In a subjective evaluation in speech synthesis, the stylization and trajectory modelling of short and long term speech prosody variations is shown to consistently model speech prosody and to outperform the conventional short-term modelling.

Introduction

In parallel to the development of high-quality speech synthesis systems [START_REF] Zen | Statistical parametric speech synthesis[END_REF], the modelling of speech prosody has raised as a major concern to improve the naturalness, the liveliness, and the variety of the synthetic speech. Speech prosody is generally described as the co-occurrence of acoustic gestures occurring simultaneously over different temporal domains [START_REF] Fujisaki | The Production of Speech[END_REF][START_REF] Van Santen | Intonation Analysis, Modelling and Technology[END_REF] and associated to different communicative functions (linguistic, expressive). A high-quality modelling of speech prosody is desirable for natural and expressive speech synthesis and adequate modelling of speaking style, and a prerequisite in real multi-media applications (e.g., avatars, story telling, dialogue systems, numeric arts).

A variety of methods has been proposed to model speech prosody variations (f0 [START_REF] Yoshimura | Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis[END_REF], temporal structure [START_REF] Zen | Hidden semi-Markov model based speech synthesis[END_REF]), and local and global variations [START_REF] Tokuda | Trajectory modeling based on HMMs with the explicit relationship between static and dynamic features[END_REF][START_REF] Toda | A speech parameter generation algorithm considering global variance for HMM-based speech synthesis[END_REF]. However, conventional methods usually models short-term variations of speech prosody (frame-based, or instantaneous variations), while long-term variations of speech prosody are not explicitly considered. Recent studies have been proposed to integrate long-term variations into HMM modelling, either for the modelling of f0 variations [START_REF] Latorre | Multilevel parametric-base F0 model for speech synthesis[END_REF][START_REF] Qian | Improved prosody generation by maximizing joint likelihood of state and longer units[END_REF], or with extension to state-duration modelling [START_REF] Gao | Duration refinement by jointly optimizing state and longer unit likelihood[END_REF]. However, the proposed methods remain a mixed model, i.e. the conventional model is used to model the instantaneous variations of f0, while stylization of long-term variations are used as trajectory constraints only. In particular, the instantaneous variations remain the minimal and target temporal domain for the modelling of speech prosody.

In this paper, a unified trajectory model based on the stylization and the joint modelling of f0 variations over various temporal domains is proposed. In the proposed approach, the syllable is used as the minimal temporal domain for the description of speech prosody, and f0 variations are stylized and modelled simultaneously over various temporal domains which cover short-term and long-term variations. During the training, a context-dependent model is estimated according to the joint stylized f0 contours over the syllable and a set of long-term temporal domains. During the synthesis, f0 variations are determined using the long-term variations as trajectory constraints.

## ## l o ∼ l o ∼ t o ∼ t a ∼ ##
syllable-based HMM with stylization of f0 contours

Stylization of Speech Prosody

The Discrete Cosine Transform (DCT) is used to stylize the f0 variations over various temporal domains [START_REF] Teutenberg | Modelling and Synthesising F0 contours with the Discrete Cosine Transform[END_REF] (figure 2). The

C ∆ (k) C C O =      C ∆ (1) C ... ∆ (K) C      high-
f0(t) = K X k=1 α k c k cos (ω k t) (1)
where c k is the k-th term of the DCT, and α k a term used for normalization.

Two classes of temporal domains are defined for the stylization of f0 variations:

Syllable context accounts for f0 variations occurring on the syllable and its immediate context (0-order represents the f0 variations over the syllable, 1-order the f0 variations over the 1-left-to-right syllable context, . . . );

Linguistic contexts account for f0 variations occurring on long-term prosodic units (e.g., minor and major prosodic groups). A minor prosodic group is defined as the prosodic unit that ends with an intermediate prosodic boundary, and is used for rhythmic grouping typical of French. A major prosodic group is defined as the prosodic unit that ends with a major prosodic boundary.

F0 variations are stylized using a 5-order DCT. F0 is linearly interpolated in the logarithmic domain prior to the stylization. The stylization over various temporal scales aims at representing f0 variations with more or less details, and to model short and long term dependencies.

Trajectory Model

The Trajectory Model has been introduced in HMM-based speech synthesis to explicitly model the dynamic (local variations) of the speech parameters [START_REF] Tokuda | Trajectory modeling based on HMMs with the explicit relationship between static and dynamic features[END_REF]. In this study, syllable is assumed as the minimal temporal domain for the description of speech prosody, and f0 variations are stylized and modelled simultaneously over different temporal domains: shortterm variations correspond to the stylization of f0 contours over the syllable, and long-term variations correspond to the stylization of f0 contours over long-term temporal domains. During the training, a context-dependent HMM is estimated from the joint short-term and long-term variations. During the synthesis, the short-term variations are determined so as to maximize the conditional probability of the short-term variations under the constraint of the long-term trajectories.

Parameters Estimation

Let q = [q1, . . . , qN ] be the sequence of linguistic contexts, where qn = [qn(1), . . . , qn(L)] ⊤ is a (Lx1) linguistic vector which describes the linguistic characteristics associated with the n-th syllable. Let c = [c1, . . . , cN ] be the static observation sequence of stylized f0 contours over the syllable-level unit, where cn = [cn(1), . . . , cn(D)] ⊤ is a (Dx1) observation vector which describes the short-term f0 characteristics associated with the n-th syllable.

Let k) cN ] be the dynamic observation sequence of stylized f0 contours over the k-th long-term temporal domain, where ∆ (k) cn = [∆ (k) cn(1), . . . , ∆ (k) cn(D)] ⊤ is a (Dx1) observation vector which describes the long-term f0 characteristics associated with the n-th syllable. Let o = [o1, . . . , oN ] be the augmented observation sequence, where on = [c ⊤ n , ∆ (1) c ⊤ n , . . . , ∆ (K) c ⊤ n ] ⊤ is a (KDx1) observation vector which describes the short-term and long term f0 characteristics associated with the n-th syllable, and K the total number of long-term temporal domains being modelled.

∆ (k) c = [∆ (k) c1, . . . , ∆ (
A HMM λq is estimated for each of the linguistic contexts. Each of the context-dependent HMMs is assumed to be a singlestate HMM with single normal distribution and diagonal covariance matrix. Then, a context-dependent HMM λ is derived based on Maximum-Likelihood Minimum-Description-Length (ML-MDL). The long-term variations are used as additional trajectory constraints to refine the clustering of the models. A conventional context-dependent HMM is used to model syllable durations.

Parameters Inference

The determination of the sequence of f0 parameters is similar to that of the Trajectory Model with the exception that the frame-based static observation is reformulated into the stylized f0 contour over the syllable, and the frame-based dynamic observation (partial derivative) is reformulated into the stylized long-term f0 contours. The sequence of syllable durations is determined with the conventional static method as the sequence of mean durations.

The optimal static observation sequence c is determined so as to maximize the log-likelihood of the short-term observation sequence o, under the constraint of the long-term trajectories ∆ (k) c.

The optimal observation sequence b o = [c o1 ⊤ , . . . , c oT ⊤ ] is determined so as to maximize the conditional probability of the observation sequence o given the model λ.

b o = argmax o max q p(o|q, λ) p(q|λ) (2)
The determination of the optimal observation sequence o divides into the following sub-problems:

b q = argmax q p(q|λ) (3)
b o = argmax o p(o|b q, λ) (4)
Assuming that each syllable is modelled by a single-state HMM, the optimal state sequence simply corresponds to the concatenated sequence of context-dependent models associated with each syllable of the syllable sequence:

b q = [q1, . . . , qN ] (5)
where N denotes is the total number of syllables in the syllable sequence.

The maximization of p(o|b q, λ) with respect to o is equivalent to the maximization of p(c|b q, λ) with respect to c under the dynamic constraints ∆ (k) c:

b o = argmax o p(o|b q, λ) ⇔ b c = argmax c p(F(c)|b q, λ) (6)
under the constraint:

o = F (c) = h c ⊤ , ∆ (k) c ⊤ , . . . , ∆ (K) c ⊤ i ⊤ (7)
A local solution to this problem is determined recursively using a quasi-Newton method. Finally, global variance is used to model global dynamics [START_REF] Toda | A speech parameter generation algorithm considering global variance for HMM-based speech synthesis[END_REF].

Evaluation

Stimuli

The proposed trajectory model was evaluated and compared to the conventional HMM-based model in a subjective evaluation in speech synthesis. Four models were compared: 1) the conventional HMM-based model (HTS), and trajectory models using different long-term temporal domains: 2) syllable + 1-order syllable-context (1ORDER), 3) syllable + minor prosodic group (AG), and 4) syllable + major prosodic group (PG). Evaluation was conducted using the HMM-based speech synthesis system [START_REF] Zen | Statistical parametric speech synthesis[END_REF]. Models were trained on 5 hours (1888 utterances) of a French single-speaker story-telling speech database using conventional linguistic contexts. 8 sentences randomly extracted from the fairy-tale "Le Petit Poucet" ("Little Tom Thumb") were used for the comparison. For each of the trajectory models, the inferred sequence of stylized f0 parameters was converted into a sequence of f0 variations with respect to the inferred syllable durations and the voice/unvoiced sequence as inferred from the conventional HMM-based f0 model. Finally, speech utterances were synthesized by the speech synthesizer. Each sentence was synthesized with the different models.

Procedure

20 native French speakers (including 13 expert and 7 naïve listeners) participated in the evaluation. The experiment consisted in a subjective comparison of the different speech prosody models. A comparison category rating test was used to compare the naturalness of the synthesized speech utterances. The evaluation was conducted according to a crowd-sourcing technique using social networks. Pairs of synthesized speech utterances were randomly presented to the participants. They were asked to attribute a preference score according to the naturalness of the speech utterances being compared on the comparison mean opinion score (CMOS) scale.

Results

Overall CMOS and preference score (PS) are presented in figure 3. The 1-order trajectory model significantly outperforms all of the other prosodic models whatever is the preference measure.

In particular, the 1-order trajectory model is overally significantly preferred to the other prosodic models (CMOS=+0.53, PS=30%), and is individually significantly preferred to each of the other prosodic models (MOS=+0.54,+0.51,+0.54 and PS=52.1%,56.3%,55.1% compared with HTS, AG, and PG models respectively). The AG trajectory model is preferred to the HTS model but not significantly (overall: CMOS=-0.18, PS=22%; pair: CMOS=+0.15, PS=46%); and significantly preferred to the PG trajectory model. Finally, the HTS model is preferred to the PG trajectory model, but not significantly (overall: CMOS=-0.34, PS=18%; pair: CMOS=+0.10, PS=28.7%).

In particular, trajectory models decrease in preference when increasing the temporal domain of the trajectory constraint (CMOS 1-order =+0.53,PS 1-order =30%; CMOSAG=-0.18, PSAG=22%; CMOSPG=-0.38, PSPG=17%).

A comparison of the preference scores depending on the expertise of the participant reveals a significant difference in the perception of speech prosody between naïve and expert listeners : naïve listeners have clearly marked preferences, but with more variability, while expert listeners have less marked preferences, but with less variability (table 1 Mean score and 95% confidence interval.

Discussion

A study case of synthesized f0 variations with respect to the speech prosody model is provided in figure 4 with prior state duration alignment. Speech prosody differences mostly concern f0 variations, and no significant differences between state-based and syllable-based modelling. The 1-order trajectory model clearly succeeds to model the local variations and dynamic of speech prosody. The synthesized f0 variations presents an expanded dynamics while less micro-prosodic details than those synthesized by the HTS model. Thus, naïve listeners may focus on global variations only, when expert listeners may pay a closer attention to finer prosodic details. The AG trajectory model appears to model middle-term prosodic variations such as initial f0 reset and local f0 declination, compared with the 1-order trajectory model and the HTS model. However, dynamics is less expended, and prosodic phrasing is more flat.

A comparison of the different trajectory models reveals that differences in speech prosody concern local (syllable contours and dynamics) and global f0 variations. However, it is observed that the increase of the trajectory domain results into noisy local f0 variations, and partially (AG) or totally (PG) inadequate global f0 contours. In particular, the PG trajectory model failed in modelling global f0 declination. The degradation is probably due to the increase in the dimensionality of the optimization problem when accounting for long-term trajectory constraints. In the absence of an explicit formulation of the gradient, the optimization method obviously failed to account for the long-term dependencies. Not surprisingly, this results both into local and global degradation in the synthesized f0 variations.

Conclusion

In this paper, a trajectory model based on the stylization and the joint modelling of f0 variations over various temporal domains was proposed. In the proposed approach, f0 variations are stylized with a Discrete Cosine Transform, and modelled simultaneously over various temporal domains which cover short-term and long-term variations. During the training, a context-dependent model is estimated according to the joint stylized f0 contours over the syllable and a set of long-term temporal domains. During the synthesis, f0 variations are inferred using the long-term variations as trajectory constraints. The evaluation consisted in a subjective comparison of different speech prosody models in speech synthesis.

The 1-order trajectory model was proved to be significantly preferred to the conventional model, and to the other trajectory models. Each of the trajectory models succeeds in modelling f0 contours that are consistent with the considered temporal domains. However, the ability of the trajectory model to account for long-term variations decreases when the temporal domain increases, due to the increase in complexity of the optimization process. In further studies, the relationship between static and dynamic trajectories will be explicitly formulated, and different combinations of trajectory constraints will be evaluated. Finally, the formulation of the trajectory model will be extend to the modelling of the local speech rate variations.

Figure 1 :

 1 Figure 1: Schematic comparison of frame-based and syllablebased modelling of f0 variations.

Figure 3 :

 3 Figure 3: CMOS and PS. Mean and 95% confidence intervals

Figure 4 :

 4 Figure 4: Comparison of synthesized f0, with PSs.

Table 1 :

 1 ). CMOS depending on the expertise of the participant.

	CMOS		naive			expert
			score	rank		score	rank
	HTS	-0.77	(± 0.44)	4	-0.20	(± 0.27)	2
	1-order	+0.88	(± 0.43)	1	+0.41	(± 0.26)	1
	AG	-0.10	(± 0.50)	2	-0.21	(± 0.28)	3
	PG	-0.20	(± 0.44)	3	-0.52	(± 0.24)	4

This study was partially funded by "La Fondation Des Treilles", and supported by ANR Rhapsodie 07 Corp-030-01; reference prosody corpus of spoken French; French National Agency of research;

2008- 2012.