Professor Athanasios Mouchtaris

Georgios Tziritas

Pavlos Mattheakis

Maria Koutsogiannaki

George Tzagkarakis

Vangelis Vasilakis

Marcelo Caetano

Gilles Degottex

Chris Margiolas

Petros Androvitsaneas

Dr Olivier Rosec

Merci Pavlos Mattheakis

4-year Ph.D. "quest", I had the opportunity to meet and work with exceptional people and scientists

Résumé en franc ¸ais

La modélisation sinusoödale est une des méthodes les plus largement utilisés paramétriques pour la parole et le traitement des signaux audio. L'estimation précise des paramètres sinusoïdaux (des amplitudes, des fréquences et des phases) est une tâche essentielle pour la représentation près du signal analysé. Dans cette thèse, basée sur les avancées récentes dans l'analyse sinusoïdale, nous proposons une grande résolution modèles sinusoïdaux d'adaptation pour l'analyse, la synthèse et les modifications des systèmes de discours. Notre objectif est de fournir des systèmes qui représentent la parole d'une manière très précise et compacte.

Inspiré par le récemment introduit Modèle adaptif quasi-harmonique (aQHM) et Modèle adaptif harmonique (aHM), nous la vue d'ensemble de la théorie de l' adaptation sinusoïdale modélisation et nous proposons un modèle nommé la Modèle étendu adaptif Quasi-harmonique (eaQHM), qui est un non modèle paramétrique de mesure d'ajuster les amplitudes et les phases instantanées de ses fonctions de base aux caractéristiques variant dans le temps de sous-jacents du signal de parole, ainsi atténuer significativement la dite hypothèse de stationnarité locale. Le eaQHM est montré à surperformer aQHM dans l'analyse et la resynthèse de la parole voisée. Sur la base de la eaQHM , un système hybride d'analyse / synthèse de la parole est présenté (eaQHNM), et aussi d' une version hybride de l' aHM (aHNM). En outre, nous présentons la motivation pour une représentation pleine bande de la parole en utilisant le eaQHM, c'est, représentant toutes les parties du discours comme haute résolution des sinusoïdes AM-FM. Les expériences montrent que l'adaptation et la quasi-harmonicité est suffisante pour fournir une qualité de transparence dans la parole non voisée resynthèse.

La pleine bande analyse eaQHM et système de synthèse est présenté à côté, ce qui surpasse l'état de l'art des systèmes, hybride ou pleine bande, dans la reconstruction de la parole, offrant une qualité transparente confirmé par des évaluations objectives et subjectives.

En ce qui concerne les applications, le eaQHM et l' aHM sont appliquées sur les modifications de la parole (de temps et pas mise à l'échelle). Les modifications qui en résultent sont de haute qualité, et suivent des règles très simples, par rapport à d'autres systèmes de modification état de l'art. Les concepts de phase relative et les retards de phase relatifs sont cruciales pour le développement du libre-artefact, modifications, de haute qualité forme invariante. Les résultats montrent que harmonicité est préféré au quasi-harmonicité de modifications de la parole du fait de la simplicité de la représentation intégrée. En outre, la pleine bande eaQHM est appliquée sur le problème de la modélisation des signaux audio, et en particulier d'instrument de musique retentit. Le 3 Inside the analysis window, the frequency trajectory of a partial (solid grey line) is assumed to be constant for stationary sinusoidal models (dotted line), while eaQHM (dashed line) iteratively adapts to the shape of the instantaneous frequency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Inside the analysis window, the amplitude trajectory of a partial (solid grey line) is assumed to be constant for stationary sinusoidal models (dotted line), while eaQHM (dashed line) iteratively adapts to the shape of the instantaneous amplitude.

General Introduction

Without any doubt, speech is the most important and the most natural means of communication between humans. For this, digital processing of speech signals has been one of the most exciting areas of signal processing. In the last decades, speech research has led scientists and engineers in the discovery of several tools that still have tremendous impact on society. Voice communication and storage has been made effective and efficient due to advances in speech coding [START_REF] Ravishankar | Speech Coding[END_REF][START_REF] Hasegawa-Johnson | Speech Coding: Fundamentals and Applications[END_REF] algorithms and technologies. Human-computer interaction became more convenient because of speech recognition [START_REF] Huang | An overview of modern speech recognition[END_REF] techniques that were able to make computers understand not only human speech but also human languages. Interactive systems increased their efficiency by corresponding to humans with natural voice: speech synthesis [START_REF] Zen | Statistical parametric speech synthesis[END_REF] algorithms are responsible for that. Speech analysis [START_REF] Quatieri | Discrete-Time Speech Signal Processing[END_REF] models and principles managed to provide deeper knowledge about human speech production system, helping medical doctors towards faster and reliable detection of pathologies and anomalies in speech. Algorithms for enhancement of speech [START_REF] Benesty | Speech Enhancement. Signals and Communication Technology[END_REF] under noisy conditions made terrestrial and satellite communications more robust. Entertainment industry got benefit from advanced speech transformation [START_REF] Stylianou | Voice transformation[END_REF] techniques to provide artificial voices in toys, films, and video-games. The list is endless and gets more and more populated day by day.

Apparently, speech processing has numerous applications and probably will have more in the future, when the convergence of computers, communications, and the Internet will grow stronger.

The Human Speech Production Mechanism

This thesis is related to speech, hence a brief review on the human speech production mechanism should not be missing. Figures 1.1a and 1.1b show the basic parts and organs that work together in speech production. These are, roughly, divided into three groups: the lungs, the larynx, and the vocal tract. First, the lungs provide the energy in the form of an airflow travelling into the trachea, where it is modulated by the glottis (see Figure 1.1b). This modulation results in either a quasi-periodic or a noisy source, which comes into the vocal tract and determines the phonation type. The latter consists of the three cavities: the oral, the nasal, and the pharyngeal. Inside the vocal tract, the source is shaped, hence giving sound its naturalness. Finally, the resulting waveform is radiated by the lips. More specifically, when voiced sounds are produced, the larynx is the source of the sound energy. First, the vocal folds are coming closer until fully attached to each other, temporarily blocking the airflow from the lungs and leading to increased subglottal pressure. When the resistance offered by the vocal folds becomes less than the subglottal pressure, the vocal folds reopen. Then, a combination of factors, including elasticity, laryngeal muscle tension, and the Bernoulli effect, lead to an immediate closure of the vocal folds. The vocal cords will continue to open and close in a quasi-periodic fashion as long as the process is maintained by a steady supply of pressurized air from the lungs. As they open and close, pulses of air flow through the glottal opening. The frequency of these pulses determines the fundamental frequency (f 0 ) of the source and contributes to the perceived pitch of the sound that is produced. This fundamental frequency varies over time, providing linguistic information, as in the different intonation patterns associated with questions and statements, and information about emotional content, such as differences in the emotional situation of the speaker. The vocal tract, consisting of both the oral and nasal cavities can serve as a time-varying acoustic filter that suppresses sound energy at certain frequencies while allowing it at others. The frequencies where local energy maxima are sustained by the vocal tract are called formants and are determined by the shape, length, and volume of the vocal tract, whereas the frequencies where local energy is suppressed are named anti-formants.

When unvoiced sounds are produced, the larynx is again the source of the sound energy; however, the vocal folds may be completely open, as in unvoiced consonants /s/ and /f/, whereas an intermediate position may also occur in phonemes like /h/. In stop consonants, such as /p/, /t/, or /k/, the vocal cords may act suddenly from a completely closed position in which they cut the air flow completely, to totally open position producing a glottal stop.

Modeling Speech

Over the years, scientists and engineers have invented numerous ways to represent the speech production mechanism in a mathematical context. Roughly speaking, there are two different, but not distinct, approaches: (a) a mathematical model that takes into account the actual speech production mechanism, considering it as a linear, time-varying system, excited by an input signal that differs according to the type of voicing (voiced or unvoiced speech), and (b) a mathematical model that represents the speech signal as a time-series, that is, a sum of amplitude and frequency modulated sinusoids. Approaches that follow the former model can be generally named as source-filter models, whereas for the latter they are said to follow a sinusoidal model.

The Source-Filter Model

The source-filter theory of speech finds its origins in the work of Fant [START_REF] Fant | Acoustic Theory of Speech Production[END_REF]. In brief, the acoustic speech output is commonly considered as a result from a combination of a source of sound energy, which is the larynx, modulated by a filter with its characteristics determined by the shape of the vocal tract. This combination results in a shaped spectrum with broadband energy peaks. This model is the so-called source-filter model of speech production.

From a signal processing point of view, the source-filter theory is implemented as follows. In voiced speech, the source of sound is modeled as a series of glottal pulses that represent the glottal volume velocity. The distance of the successive pulses determines the fundamental frequency of the signal. In unvoiced speech, there are no pulses, since the vocal folds do not quasi-periodically open and close. Thus, zero-mean white noise is used to model the characteristics of unvoiced speech. Thereafter, the source signal excites a time-varying filter that represents the vocal tract characteristics, as described earlier. Some models incorporate the lip radiation into the source, since the spectral characteristics of the lip radiation follow a high-pass-like structure. The source is then the derivative of the glottal flow [FLL85, PQR99], and is driven into the vocal tract filter. The accurate estimation of the source, especially in voiced speech, and the vocal tract signal is a notoriously challenging problem is speech processing [Mak75, Alk92, EJM91, LM95, YV98, PQR99, COCM01, BDDD05, FM06, DWBH06, VRR06, VRC07, Deg10, Alk11].

The Sinusoidal Models

The sinusoidal-based models for speech representation generalize the binary glottal excitation model (glottal pulses for voiced and random noise for unvoiced speech) used in the source-filter theory. In these models, the excitation waveform is assumed to be composed of sinusoidal components of arbitrary amplitudes, frequencies, and phases. The process of estimating the sinusoidal parameters, along with the assumptions on the nature of the model, is the main difference among sinusoidal-based representations [MQ86, Gri87, Ser89, SMFS89, Sty96, GS97, JJH99, HVK02, JH02]. Thus, the speech waveform is modeled as time and frequency modulations of these sinusoids, as they pass through the vocal tract, and radiated by the lips. The basic assumption these models hold is that speech does not significantly change in short time intervals, or -as it is often stated in the literature -speech is locally stationary. In practical terms, this means that in a short time analysis window (20 -30 ms), one can model speech as sinusoids that have constant amplitude and frequency values. This is a convenient assumption that is not entirely valid but has been proven useful in practical implementations of the models. An example of sinusoidal frequency decomposition is given in Figure 1.2.

Thesis Subject

In this thesis, the main focus is on Adaptive Sinusoidal Models of Speech with Applications on Speech Modifications and Audio Analysis.

Adaptive Sinusoidal Models (aSMs), although not entirely developed for this thesis, are presented under a common perspective, as sinusoidal models that are adaptive to the local characteristics of the underlying speech signal. In other words, they do not share the stationarity assumption the aforementioned models do. The first aSM was developed in 2010 by Pantazis [START_REF] Pantazis | Adaptive AM-•FM Signal Decomposition With Application to Speech Analysis[END_REF], following the model suggested by Laroche [START_REF] Laroche | A new analysis/synthesis system of musical signals using Prony's method. Application to heavily damped percussive sounds[END_REF], and successively refining it [PRS08, PRS09b, PRS09a, PRS10] until reaching a time where the model is non-parametric and adaptive to the phase characteristics of the analyzed signal. This model is called the adaptive Quasi-Harmonic Model (aQHM) [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF]. Two years later, Degottex and Stylianou proposed a full-band adaptive Harmonic Model (aHM) [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], where the analysis uses a quasi-harmonic scheme for an accurate f 0 estimation but the synthesis is strictly harmonic. A little after, the model which occupies most of this thesis, the extended adaptive Quasi-Harmonic Model was suggested [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF][START_REF] Kafentzis | Robust full-band adaptive sinusoidal analysis and synthesis of speech[END_REF], where adaptation includes both amplitude and phase. The main advantage of aSMs is the accuracy of the parameter estimation process, due to either a very accurate f 0 estimation (aHM) or a very accurate amplitude, frequency, and phase estimation (aQHM, eaQHM). Later, a detailed description of the models will be presented, the advantages of each model will be highlighted, and a comparison on speech analysis and resynthesis will take place.

In addition, Speech Transformations refer to the various modifications that may be applied to the speech signal. It covers a wide area of research from speech production modelling and understanding to speech perception, and from natural language processing, modelling and control of speaking style, to pattern recognition and statistical signal processing.

Speech Transformations (the terms "transformations" and "modifications" are used interchangeably in the rest of this text) have many potential applications in areas such as speech synthesis (e.g. for interactive Voice Response Systems, dialogue systems, text-to-speech systems), entertainment, film and music industry, toys, chat rooms and games, high-end hearing aids, vocal pathology and voice restoration. The objective of the research is to propose and implement algorithms for speech modifications based on the aSMs. More specifically, two main goals are to be addressed: time and pitch-scale modifications. Let us now give a formal definition on these two terms.

• In time-scale modification, the rate of articulation is changed without affecting the perceptual quality of the original speech.

• In pitch-scale modification, the fundamental frequency is changed while preserving the short-time envelope (vocal tract) characteristics as well as the duration of the original speech..

Although other types of transformations do exist, such as voice conversion (where speech of a source speaker is transformed to match as closely as possible the speech of a target speaker), this work is focused only on the modifications with no specific target speaker. It is interesting to note that modifications include terms such as rate of articulation and short-time envelope (vocal tract), implying that there is a combination of sinusoidal-model and source-filter theory in order to provide high-quality speech modifications [START_REF] Quatieri | Speech transformations based on a sinusoidal representation[END_REF][START_REF] Quatieri | Shape-Invariant Time-Scale and Pitch Modifications of Speech[END_REF][START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. Moreover, a variety of interesting applications of the aSMs are presented in this thesis. These include the Modeling of Musical Instrument Sounds and the Analysis and Classification of Expressive Speech using the adaptive Sinusoidal Models (and mostly, the eaQHM). For the former, the eaQHM is compared to the Exponentially Damped Sinusoidal Model (EDSM), which is considered as the state of the art in audio modeling, and the superiority of the eaQHM is demonstrated in terms of musical instrument sounds representation. Regarding the latter, the eaQHM is compared to the standard SM in representing expressive speech, and some preliminary results regarding recognition and classification are shown.

Thesis Contribution

The contribution of this thesis with respect to the aforementioned speech processing fields of the introductory paragraph can be depicted in Figure 1.3.

Speech

Analysis Speech Classification

Audio Modeling Speech Transformations

This thesis Namely, this thesis contributes the following achievements:

• An extension of the adaptive Quasi-Harmonic Model, called the extended adaptive Quasi-Harmonic Model -eaQHM. The eaQHM performs full signal adaptation, e.g. not only the phase is adapted to the signal but also the amplitude as well. Thus, the signal can be modelled more accurately, and this can be demonstrated both subjectively and objectively. This work has been published in ICASSP 2012 [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF].

• The family of adaptive Sinusoidal Models is introduced, which is a collection of models that can locally adapt their parameters on the analyzed signal. This set of models include the recently proposed adaptive Quasi-Harmonic Model -(aQHM), the extended adaptive Quasi-Harmonic Model -(eaQHM), which is proposed in this thesis, and recently suggested adaptive Harmonic Model -(aHM).

• The eaQHM is shown to accurately model unvoiced parts of speech, such as stop consonants and fricatives, as adaptive AM-FM components. Since the eaQHM models both voiced and unvoiced speech accurately, the eaQHM is suggested as a model for a residual-free, full-band model of speech. Part of this work has been published in Interspeech 2012 [START_REF] Kafentzis | On the Modeling of Voiceless Stop Sounds of Speech using Adaptive Quasi-Harmonic Models[END_REF].

• In this context, hybrid and full-band systems of analysis and synthesis of speech based on the aSMs are revised, developed, and described. The eaQHM-based system is shown to stand among adaptive models and outperforms the state-of-the-art in terms of both accuracy in signal reconstruction and perceptual transparency compared to the original signal. This work has been published in ICASSP 2014 [START_REF] Kafentzis | Robust full-band adaptive sinusoidal analysis and synthesis of speech[END_REF].

• Speech modifications are demonstrated: pitch and time-scaling. The modifications are characterized by high quality, simplicity, flexibility, and freedom of common artefacts. Hybrid and full-band analysis, synthesis, and modifications systems based on aSMs are presented and compared. The modifications are of high quality and are implemented by very simple rules, making the transformations very attractive. Specifically, time-scaling is simply implemented by resampling and interpolating the instantaneous amplitude and frequency of the speech signal. For the phase, a very simple approach is suggested based on the notions of relative phase and relative phase delays. For pitch scaling, the estimation of a spectral magnitude envelope, but no phase envelope estimation is necessary, thus significantly simplifying modification. Part of this work has been published in ICASSP 2013 [START_REF] Kafentzis | Time-scale Modifications based on an Adaptive Harmonic Model[END_REF] and ICASSP 2014 [START_REF] Kafentzis | Pitch modifications of speech based on an adaptive harmonic model[END_REF].

• Applications of the eaQHM are discussed:

1. The eaQHM is applied on modelling musical instrument sounds. It is shown that the eaQHM can handle all parts of the sounds, including silence, onset, transient, and stationary part. The accuracy of the reconstruction is shown to be high and the synthesized sound is indistinguishable from the original. Extensive comparisons to the state-of-the-art have been made and the superiority of the proposed model over the competition is demonstrated. This work has been published in EUSIPCO [START_REF] Caetano | Adaptive sinusoidal modeling of percussive musical instrument sounds[END_REF], WASPAA [CKD + 13], and IEEE Transactions on Audio, Speech, and Language Processing [CKMS] (under peer review).

2. The eaQHM is applied on expressive speech analysis and classification. It is shown that aSMs can analyze and resynthesize various styles of expressive speech with very high accuracy and quality, whereas the classification rates are higher than standard sinusoidal approaches in emotion classification. This work has been accepted in EUSIPCO 2014 [START_REF] Kafentzis | Analysis of emotional speech using an adaptive sinusoidal model[END_REF] and submitted in Interspeech 2014 [START_REF] Yakoumaki | Emotional speech classification using adaptive sinusoidal modelling[END_REF].

Thesis Organization

The rest of this thesis is organized in three parts, as follows:

1. In Part I: Adaptive Sinusoidal Models (a) Chapter 2 discusses the adaptive Sinusoidal Models, as standalone signal analysis tools for speech. Part I

Adaptive Sinusoidal Models

Chapter 2

Adaptive Sinusoidal Modeling

The term adaptive Sinusoidal Models -aSM refers to the family of sinusoidal models that are able to adapt to the local characteristics (phase and/or amplitude) of the analyzed speech signal. It should be highlighted that non-adaptive (conventional) sinusoidal models do consider local stationarity of a signal in their representation.

Before going into details, the notions of stationarity and adaptivity should be precisely defined. It should be noted that stationarity is a general term that can be used to characterize any signal; however, adaptation will be defined in a context of representing a signal with a set of complex exponential basis functions. Now let us define the stationary and adaptation terms for sinusoidal analysis.

Consider a set of complex exponential basis functions that a signal is projected onto.

• A complex exponential is called stationary in a well-defined time interval when its frequency and/or its amplitude is constant.

• A complex exponential is called adaptive in a well-defined time interval when its amplitude and/or frequency are computed by taking into account the local characteristics of the signal which is projected onto it.

In general, the aSMs are founded on the principle of projecting a signal segment onto a set of non-parametric, timevarying, non-stationary complex exponential basis functions inside an analysis window, whereas conventional sinusoidal models consider that speech characteristics remain relatively unchanged in a local level, thus their basis functions are stationary. The construction of this set of time-varying basis function depends on the adaptive model, as it will become apparent in this chapter.

First, the heart of all aSMs is presented, the so-called Quasi-Harmonic Model -QHM. The QHM is not an adaptive model itself, but it provides the mechanism for adaptation, that is a frequency correction mechanism, which yields an estimate of the mismatch between the actual and estimated frequencies. This frequency correction is added to the estimated frequencies to allow a closer representation of the underlying signal. An iterative procedure on the parameter estimation leads to the iterative Quasi-Harmonic Model -iQHM, which successively updates the frequencies until a convergence criterion is met. However, both QHM and iQHM hold the local stationarity assumption, that is, all parameters are obtained by projecting the signal on a stationary set of exponential basis functions.

To alleviate the local stationarity assumption, the adaptive Quasi-Harmonic Model -aQHM developed by Pantazis et al [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] is presented and then, we propose an extension, the extended adaptive Quasi-Harmonic Model -eaQHM [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]. These models go one step further and exploit the frequency correction mechanism of QHM to refine their frequency estimations, along with the iterative construction of a time-varying, non-parametric, and non-stationary set of basis functions. Hence, a definition for the adaptation term is given as follows:

Consider a set of complex exponential basis functions that a signal is projected onto.

Adaptation is an iterative construction of the set of complex exponential basis functions according to the local characteristics of the underlying signal, in parallel to a refinement of the instantaneous components of the signal, e.g. the instantaneous amplitude, frequency, and phase.

Moreover, the QHM works as an initializer for aQHM and eaQHM, providing a well-estimated set of frequency tracks. Finally, another recent approach, referred to as the adaptive Harmonic Model -aHM [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF] is briefly presented. The aHM is a purely harmonic model but benefits from the idea of non-stationary basis functions to provide a closer representation of speech via an iterative refinement of the fundamental frequency.

Introduction

In general, an aSM can be described as

x(t) = � K � k=-K C k (t)ψ k (t) � w(t) (2.1)
where ψ k (t) denotes the set of non-stationary basis functions, C k (t) denotes the amplitude term of the model, 2K + 1 is the number of exponentials (hence, K + 1 sinusoids), and finally w(t) is the analysis window with support in [-T, T ].

In conventional sinusoidal models, including the Sinusoidal Model [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF], the Harmonic Model (HM) [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], and others, the set of basis functions ψ k (t) in the analysis part is stationary in frequency and in amplitude. For example, the basis functions in the SM are in the form of

ψ SM k (t) = 1 • e j2πf k t , C SM k (t) = a k (2.2)
where the amplitudes and frequencies of the basis functions are constant inside the analysis window (1 and f k , respectively). However, in the aSMs, as will be described in the following sections, amplitudes and frequencies of the basis functions are non-parametric and depend on the actual characteristics of the analyzed signal:

ψ aSM k (t) = α k (t) • e jφ k (t) , (2.3)
where α k (t) is the time-varying instantaneous amplitude of the k th basis function, φ k (t) is the instantaneous phase of the k th basis function, computed as the integral of the instantaneous frequency, f k (t). The amplitude term of the model, C aSM k (t), is time-varying for all aSMs. Specifically,

ψ aQHM k (t) = 1 • e jφ k (t) , C aQHM k (t) = a k + tb k , (2.4)
ψ eaQHM k (t) = α k (t) • e jφ k (t) , C eaQHM k (t) = a k + tb k (2.5) and ψ aHM k (t) = 1 • e jkφ0(t) , C aHM k (t) = a k + tb k (2.6)
where a k , b k are the complex amplitude and slope of the k th component of the model, φ 0 (t) is the instantaneous phase of the fundamental frequency, computed as the integral of the latter.

Based on this introductory analysis, the description of the aSMs follows next.

The Quasi Harmonic Model (QHM)

As we mentioned in sinusoidal modelling, a signal can be represented as follows:

x(t) = � K � k=-K a k e j2πf k t � w(t), (2.7)
where 2K + 1 is the number of components with complex amplitudes a k at frequencies f k , and w(t) is the analysis window. Let us assume that f k denote the correct frequencies of the signal components. In sinusoidal modelling, frequencies are estimated first (e.g., by peak-picking, by considering harmonics of a fundamental frequency, etc.), before the estimation of the complex amplitudes. The estimated frequencies will be denoted here by fk . Then, we may write:

f k = fk + η k , k = -K, ..., K (2.8)
If the error, η k , is high, then the estimation of the complex amplitudes, a k , is severely biased.

To cope with this problem, in [START_REF] Pantazis | AM-FM estimation for speech based on a time-varying sinusoidal model[END_REF] and [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] the use of the Quasi-Harmonic Model (QHM) for the representation of speech was suggested:

x(t) = � K � k=-K (a k + tb k )e j2π fk t � w(t), (2.9)
where b k denotes the complex slope of the k th component. In the frequency domain, the k th component is written as:

X k (f ) = a k W (f -fk ) + j b k 2π W � (f -fk ) (2.10)
where W (f ) is the Fourier transform of the analysis window and W � (f ) is the derivative of W (f ) over f . In [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF], it was shown that QHM is able to correct frequency mismatches using the projection of b k onto a k :

b k = ρ 1,k a k + ρ 2,k ja k (2.11)
where ja k denotes the perpendicular (vector) to a k . The ρ 1 , ρ 2 parameters are computed as:

ρ 1 = �{a k }�{b k } + �{a k }�{b k } |a k | 2 , (2.12) and ρ 2 = �{a k }�{b k } -�{a k }�{b k } |a k | 2 , (2.13)
where �{a k }, �{b k } and �{a k }, �{b k } are the real and imaginary parts of a k and b k , respectively. By substitution in (2.10) and if we consider the Taylor Series expansion of W (f -fk -

ρ 2,k 2π
), and the value of term W �� (f ) at f k as small, then for small values of ρ 2,k , it can be shown [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] that the k th component can be expressed in time domain as:

X k (f ) = a k � W (f -fk ) - ρ 2,k 2π W � (f -fk ) + j ρ 1,k 2π W � (f -fk ) � (2.14)
If we consider the Taylor series expansion of W (f -fk -

ρ 2,k 2π ), W (f -fk - ρ 2,k 2π ) = W (f -fk ) - ρ 2,k 2π W � (f -fk ) + O(ρ 2 2,k W �� (f -fk )) (2.15)
and the value of term W �� (f ) at f k as small, then for small values of ρ 2,k , it can be shown [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] that the k th component can be expressed in time domain as:

x k (t) ≈ a k � e j(2π fk +ρ 2,k )t + ρ 1,k te j2π fk t � w(t) (2.16)
Thus, ρ 2,k /2π is an estimator of the frequency error η k :

ηk = ρ 2,k 2π = 1 2π �{a k }�{b k } -�{a k }�{b k } |a k | 2 , (2.17)
where ρ 1,k accounts for the normalized amplitude slope of the k th component. In [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF], it was also shown that this correction depends on the magnitude of ρ 2,k and the value of the term W �� (f ) at f k . The frequency mismatch correction mechanism can be depicted in Figure 2.1, where in Figure 2.1a, a harmonic template is estimated from the known f 0 value, and in Figure 2.1b, the frequency correction estimates ηk are applied. The estimation of a k , b k is performed via Least Squares (LS) in the following way:

Let us define the parameter vector x as

x = � a b � (2.18)
The error is defined in discrete time by

�(a, b) = N � n=-N |s[n] -s q [n]| 2 (2.19) = N � n=-N (s[n] -s q [n]) * (s[n] -s q [n]) (2.20)
where s[n] is the original windowed signal, s q [n] is the Quasi-Harmonic representation of Eq. (2.7), and 2N + 1 is the window size. In matrix notation, if we separate the window values from the samples, Eq. (2.20) becomes

�(a, b) = (Ws -Ws q ) H (Ws -Ws q ) (2.21) = (W(s -s q )) H W(s -s q ) (2.22) = (s -s q ) H W H W(s -s q ) (2.23)
Now, the QHM can be written in matrix notation as

s q = N � n=-N (a k + nb k )e j2πf k n/fs (2.24) = N � n=-N a k e j2πf k n/fs + N � n=-N nb k e j2πf k n/fs (2.25) = E 0 a + E 1 b = � E 0 |E 1 � � a b � = Ex (2.26)
where

E 0 = (E 0 ) n,k = e j2π f k n fs , E 1 = (E 1 ) n,k = n(E 0 ) n,k = ne j2π f k n fs (2.27) and E = � E 0 |E 1 � (2.28)
Hence, the minimization comes to

∂�(x) ∂x = 0 (2.29) ∂ ∂x (s -Ex) H W H W(s -Ex) = 0 (2.30)
The solution is given by

x = � â b� = (E H W H WE) -1 E H W H Ws (2.31)
Finally, the signal can be locally approximated as

x(t) = K � k=-K |â k |e j(2π( fk +η k )t+ φk ) w(t) (2.32)
where φk = ∠â k (2.33)

Although the QHM has been proved to perform better than standard Sinusoidal or Harmonic Models [START_REF] Pantazis | AM-FM estimation for speech based on a time-varying sinusoidal model[END_REF], it still assumes that the signal is stationary inside the analysis window.

The adaptive Quasi-Harmonic Model, aQHM

To alleviate the non-stationarity assumption, an adaptive Quasi-Harmonic Model (aQHM) has been suggested by Pantazis et al [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF].

x(t) = � K � k=-K (a k + tb k )e j( φk (t+ti)-φk (ti)) � w(t), t ∈ [-T, T ] (2.34)
where φ k (t) denotes the instantaneous phase function of the k th component and t i is the center of the analysis window.

The term b k plays the same role as in QHM; it provides a means to update the frequency of the underlying sine wave at the center of the analysis window, t i . Given the samples of the input signal in vector s, the model parameters are found via LS, as in QHM:

� â b� = (E H W H WE) -1 E H W H Ws (2.35)
where W is the matrix containing the window values in the diagonal, s is the input signal vector, the matrix E is defined as E = [E 0 |E 1 ], the submatrices E i , i = 0, 1 have elements given by

(E 0 ) n,k = e j(φ k (tn+ti)-φ k (ti)) (2.36)
and

(E 1 ) n,k = t n e j(φ k (tn+ti)-φ k (ti)) = t n (E 0 ) n,k , (2.37)
and the instantaneous phase of the k th component can be computed as

φk (t) = φk (t i ) + � ti+t ti 2πf k (u)du, t ∈ [-T, T ], (2.38)
where f k (t) is the frequency trajectory of the k th component.

Using the definition of phase, the instantaneous phase of a single component, φ(t), is computed as the integral of the instantaneous frequency, f (t). The instantaneous frequency is obtained from an initial parameter estimation, such as in QHM. In order to interpolate phase values between two successive time instants, t i , t i+1 , the following equation is proposed:

φ(t) = φ(t i ) + � t+ti ti 2π f (u)du (2.39)
where φ(t i ) is the instantaneous phase estimate at time instant t i . However, this solution does not take into account the frame boundary conditions at time instant t i+1 . Hence, there is no guarantee that the phase value at time instant t i+1 ,

φ(t) � � t=ti+1 = φ(t i+1 ) + 2πM (2.40)
where M is an integer appropriately selected to be as close as possible to

M = round � φ(t i+1 ) -φ(t i ) 2π � (2.41)
where round(•) is the rounding to closest integer function. In order to ensure phase continuation over frame boundaries, it is suggested [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] to modify Eq. (2.39) as:

φ(t) = φ(t i ) + � t+ti ti (2π f (u) + c(u))du (2.42)
where c(u) is given by

c(u) = r(t i+1 ) sin � π(u -t i ) t i+1 -t i � (2.43)
This way, Eq. (2.40) is satisfied if we choose r(t i+1 ) as

r(t i+1 ) = π(φ(t i+1 ) + 2πM -φ(t i+1 ) 2(t i+1 -t i ) (2.44)
where M is computed as in Eq. (2.41).

In contrast to QHM, where the argument of the basis functions is parametric and stationary, in aQHM the argument of the basis functions is neither parametric nor necessarily stationary. Moreover, the aQHM basis functions use the instantaneous phases which have been estimated from the input signal. In that sense, these are also adaptive to the estimates of the current phase characteristics of the signal. The process of successive adaptations is shown in Figure 2.2. However, only phase adaptation is allowed in this model. Naturally, amplitude adaptation should be included as well,

Basis

The extended adaptive Quasi-Harmonic Model -eaQHM

In this thesis, we propose an extension of aQHM to include amplitude adaptation, called the extended adaptive Quasi-Harmonic Model (eaQHM):

x(t) = � K � k=-K (a k + tb k ) αk (t)e j( φk (t+ti)-φk (ti)) � w(t), t ∈ [-T, T ] (2.45)
where

αk (t) = A k (t + t i ) A k (t i ) (2.46)
where t i is still the center of the analysis window and A k (t) is the instantaneous amplitude of the k th component obtained from a previous adaptation step. The estimation of the unknown parameters of eaQHM is similar to that of QHM:

� â b� = (E e H W H WE e ) -1 E e H W H Ws (2.47)
where W is the matrix containing the window values in the diagonal, s is the input signal vector, the matrix E e is defined as E e = [E e0 |E e1 ], and the submatrices E ei , i = 0, 1 have elements given by

(E e0 ) n,k = α k (t n )e j(φ k (tn+ti)-φ k (ti)) (2.48) and (E e1 ) n,k = t n α k (t n )e j(φ k (tn+ti)-φ k (ti)) = t n (E e0 ) n,k , (2.49)
It is clear that the basis functions are adapted to the local amplitude characteristics of the signal. Note that the instantaneous amplitude A k (t) is divided by A k (t i ), so as to have unit value at the center of the analysis window.

In picturing the amplitude and frequency modelling of the eaQHM within the analysis window, Figures 2.3 and 2.4 show how conventional sinusoidal models like HM, SM, or QHM behave inside their analysis window. Their exponential basis functions are stationary in frequency, thus being inefficient on the representation of highly non-stationary frequency curves. The same argument applies for amplitude curves, although frequency estimation is far more important than amplitude estimation. Inside the analysis window, the amplitude trajectory of a partial (solid grey line) is assumed to be constant for stationary sinusoidal models (dotted line), while eaQHM (dashed line) iteratively adapts to the shape of the instantaneous amplitude.

Algorithm for Adaptive Sinusoidal Analysis

Adaptive Sinusoidal models require an initialization step, so QHM will be used for this purpose, although any frequency estimation algorithm can be used. Thus, the initials steps consist of the following:

f 0 k (t i ) = f 0 k (t i-1 ) + ηk (2.50) Â0 k (t i ) = |a i k |, φ0 k (t i ) = ∠a i k (2.51)
where t i is the center of the i th analysis frame. The AM-FM decomposition algorithm using eaQHM is provided in Algorithm 1. The convergence criterion for the algorithm was selected to be the following:

SRER m-1 -SRER m SRER m-1 < � (2.52)
Algorithm 1 AM-FM decomposition using eaQHM Require: Provide initial frequency estimate f 0 k (t 1 )

for frame i = 1 to L do Compute a i k , b i k using LS Update f 0 k (t i ) using Eq. (2.50) Compute Â0 k (t i ) and φ0 k (t i ) using Eq. (2.51) Set f 0 k (t i+1 ) = f 0 k (t i ) end for Interpolation of the parameters { Â0 k (t), f 0 k (t), φ0 k (t)} Adaptation of amplitudes and phases: for adaptation m = 1 to • • • do for frame i = 1 to L do Compute a i k , b i k using φm-1 k (t) of Eq. (2.42) and Eq. (2.47) Set f m k (t i ) = f m k (t l-1 ) + ηk Set Âm k (t i ) = |a i k | and φm k (t i ) = ∠a i k end for end for Interpolation of the parameters { Âm k (t), f m k (t), φm k (t)}
where SRER m is the Signal-to-Reconstruction-Error Ratio of the resynthesized signal in the m th adaptation, defined as

SRER = 20 log 10 σ x(t) σ x(t)-x(t) (2.53)
where σ x denotes the standard deviation of x(t), x(t) is the actual signal and x(t) is the reconstructed signal, and where � is a threshold for convergence, typically set to 0.02. As a last step of the algorithm, the signal can finally be approximated as the sum of its AM-FM components:

x(t) = K � k=-K Âk (t)e j φk (t)

Evaluations

In this section, a performance comparison between the two adaptive quasi-harmonic models described so far on synthetic and real voiced speech is presented.

Validation on Synthetic Signals

For the purpose of demonstrating the performance of eaQHM, we consider a two-component signal with modulated amplitudes and frequencies, defined as:

x(t) = a 1 (t)e j(2πf1t+φ1(t)) + a 2 (t)e j(2πf2t+φ2(t)) (2.54)
where the above parameters are given in To test the robustness of the estimations provided by the eaQHM, additive white Gaussian noise of 20 and 10 dB SNR was added to the synthetic signal x(t) described above. For comparison purposes, results for the aQHM are also provided. The performance of the algorithms is measured through the Mean Absolute Error (MAE) for amplitudes and frequencies. The MAE of a parameter θ is defined as:

F i = f i + 1 2π d dt φ i (t), for i = 1,
M AE{ θ} = 1 M M � i=1 | θ(i) -θ| (2.55)
where θ (i) is the estimated parameter at the i th simulation, and M is the number of Monte Carlo simulations. The results shown in this section are based on M = 10000 Monte Carlo simulations and the length of a Hamming analysis window for both models was 10 ms. The analysis step size was set to 1 sample. Table 2.2 presents the MAE and SRER scores for the aforementioned levels of noise.

Validation on Voiced Speech

The next step is to test the proposed model on real speech, and in particular, on voiced speech signals. The suggested iterative AM-FM decomposition algorithm based on aQHM/eaQHM can be applied on voiced speech signals in a straightforward way. Actually, the aQHM/eaQHM algorithm can be applied on a large voiced speech segment. Indeed, assuming that voiced speech is quasi-periodic and that the frequency content of voiced speech signals does not change very fast, then we only need to provide the fundamental frequency of the first voiced frame at the beginning of the voiced 0 0.05 0.1 1 1.5 segment, f 0 (t 1 ), and then assume f 0 k (t 1 ) = kf 0 (t 1 ). Applying QHM analysis on the first voiced frame, an updated set of fk can be obtained for that frame. The updated set of frequencies can then be used as initial estimates for the next voiced frame. Continuing in this way, the whole voiced region will be analyzed by providing just the fundamental frequency for the first frame of the voiced segment. It is worth noting that the accuracy of the fundamental frequency estimator is not crucial for aQHM/eaQHM, since frequency mismatches are easily corrected. For our purpose, we consider a voiced speech signal from the CMU-ARCTIC database with sampling frequency F s = 16 kHz and duration of about 0.35 sec. For both algorithms, the number of harmonics was set to K = 40 and an estimate of the fundamental frequency of the beginning of the segment was given to the algorithm. At most 10 adaptations were allowed to the models. The analysis window size was 2.5 pitch periods and the analysis step size was 1 sample. In the following, the signals are considered up to a fixed maximum voiced frequency (5500 Hz). The original signal, along with the aQHM/eaQHM reconstructed signals and corresponding reconstruction errors, are shown in Figure 2.7. To objectively compare the performance of both algorithms, the SRER defined in Eq. (2.53) was used. The SRER was 41.2607 dB for aQHM and 45.2166 dB for eaQHM. Two adaptations for aQHM and three adaptations for eaQHM were necessary for the models to converge. To confirm these results, a large-scale objective test was performed. Using three different step sizes, namely 1ms, 2ms, and 4ms, we analyzed and reconstructed about 50 minutes of voiced speech from 3 speakers in the ARCTIC database. The sampling frequency of the speech signals was downsampled to 16kHz. A Hamming window of fixed length was used; 3 times the average pitch period of the speaker. The same window was used for both aQHM and eaQHM. The number of components was set to K = 30. The average and standard deviation of the SRER (in dB) is provided in Table 2.3, along with various time-steps. slightly more iterations than aQHM.

The adaptive Harmonic Model -aHM

For completeness in this section, a very brief review of the adaptive Harmonic Model (aHM) is presented [START_REF] Degottex | A full-band adaptive harmonic representation of speech[END_REF]. The adaptive Harmonic Model can be mathematically described as:

x(t) = K � k=-K a k (t)e jkφ0(t) w(t) (2.56)
where a k (t) is a time-varying complex function that copes with the amplitude and the instantaneous phase of the k th harmonic component, w(t) is the analysis window with support in [-T, T ], while K is the number of the harmonics, and φ 0 (t) is a real function defined as the integral of the fundamental frequency f 0 (t):

φ 0 (t) = � t 0 2πf 0 (u)du (2.57)
The aHM has been successfully applied in speech analysis. Although it supports a strictly harmonic representation, the adaptation part comes from the aQHM-based theory: aHM projects a speech segment on a time-varying, harmonicallyrelated set of exponential basis functions:

x(t) = K � k=-K (a k + tb k )e jkφ0(t) w(t) (2.58)
where x(t) is the modeled analytic signal of the speech segment, and a k , b k are the already introduced parameters of the QHM. To obtain the a k , b k parameters, a Least-Squares minimization is set up:

� â b� = (E h H W H WE h ) -1 E h H W H Ws (2.59)
where W is the matrix containing the window values in the diagonal, s is the input signal vector, the matrix

E h = [E h0 |E h1 ],
and the submatrices E hi , i = 0, 1 have elements given by

(E h0 ) n,k = e jkφ0(tn) (2.60)
and

(E h1 ) n,k = t n e jkφ0(tn) = t n (E h0 ) n,k , (2.61)
a solution structure similar to the one described earlier for the Quasi-Harmonic Models. The parameters âk , bk provide a frequency correction estimate ηk for each frequency. These corrections combine together to form a correction term related to the fundamental frequency, as a mean of the correction terms ηk relative to f 0 :

f corr 0 = 1 K K � k=1 ηk k (2.62)
where K is the number of harmonics, and K is small at first (K ≈ 4 -5). The f corr 0 term updates the f 0 estimation (and thus, the φ 0 of Eq. (2.57)). When | f corr 0 | is small, the current set of harmonics are considered to have converged to the actual frequency values, and thus more harmonics can be added, and a new f corr 0 can be estimated. Based on this iterative process, a dedicated algorithm that successively refines the fundamental frequency curve to accurately localize the frequencies of the harmonics up to the Nyquist frequency is built. This algorithm is named as the Adaptive Iterative Refinement -AIR of f 0 . Further information on AIR can be found in [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], but an illustrative example will be presented here.

Let us consider the spectrum of a speech segment, which is purely harmonically modeled in its low band, where the frequency mismatch is considered small, as in Fig. 2.8a. Using the frequency correction mechanism of the QHM, a better estimate of the fundamental is obtained, and thus the first few harmonics in the low band are refined (Fig. 2.8b). Then, more harmonics are added (Fig. 2.8c) in the process, and further correction is applied on the fundamental frequency, resulting in more precise estimates of the harmonics up to the mid-band (Fig. 2.8d). This scheme continues until all harmonics are well localized up to the Nyquist Frequency (Figs. 2.8e, 2.8f). Compared to aQHM and eaQHM, the aHM adapts the parameters of the model only once during analysis. However, during AIR-f 0 , the a k and b k parameters of aQHM are iteratively estimated, since they are involved in the refinement of the f 0 . The aHM will be presented in more (a) At first, a small number of harmonics is used, where the frequency error is considered small. depth in the following chapter.

Number of parameters

For analysis and synthesis of a single frame, Table 2.4 presents an overview of the analysis and synthesis complexity of the SM, aQHM, eaQHM, and aHM to allow comparison. Complexity is considered as the number of parameters per frame each model requires to estimate (analysis) and represent (synthesis) K sinusoids. Please notice that the SM has the same complexity in the analysis and synthesis stages, while aQHM and eaQHM fit more parameters iteratively (a few times until convergence) during the analysis stage than for resynthesis. Finally, although the aHM is a harmonic model and consequently has less parameters for synthesis (2K + 2), when the AIR-f 0 algorithm is used, the a k and b k parameters of QHM are involved in the fundamental frequency refinement, and that implies an increase in complexity.

Conclusions

In this chapter, the adaptive sinusoidal models were presented. Three variants were discussed: the aQHM, the aHM, and the eaQHM which is proposed in this thesis. In the aQHM, the frequency (and thus, the phase) of the signal is adapted to the local characteristics of the analyzed signal. In the eaQHM, the amplitude, along with the frequency of the signal, is included in the adaptation process in a straightforward way. In the aHM, the instantaneous phases of the basis Real parameters per frame SM aQHM eaQHM AIR-aHM Analysis 3K + 1 5K + 1 5K + 1 (4K + 1) + 1 Synthesis 3K + 1 3K + 1 3K + 1 (2K + 1) + 1 Table 2.4: Comparison of model complexity between SM, aQHM, eaQHM, and AIR-aHM for the analysis and synthesis stages. The table presents the number of real parameters per frame as a function of the number of sinusoids K to estimate (analysis complexity) and to represent (synthesis complexity) signals. Please note that the +1 term in all models corresponds to the mean value (DC component) of the signal.

functions are integer multiples of the instantaneous phase of the fundamental frequency, which is iteratively estimated via a dedicated algorithm. The parameters of all models are computed using Least-Squares minimization. Experiments on synthetic signals showed that eaQHM performs better than aQHM in terms of MAE and SRER. The robustness of the eaQHM in the presence of white Gaussian noise is also demonstrated. Experiments on voiced speech using the ARCTIC database showed that eaQHM outperforms aQHM in terms of signal reconstruction.

Part II

Speech Analysis, Synthesis, and Modifications

Chapter 3

Related Work

In this Chapter, related work on the subject of speech analysis, synthesis, and modifications is presented. For the sake of clarity and convenience, related work is divided into two subsections: parametric and non-parametric techniques. A presentation of the most important schemes will be carried out in this work. All methods of speech analysis, modification, and synthesis will be described starting from earlier approaches up until the latest ones, in order to show the evolution of the scientific area throughout the years. Also, this way it is easier to highlight major improvements over the methods. However, emphasis will be given in parametric approaches, since they are closer to the models-in-hand.

Non Parametric Techniques

Non-parametric techniques are mostly based on the Short-Time Fourier Transform (STFT), with the so-called Phase Vocoder being the most well-known representative. Phase Vocoder is almost totally a frequency based technique. Other approaches include time domain techniques, such as the Overlap-Add method (OLA) and its variants, with Pitch-Synchronous OLA (PSOLA) being a milestone. The most significant ones from each perspective are discussed next.

The Phase Vocoder

The Short-time Fourier Transform (STFT) is the basis for the Phase Vocoder. Here, the basic mechanisms behind STFT analysis, modifications, and synthesis are presented.

Analysis-Synthesis

The idea behind STFT analysis is that a Fourier Transform is performed on a windowed speech segment, then a shift in time is made, and another Fourier Transform is applied, and so on, as it is depicted in Figure 3.1. The successive window locations are called analysis time instants, denoted by t a (k). Usually, the time shift is constant, and called frame rate. That means the analysis time instants are regularly spaced, t a (k) = lR, where R is the frame rate.

Thus, the STFT can be mathematically described as

X(t a (k), ω) = ∞ � n=-∞ h u [n]x[t a (k) + n]e -jωn (3.1)
where h u [n] is the analysis window, and x[n] is the speech signal. The role of the window is to select and weighten the speech segment to be analyzed. The window is of finite duration and symmetric, and is of low-pass type. X(t a (k), ω) is also called the short-time analysis spectrum around time instant t a (k). It can be easily observed that for a given value of ω, the STFT X(t a (k), ω), can be viewed as the output of a complex band-pass filter centered at ω, and sampled at the successive time-instants t a (k). The STFT can also be described in terms of polar coordinates, magnitude and phase (hence the name):

X(t a (k), ω) = M (t a (k), ω)e jφ(ta(k),ω) = M (k)e jφ(k) (3.2)
In practice, the STFT is evaluated at discrete frequencies, Ω l = 2πl N , using the FFT, where N is the length of the FFT.

Having this time-frequency representation of speech, the modification stage consists of applying the desired modifications to the stream of short-time analysis spectra, to obtain the short-term synthesis spectra, Y (t a (k), ω), and then to synchronize them on a new set of time instants, called the synthesis time instants, t s (k). This stream of synthesis time instants is derived from the analysis time instants, according to the desired modification, as it will be explained later. Finally, the synchronized short-time synthesis spectra should be combined to produce the desired modified signal. However, it should be noted that, in general, there is no time-domain signal y[n] that has as its STFT the Y (t s (k), ω). The stream of short-time synthesis spectra must satisfy strong consistency conditions, described in [START_REF] Portnoff | Time-frequency representation of digital signals and systems based on short-time fourier analysis[END_REF]. That is because the stream of short-time synthesis spectra correspond to overlapping short-time signals. A solution to that was proposed by Griffin [START_REF] Griffin | Signal estimation from modified short-time fourier transform[END_REF], by using the euclidean norm to estimate the signal y[n] whose STFT Ŷ (t s (k), ω) around time instants t s (k) best fits the modified STFT Y (t s (k), ω). Solution to this minimization problem can be derived in closed form and is similar to the overlap-add procedure.

However, for time and pitch scale modifications, it is convenient to refer to a parametric model of speech production, although its parameters are not explicitly estimated. Portnoff [START_REF] Portnoff | Short-time fourier analysis of sampled speech[END_REF] introduced a flexible quasi-stationary model of speech, and this will be discussed next. The connection between STFT analysis and this model will be presented, as well as the time and pitch scale modifications, with reference to this speech production model.

Stationary representation of speech

As it is generally accepted, the speech production model is based on a timevarying linear filter driven by an excitation signal, which is either a sum of narrow-band signals with nearly harmonic instantaneous frequencies (voiced speech), or a stationary random sequence, with a flat power spectrum (unvoiced speech). If we denote as g n [m] the response of the system at time n to a unit-sample applied at time

(n -m), then G(n, ω) is the Fourier transform of g n [m] with respect to index m, i.e. ∞ � m=-∞ g n [m]e -jωm = G(n, ω)e jψ(n,ω) (3.3)
G(n, ω) and ψ(n, ω) are referred to as the time-varying amplitude and phase of the system, respectively. Thus, g n [n] can be considered as nearly constant, since it represents the movements of physical articulators and it is usually relatively slow, compared to the variation of the speech waveform. In voiced speech, the excitation waveform e[n] is represented as a sum of harmonic complex exponentials with unit amplitudes, zero inital phase, and slowly varying fundamental frequency function 2π/P [n], where P [n] is the local pitch period. Mathematically,

e[n] = P [n]-1 � k=0 e jΦ k [n] (3.4)
where Φ k [n] is the excitation phase of the k th harmonic. That phase is defined as the integral of the time-varying frequency ω k [n] = 2πk/P [n]:

Φ k [n] = n � m=0 ω k [m] = n � m=0 2πk P [m]
(3.5)

Considering P [n] as nearly constant around time instant n, the following approximation can be derived:

Φ k [m] ≈ Φ k [n] + ω k [n][m -n], |m -n| < � (3.6)
where � is a small constant. Now, the output of the time-varying filter is given by the convolution of the excitation and the filter impulse response:

x[n] = ∞ � m=-∞ g n [m]e[n -m] (3.7)
Using the assumption of constant P [n] for the duration of g n [m], the excitation signal can be replaced by its local harmonic representation:

x[n] = P [n]-1 � k=0 G(n, ω k [n])e j(Φ k [n]+ψ(n,ω k [n])) = P [n]-1 � k=0 A k [n]e jθ k [n] (3.8)
where A k [n] is the system amplitude at the harmonic frequency ω k [n]. The phase θ k [n] of the k th harmonic is the sum of the excitation phase Φ k [n] and the system phase

ψ k [n] = ψ(n, ω k [n]): θ k [n] = Φ k [n] + ψ(n, ω k [n]) = Φ k [n] + ψ k [n] (3.9)
The term θ k [n] is referred to as the instantaneous phase of the k th harmonic. Finally, since the system phase is slowly varying, the instantaneous phase can be expressed as

θ k [m] = θ k [n] + ω k [n](m -n), |m -n| < � (3.10)
Further information on how these parameters are estimated are given in [START_REF] Moulines | Non-parametric techniques for pitch-scale and time-scale modification of speech[END_REF]

Pitch and Time Scale Modifications

With this brief description of the speech production model in hand, time and pitch scaling modifications can be addressed.

The object of time scaling is to alter the rate of articulation without affecting the spectral content. The pitch and the time evolution of the formant structure should be time-scaled, but not modified in any other way. A time-scale warping function would be useful, to map time instants in the original signal and time instants in the modified signal. This mapping t → t � = D(t) is called the time-scale warping function, and is often convenient to be expressed as an integral:

D(t) = � t 0 β(τ )dτ (3.11)
where β(τ ) is positive and is called the time modification rate. For a fixed β(τ ) = β, the time scaling warping function is linear, and if β > 1, then speech is slowed down (time-scale expansion), whereas if β < 1, speech is sped up (timescale compression). For time-varying time-modification rates, the time-scale warping function is non linear. Now, with reference to the speech production model introduced in the previous section, the speech parameters should be modified as follows:

n � → P � [n � ] = P [D -1 (n � )] (3.12) n � → A � i [n � ] = G � i [n � ] = G[D -1 (n � ), ω i [D -1 (n � )]] (3.13) n � → θ � i [n � ] = Φ � i [n � ] + ψ � D -1 (n � ), 2iπ P [D -1 (n � )] � (3.14) n � → Φ � i [n � ] = n � � m=0 2iπ P [D -1 (n � )] (3.15)
The object of pitch scaling is to alter the fundamental frequency of speech without affecting the spectral envelope. A time-varying pitch-modification factor t → a(t) > 0 is defined, which affects the pitch contour. Hence, the new pitch contour will be

n → P � [n] = P [n] a[n] (3.16)
For a(t) > 1, the local pitch is increased by a factor of a(t), whereas for a(t) < 1, the pitch is lowered by the same factor. Thus, the speech parameters are modified as follows:

n � → P � [n � ] = a[n � ]P [n � ] (3.17) n � → A � i [n � ] = G � i [n � ] = G[n � , a[n � ]ω i (n � )] (3.18) n � → θ � i [n � ] = Φ � i [n � ] + ψ(n � , a[n � ]ω i [n � ]) (3.19) n � → Φ � i [n � ] = n � � m=0 a[m]ω i [m]
(3.20)

The Speech Transformation and Representation using Adaptive Interpolation of Weighted Spectrum (STRAIGHT) model

STRAIGHT is a hight-quality vocoder that uses pitch-adaptive spectral analysis combined with a surface reconstruction method in the time-frequency region, and an excitation source design based on phase manipulation. It preserves the bilinear surface in the time-frequency region and allows for over 600% manipulation of such speech parameters as pitch, vocal tract length, and speaking rate, without further degradation due to the parameter manipulation. STRAIGHT is developed based on human speech perception system, which decomposes input sounds in terms of excitation (source) and resonant (filter) characteristics and it performs a periodic/aperiodic decomposition of the source signal and the estimation of pitch, as well as the spectrum. The quality of the resynthesized speech is high. However, the computational complexity is also very high and other parameters related to voice quality are not explicitly estimated.

More specifically, STRAIGHT decomposes the speech signal into three terms

• A smooth spectrogram, free from periodicities in time and frequency

• An f 0 contour, and

• A time-frequency periodicity map, which captures the spectral shape of the noise and also its temporal envelope

During the analysis, an f 0 contour is accurately estimated using a fixed-point algorithm. Then, this f 0 estimate is used to smooth out periodicity in the short-time spectrum using an f 0 -adaptive filter and a surface reconstruction method. The result is a smooth spectrogram that captures vocal tract and glottal filters, but is free from the influence of f 0 . During synthesis, pulses or noise with a flat spectrum are generated in accordance with voicing information and the f 0 contour. Speech is resynthesized from the smoothed spectrum and the pulse/noise component using an inverse FFT with an OLA technique.

From a modifications point of view, the step to be followed are very simple:

• Time-scale modification reduces to duplicating/removing ST slices from the STRAIGHT spectrogram and aperiodicity map.

• Pitch-scale modification reduces to modifying the f 0 contour

The three terms in STRAIGHT can be manipulated independently, which provides increased flexibility. STRAIGHT allows very high-factor prosodic modifications while maintaining the naturalness of the synthesized speech. The main disadvantage of STRAIGHT is its computational intensity. A schematic diagram for STRAIGHT is given in Figure 3.2.

The Overlap-Add (OLA) Methods

Pitch-Synchronous Overlap-Add (PS-OLA) is the basis for most time-domain techniques for time and pitch scale modifications. A simple description of this technique follows next, along with its most important variants.

Analysis-Synthesis

The analysis step consists of decomposing the speech signal x[n] into a stream of short-time analysis signals, which can be denoted as x[t a (k), n], where t a (k) is the index of the short-time signal. the signal in order to extract the short-time waveform, i.e.

x[t a (k), n] = x[n]h[n -t a (k)] (3.21)
The time instants t a (k) are called analysis pitch-marks, and they are set in a pitch-synchronous manner on the voiced segments of speech and in a constant manner on the unvoiced ones. The length of the analysis window, denotes as T , varies between different PSOLA schemes, but it is proportional to the local pitch period, P (k). For example, for Time-Domain PSOLA (TD-PSOLA), T is around 2P (k), whereas for Frequency-Domain PSOLA (FD-PSOLA), T is around 4P (k). Usually, a Hanning window is used in most implementations, although other choices can be made, according to mainlobe bandwidth and side-lobe attenuation. The synthesis step is about transforming the stream of short-time analysis signals into a stream of short-time synthesis signals. These synthesis signals are synchronized on a new set of time instants, called synthesis time instants, t s (k), which are referred to as the synthesis pitch-marks. The stream of synthesis pitch-marks is determined from the analysis pitch-marks, according to the desired prosodic modification. Also, a mapping t s (k) → t a (k) is determined, which controls which analysis short-time signals should be selected for any given synthesis pitch-mark. A simple case consists of elimination or duplication of the analysis short-time signals, under the assumption that there is a one-to-one mapping between analysis and synthesis pitch-marks. In a more sophisticated approach, where there is no longer a one-to-one pitch-mark mapping, interpolation between successive short-time analysis signals lying closest to the synthesis pitchmark is performed.

Finally, the synthetic output y[n] is obtained by applying a weighted least-squares OLA procedure on the synchronized short-time synthesis signals. The analysis and synthesis windows are the same for TD-PSOLA, whereas in FD-PSOLA, the synthesis window is different, in order to account for the inherent changes in the frequency domain.

Pitch and Time Scale Modifications

The modifications can be considered as an intermediate step between analysis and synthesis. At first, the synthesis pitch-marks should be generated according to the desired pitch or/and time-scale modification, and then each synthesis pitch-mark is mapped with one or more analysis pitch-mark. Finally, the synthesis is carried out as described in the previous paragraph.

For pitch-scale modifications, the analysis pitch-marks are positioned pitch-synchronously, i.e.

t a (k + 1) -t a (k) = P (t a (k)) (3.22)
where P (t) is the pitch contour function t → P (t), and is considered constant within the analysis pitch-marks:

P (t) = P (t a (k)), t a (k) ≤ t < t a (k + 1) (3.23)
A new pitch contour, P � (t), is specified according to the desired pitch modification. The stream of synthesis pitch-marks should be positioned pitch-synchronously with respect to this new pitch contour, that is

t s (k + 1) = t s (k) + P � (t s (k)) (3.24)
For each synthesis pitch-mark, t s (k), we have

P � (t s (k)) ≈ P (t s (k)) β(t s (k)) (3.25)
where β(t s (k)) is the pitch-scale modification factor. A recursive equation is available to determine these synthesis pitch-marks: For time-scaling, a time-scale modification factor a s is associated with each analysis pitch-mark, from which a timescale warping function can be formulated:

t s (k + 1) -t s (k) = 1 t s (k + 1) -t s (k) � ts(
D(t a (1)) = 0 (3.28) and D(t) = D(t a (k)) + a s (t -t a (k)), t a (k) ≤ t < t a (k + 1) (3.29)
Since the pitch contour must be preserved in time-scaling, a stream of synthesis pitch-marks is obtained from the analysis pitch-marks, using the time-scale warping function. The pitch in the time-scaled signal at time t should be close to the pitch in the original signal at time t � = D -1 (t). Now, a stream of synthesis pitch-marks should be found, such that For this purpose, the introduction of the so-called virtual pitch-marks, t v (k), in the original signal is convenient, such that

t s (k + 1) = t s (k) + P � (t s (k)) (3.
t s (k) = D(t v (k)), t v (k) = D -1 (t s (k)) (3.31)
Hence, we need to specify t s (k + 1) so as t s (k + 1) -t s (k) is approximately equal to the original pitch at time t v (k).

Mathematically, this can be expressed as

t s (k + 1) -t s (k) = 1 t v (k + 1) -t v (k) � tv(k+1) tv(k) P (t)dt (3.32)
with t s (k + 1) = D(t v (k + 1). In other words, the synthesis pitch period t s (k + 1) -t s (k) at time t s (k) equals to the mean value of the pitch in the original signal calculated over the "virtual" interval [t v (k), t v (k + 1)]. Time scale modification for TD-PSOLA is shown in Fig. 3.5. In Fig. 3.6, phase-vocoder and TD-PSOLA time-scaling are compared.

Other Approaches

Other variants of PSOLA include Frequency Domain PSOLA (FD-PSOLA) [START_REF] Charpentier | Diphone synthesis using an overlap-add technique for speech waveforms concatenation[END_REF], which is only used for pitchscaling and differs from TD-PSOLA in the definition of the short-time synthesis signals. Also, the modification is performed in the frequency domain of the short-time analysis spectrum, by adjusting the spacing between the harmonics by a factor β s .

Another approach is Linear Predictive PSOLA (LP-PSOLA) [START_REF] Moulines | Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones[END_REF]. The PSOLA scheme is embedded in a residual-excited vocoder, where AR models are fitted on the speech signal at a pitch-synchronous rate, and prosodic modifications are performed on the residual, which is obtained by methods of inverse filtering. The output is synthesized by filtering the modified residual with the time-varying synthesis filters. More details can be found in [START_REF] Moulines | Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones[END_REF] [START_REF] Valbret | Voice transformation using psola techniques[END_REF].

Similar to TD-PSOLA is the Waveform Similarity Overlap-Add (WSOLA) [START_REF] Verhelst | An Overlap-Add Technique Based on Waveform Similarity (WSOLA) for High Quality Time-Scale Modification of Speech[END_REF] method, where neighbouring frame similarities are exploited using autocorrelation techniques in order to avoid pitch period discontinuities or phase jumps at the synthesis boundaries of the traditional TD-PSOLA or SOLA [START_REF] Roucos | High-quality time-scale modification for speech[END_REF].

The pioneering work of Griffin and Lim [START_REF] Griffin | Multiband Excitation Vocoder[END_REF] on the Multiband Excitation (MBE) Vocoder should be mentioned. In MBE Vocoder, bands of the spectrum are separated as voiced or unvoiced. Methods to estimate the parameters of the speech model are presented and methods to synthesize speech from the model parameters are described. Specifically, the excitation and the spectral envelope parameters are estimated so that the synthesized spectrum fits, in the Least Squares sense, the original spectrum. However, the system is only capable of time-scaling speech signals.

Improvement over the Phase Vocoder have been proposed over the years. Laroche et al [START_REF] Laroche | Improved Phase Vocoder Time Scale Modification of Audio[END_REF] [Puc95] proposed an explanation for the phasiness problem (often referred to as reverberation) and suggested an improvement for phase calculation that significantly reduces the problem. Also, the computational cost was reduced by more than a factor of two. Other improvements have been proposed in [START_REF] Puckette | Phase-locked Vocoder[END_REF]

Parametric Techniques

Parametric techniques typically refer to methods which rely on a model of speech production, whose parameters are to be estimated. Two are the main representatives of such techniques: (1) a model with several in-series systems that represent the different stages of the human speech production system, i.e. excitation system, vocal tract system, and lips system, and (2) a model of time-series representation, i.e. a sum of frequency and/or amplitude modulated cosines.

Speech production based models have their origin in the work of Fant [START_REF] Fant | Indirect studies of glottal cycles by synchronous inverse filtering and photoelectric glottography[END_REF]. A simplification of the human speech production system is the following: (1) an excitation, which consists of a series of pulses, quasi-periodically placed in time, and represent the pressure signal that comes from the lungs, passes through the vocal cords. Models of the excitation, called glottal models, can be found in [START_REF] Alku | Glottal inverse filtering analysis of human voice production : a review of estimation and parameterization methods of the glottal excitation and their applications[END_REF]. (2) Subsequently, this excitation signal is modulated by the vocal tract, which boosts or suppresses certain frequencies according to vocal tract resonances, the so-called vocal tract formants. The vocal tract is shown to be effectively represented by an auto-regressive -AR process. (3) Finally, the signal passes through the lips, which can be represented by a first order high-pass filter, and thus enforces high frequency components. The output of this system is the final speech signal. Although this model is simplified and is considered as a linear system -which is not the exact case for speech production -it can represent speech phenomena quite well. Parameters of such systems are typically the pitch period (distance between pulses of the excitation), the formant frequencies and bandwidths, and the order of the AR process, although other parameters may be added, according to a specific model. Modifications based on such systems include the variation of these parameters; pitch modification, formant and bandwidth increase or reduction, glottal parameter manipulations, etc. On the other hand, time-series based parametric representations include the decomposition of speech into components: a deterministic part, which is usually modelled as a sum of frequency and/or amplitude modulated components, and a stochastic part, which is modelled as frequency-modulated Gaussian noise, usually weighted by a time-domain envelope. However, noise can be modelled by a sum of cosines as well, as in [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF]. Typically, the deterministic part represents voiced speech, whereas the stochastic parts represents unvoiced speech, friction noise, etc. More specifically, if the frequencies of the deterministic part are harmonically related, then the general model is called the Harmonic model. Therefore, various combinations have been made in literature: Deterministic plus Stochastic model [Ser89] [Sty96], Harmonic plus Noise model [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], Sinusoidal plus Noise model [START_REF] Oomen | Sinusoids plus noise modelling for audio signals[END_REF] and Quasi-Harmonic plus Noise model [START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF]. Moreover, because of the inability of such models to represent highly non-stationary parts of speech, such as stop consonants or transient speech areas, extended models have been suggested, generally called Sinusoidal plus Noise plus Transients models [Lev99] [START_REF] Thornburg | Detection and Modeling of Transient Audio Signals with Prior Information[END_REF].

Typical parameters of these models include the (harmonic or not) frequencies, amplitudes, and phases of the deterministic part, the number of sinusoids, whether an analysis frame is voiced or unvoiced, the time envelope of the noise, etc. Modifications using such models include parameter trajectory scaling. Interpolation schemes such as linear, cubic, or spline, are used to estimate the parameter trajectories in between analysis frames. However, attention should be paid both in phase coherence and in shape invariance of the resulting modified waveform.

The first approach was the Linear Predictive Vocoder [START_REF] Atal | Speech analysis and synthesis by linear prediction of the speech wave[END_REF], in which voiced speech is modelled as a convolution between a periodic train of pulses and a time-varying AR filter. Although quite popular at the beginning, it was soon abandoned due to its failure to provide high-quality modified speech. Later, Almeida and Marques [START_REF] Almeida | Variable-frequency synthesis: an improved harmonic coding scheme[END_REF] [START_REF] Marques | Frequency-varying sinusoidal modeling of speech[END_REF] were the first to propose the use of sinusoids for speech analysis and synthesis. In the late 80ies, Griffin [START_REF] Griffin | Multiband Excitation Vocoder[END_REF] and Serra and Smith [START_REF] Serra | A System for Sound Analysis, Transformation, Synthsis based on a Determistic plus Stochastic Decomposition[END_REF], had proposed a model where the sinusoids were harmonically related. However, milestones in sinusoidal modelling were the work of McAulay and Quatieri, the so-called Sinusoidal Model, which was followed by several variants, such as [START_REF] George | Analysis-by-Synthesis Overlap-Add Sinusoidal Modeling Applied to the Synthesis of Musical Tones[END_REF], and that of Stylianou [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF] on harmonic modelling, referred to as the Harmonic Plus Noise Model. The Sinusoidal Model is described next and the Harmonic Plus Noise Model follows. Finally, a recent source-filter model referred to as the LF-ARX model [START_REF] Vincent | A new method for speech synthesis and transformation based on an ARX-LF source-filter decomposition and HNM modeling[END_REF] will be briefly discussed.

The Sinusoidal Model (SM)

In 1986, McAulay and Quatieri suggested their famous Sinusoidal Model (SM). In In this work, the speech waveform s(t) is assumed to be the output of passing a vocal excitation waveform e(t) through a linear system h(t) representing the characteristics of the vocal tract. The system h(t) is assumed to account for both the shape of the glottal pulse and the vocal tract impulse response.

Analysis-Synthesis

The commonly used binary unvoiced/voiced excitation model is replaced by a sum of sine waves of the form

e(t) = N � k=1 a k (t) cos(Ω k (t)) (3.33)
where N is the number of sinusoids, a k (t) is the time-varying amplitude associated with each sinewave, and the excitation phase Ω k (t) is the integral of the time-varying frequency ω k (t) where φ k is the fixed phase offset because the sinewaves are not generally in phase. The vocal tract transfer function H(ω; t) can be written as H(ω; t) = M (ω; t)e jψ(ω;t) (3.35)

Ω k (t) = � t 0 ω k (σ)dσ + φ k (3.

Analysis part

Synthesis part

) (t s ) ( ˆt s ) ( i k t A ) ( i k t f ) ( i k t  ) ( i k t  ) (t A k ) (t k  Homomorphic Analysis & System- Excitation Decomposition Modification factors ρ β Modification part ) ( ) (   t a t M k k ) ( ) ( t t k k  
The dependence on t is due to the fact that the vocal tract impulse response is time-varying. The system amplitude and phase along each frequency trajectory ω k (t) are given by

M k (t) = M [ω k (t); t] (3.36) and ψ k (t) = ψ[ω k (t); t] (3.37)
So, when the excitation signal e(t) passes through the linear time-varying vocal tract h(t), the output is the sinusoidal representation of the speech signal

s(t) = N � k=1 A k (t) cos[θ k (t)] (3.38) where A k (t) = a k (t)M k (t) (3.39) and θ k (t) = Ω k (t) + ψ k (t) + φ k (3.40)
represent the amplitude and phase of each sinewave along the frequency trajectory ω k (t). The validity of this representation is subject to the stationarity assumption of the excitation amplitudes and frequencies, compared to the vocal tract impulse response.

The analysis consists of two steps. First, the estimation of frequencies, composite amplitudes and phases is performed using a high-resolution Fourier Transform magnitude. This is done in a frame-by-frame scheme, after applying a window on the speech frame. The second step accounts for the separation of the system and excitation components.

For the first step, let S(ω, kR) be the short-time Fourier Transform of the speech signal, and let R be the frame rate, so the estimated values are taken at kR sample indices. Hence, if ωk l is the l th frequency estimate of the k th analysis frame, then the corresponding amplitudes and phases are given as

Âk l = |S(ω k l , kR)| (3.41) and θk l = arg[S(ω k l , kR)] (3.42)
where arg denotes the principal value.

For the second step, the separation of system and excitation parameters is done using homomorphic deconvolution, under the assumption of the vocal tract transfer function being minimum phase. Thus, the excitation parameters at each analysis frame boundary are obtained as

âk l = Âk l / M k l (3.43) and Ωk l = θk l -ψk l (3.44)
The synthesis is performed in three steps. First, there is a matching procedure between parameter values computed at two consecutive frame boundaries. Then, an appropriate interpolation scheme is applied on the resulting pairs of amplitude and phase samples of excitation and vocal tract functions over each frame, and finally the sinewaves are generated using the interpolated components.

The first step is about matching the excitation frequencies measured on frame k with those measured on frame k + 1. After matching these frequencies, the matching of all other parameters comes easily, since they are measured at the excitation frequencies. An algorithm for matching the location of the spectral peaks was proposed in [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF], which uses a purely sinewave based model. One basic concept of this algorithm is the "birth-death" of a sinewave frequency. A brief example of how this algorithm works is the following:

Let ω k l and ω k+1 l be the excitation frequency estimates of the l th sinewave over frame k and k + 1, respectively. In order to match these two frequencies (i.e. to assume that they belong to the same frequency trajectory), frequency ω k+1 l should lie in the interval [ω k l -∆, ω k l + ∆]. If it is not, then the frequency track associated with ω k l is considered "dead" in frame k + 1, its amplitude is zeroed in this frame, and frequency ω k l is not considered any more. If, however, frequency ω k+1 l lies in this "matching interval", and is the closest one to ω k l (since there might be more than one that lie in that interval -called candidates for matching), then it is declared as a definite match. After all matching is done, there might be some unmatched frequencies in frame k + 1. If so, then new frequencies are "born" in frame k, with zero amplitude.

After parameter matching, the second step is about parameter interpolation. This is based on the assumption that the excitation and system functions are slowly varying across each frame along frequency tracks ω l (t). System amplitudes M i l and excitation amplitudes a i l can be linearly interpolated. The system phases ψ i l can also be linearly interpolated, but this is not the case for the excitation phases and frequencies. Thus, a cubic polynomial is fitted on the excitation phase [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF].

Finally, the synthetic waveform is given by

ŝ[n] = L[n] � l=1 Âl [n] cos( θl [n]) (3.45)
where

Âl [n] = âl [n] Ml [n] (3.46) and θl [n] = Ωl [n] + ψl [n] (3.47)
where L[n] is the number of sinewaves estimated at time n.

Pitch and Time Scale Modifications

Having this sinewave based model, prosody modifications are straightforward.

For time-scale transformations, the parameters that are scaled are the system amplitudes and phases, M (ω, t), and ψ(ω, t), and the excitation amplitudes and frequencies, a l (t) and ω l (t). The modification of these parameters correspond to moving slower or faster the vocal tract articulators and to stretching or compressing the frequency trajectories, respectively. Specifically, for an arbitrary time scale transformation, the time t 0 of the original articulation rate is mapped to the transformed time t � 0 through the mapping

t � 0 = W (t 0 ) (3.48)
where W (t) is the so-called time warping function. Fixed rate time-scaling will be briefly discussed here, since it is more convenient for understanding and the extension to time-varying rate change is straightforward. For fixed rate change ρ, the time warping function becomes t � = ρt ⇒ t = ρ -1 t � . The mathematical model for time-scaled speech s � (t � ) is given below:

s � (t � ) = L(t � ) � l=1 A � l (t) cos(θ � l (t � )) (3.49)
where

A � l (t � ) = A l (ρ -1 t � ) = a l (ρ -1 t � )M l (ρ -1 t � ) (3.50) and θ � l (t � ) = Ω � l (t � ) + ψ l (ρ -1 t � ) (3.51)
where

Ω � l (t � ) = � t � t � l ω l (ρ -1 τ )dτ + φ l (3.52)
The initial phase offset φ l is chosen to preserve the impulselike nature of the excitation function during voicing, but in practice the onset timing may be different among sinewave components. This leads to the phase dispersion problem, which was handled in [START_REF] Quatieri | Shape-Invariant Time-Scale and Pitch Modifications of Speech[END_REF], by onset timing detection.

For pitch modification, the excitation frequencies must be shifted to new frequencies, according to a pitch scale modification factor β. Thus, frequency track ω l (t) in the original signal corresponds to frequency track βω l (t) However, the model does not account for changes in the vocal tract spectral characteristics. For that, the system amplitudes, M (ω, t), and phases, ψ(ω, t), must be computed at the new frequency track locations βω(t). Hence, the mathematical model for pitch-scaled speech s � (t) is the following:

s � (t) = L(t) � k=1 âl (t) M � l (t) cos( Ω� l (t) + ψ� l (t)) (3.53) where M � l (t) = Ml (β ωl , t), ψ� l (t) = ψl (β ωl , t), Ω� l (t) = β � t t l ω l (τ )dτ + φ l (3.54)
Fig. 3.7 shows schematically the analysis, modification, and synthesis procedures.

The Harmonic Plus Noise Model (HNM)

During the early and mid-90ies, Stylianou proposed a new model, called the Harmonic plus Noise model (HNM) [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. In this model, speech spectrum is separated into two parts: a deterministic and a stochastic part, delimited by a timevarying maximum voiced frequency. In the lower band (deterministic part), the signal is considered to be harmonic. The stochastic part, which is the residual of the original signal minus the deterministic part, is modelled by an AR model and its time domain behaviour is imposed by a parametric time domain envelope. In this model, both the analysis and synthesis is performed in a pitch-synchronous manner, inspired by PSOLA. Thus, it can provide flexible techniques for time and pitch scaling, which will be discussed shortly.

The harmonic part accounts for the quasi-periodic phenomena of speech, while the noise part models non-periodic components, which typically include friction noise, unvoiced speech, etc. The time-varying maximum voiced frequency is used to determine the limit between the two parts.

In the lower band, the signal can be modelled as a sum of harmonically related sinusoids with slowly varying amplitudes and frequencies:

h(t) = K(t) � k=1 A k (t) cos(kθ(t) + φ k (t)) (3.55)
where

θ(t) = � t -∞ ω 0 (u)du (3.56)
and where A k (t), φ k (t) denote the amplitude and phase at time t of the k th harmonic respectively, ω 0 (t) is the fundamental frequency and K(t) is the number of harmonics included in the harmonic part. The upper band contains the noise part. In voiced speech, the noise part exhibits a specific time domain structure in terms of energy distribution, i.e. it is concentrated in the part of the pitch period where the glottis is open. Thus, the frequency components of the noise part is described by a time-varying AR model, and its time domain structure is formed by modulation using a parametric envelope.

Analysis-Synthesis

Before applying the model on speech, an estimation of the fundamental frequency and the maximum voiced frequency is required. Hence, a pitch estimator similar to the one used in [START_REF] Griffin | Multiband Excitation Vocoder[END_REF] is used. Then, a voicing decision is made, and finally a refined pitch is defined as the fundamental frequency whose harmonics better fit the voiced frequencies detected in the lower band. Using this stream of pitch values, the position and duration of the analysis frames are set at a pitchsynchronous rare on the voiced portions of speech and at a fixed rate on unvoiced parts.

On the voiced frames, an estimate of the parameters are obtained at the center t i of the analysis window. Thus, the model can be rewritten as

h(t) = K � k=-K a k (t i )e jkω0t = K � k=1 A k (t i ) cos(kω 0 t + φ k (t i )), t i -N ≤ t ≤ t i + N (3.57)
where 2N + 1 represents the length of the analysis frame in samples,

K = F M (t i ) ω 0 (t i )
is the number of harmonics included in the harmonic part, and F M (t i ) denotes the maximum voiced frequency. The estimation of the parameters is performed using weighted least squares, that is

� = N � t=-N w(t)(x(t) -h(t)) 2 (3.58)
where x(t) is the original signal. This is a different approach than the previously discussed Sinusoidal Model (SM), which performs peak peaking over the speech spectrum. Since the parameter estimation is entirely done in the time domain, shorter windows can be used. It is reminded that in SM (and other approaches that use FFT methods), a typical analysis window has a length of three to four pitch periods, while in HNM two pitch periods are used. This is an important property of HNM, since it is convenient for modelling segments where speech exhibits high pitch or amplitude non-stationarity.

For the noise part, n(t), the estimation of the parameters is as follows. In each analysis frame, the power density function of the original signal is modelled by a p th -order all-pole filter (p = 15 for a 16 kHz signal), and the variance of the signal is calculated. Then, a parametric envelope is estimated in each frame. A triangular type time-domain envelope has proved to provide satisfactory results. However, in [START_REF] Pantazis | Improving the Modeling of the Noise Part in the Harmonic plus Noise Model of Speech[END_REF], it was shown that an energy based time domain envelope outperforms the triangle type approach.

Synthesis

The synthesis is performed in a pitch-synchronous way. In a plain synthesis (without modifications) scheme, the analysis time instants, t a i , coincide with the synthesis time instants, t s i . For the harmonic part, the amplitudes and phases are estimated via LS, as previously mentioned, and are linearly interpolated between successive frames. Please note that the phases are unwrapped before applying interpolation. This is done by a predicting the phase of the current frame, using the phase of the previous one and the average instantaneous frequency. The noise part is synthesized using an Overlap-Add (OLA) procedure, in order to avoid discontinuities at the frame boundaries. Given a synthesis time instant, t i s , two pitch periods are synthesized by filtering a unit variance, white Gaussian noise through a normalized lattice filter, and multiplying the output by the variance estimated at the analysis time instant, t a i . If the frame is voiced, then the lower part is synthesized using harmonics, up to the estimated maximum voiced frequency, F M (t i a ). Thus, the noise part is filtered by a high-pass filter with a cut-off frequency f c = F M (t i a ). Then, the synthetic noise part is obtained by applying OLA on two noise parts, one synthesized at synthesis time instant t i s , and the other synthesized at t i-1 s . Finally, for voiced frames, the triangular time domain envelope is applied directly on the synthetic noise part. The

Analysis part

Modifications part

Synthesis part final synthetic speech signal is obtained by adding the two parts,

) ( s k t  ) ( ), ( t t A k k  ) (t n
s(t) = h(t) + n(t) (3.59)
It is worth mentioning that since a harmonic assumption is held in this model, there is no need for frame-to-frame frequency matching, as in the Sinusoidal Model.

Pitch and Time Scale Modifications

For the sake of convenience, the case of joint time-scale and pitch-scale modifications is discussed here. As a first step, the new time synthesis instants should be found, according to the analysis time instants and the desired pitch and time scale modifications. HNM follows a pitch-synchronous scheme, inspired by PSOLA, which was briefly discussed in the previous section. A mapping between the synthesis and analysis instants is determined, specifying which analysis instant should be selected for a given synthesis one. This is performed according to the following constraints: for pitch scaling, the time evolution must be preserved, and for time scaling, the pitch contour must be preserved.

In case of pitch scaling, one should compute the amplitude and phases for the modified harmonics. For this, a spectral and phase envelope estimation is necessary. In HNM context, a regularized cepstrum technique is used [START_REF] O. Cappe | Regularized estimation of cepstrum envelope from discrete frequency points[END_REF], where discrete cepstral coefficients are calculated, with a frequency domain LS criterion, combined with a regularization method to increase robustness of the estimation. The amplitudes are then obtained by sampling the estimated envelope at the new harmonic frequencies. For the phase envelope, another approach is followed. Consider a voiced frame of a voiced portion of speech. The phase is unwrapped in the frequency domain by adding integer multiples of 2π, in order to keep the frequency slope variation, dφ k = φ k+1 -φ k , as smooth as possible, where k is the k th harmonic. In the next voiced frame, the phases are unwrapped by using the frequency slopes from the previous frame and not the frequency slopes of the current one. This way, phase continuation is guaranteed both in time and in frequency domain. Finally, the new phases are obtained by sampling the phase envelope at the modified pitch harmonics.

Having the new amplitudes and phases, the synthesis of modified speech is performed in the same manner as in synthesis without modifications. It is observed that modified speech is free of artefacts like "buzziness" or "metallic" notion, like in SM or other approaches. An overall flow diagram for HNM is depicted in Figure 3.8

The LF+ARX model

A common approach in speech processing is to represent the speech production system as a source-filter model. In such representations, the source is referred to as the glottal flow derivative (GFD) (as the convolution of the glottal flow and the lip radiation). A well-known model for GFD representation is the Liljencrants-Fant model (LF model) [START_REF] Fant | A four-parameter model of glottal flow[END_REF], which characterizes the GFD using a number of parameters -usually five: one for the location of the glottal source, one for the amplitude, and three for the shape of the glottal flow. parameters that define the shape of the waveform include the Open Quotient (O q = Tc T0 ), the Asymmetry Coefficient (a m = Tz Te ), and the Return Phase Quotient (Q a = Ta (1-Oq)T0 ). So, a period of the LF model can be mathematically described as

u LF (t) = � E 1 e at sin(ωt) 0 ≤ t ≤ T e -E 2 [e -b(t-Te) -e -b(T0-Te) ] T e ≤ t ≤ T 0 (3.60)
where the parameters a, b, ω are related to the previously mentioned coefficients that define the shape of the GFD. Hence, the speech signal can be represented by means of an Auto-regressive model with an exogenous input (ARX model) [START_REF] Ding | Simultaneous estimation of vocal tract and voice source parameters based on an ARX model[END_REF]:

s[n] = - p � k=1 a k [n]s[n -k] + b 0 u LF [n] + e[n]
(3.61) where a k [n] are the time-varying coefficients of the order p AR model that describes the vocal tract, and e[n] is the residual signal that carries information that remains uncaptured by the ARX-LF model.

Analysis-Synthesis

The analysis procedure consists of two steps: One, the estimation of the ARX-LF model parameters, and two, the decomposition of the residual signal. The residual signal has been proposed to decompose in three different ways [AR09]: a HNM-based decomposition, a modulated noise model, and a harmonic model.

At first, the parameters of the ARX-LF model are estimated in the following way: initially, an estimate for the f 0 is provided. Second, an estimation of the glottal closure instant (GCI) is performed [START_REF] Vincent | A new method for speech synthesis and transformation based on an ARX-LF source-filter decomposition and HNM modeling[END_REF]. Third, a Viterbi algorithm is applied to regularize the LF source. Finally, the AR parameters are estimated. After this procedure, the residual can be modelled using any of the aforementioned ways. This analysis is for voiced parts -for unvoiced parts, the above analysis scheme is replaced by a simple Warped Linear Predictive (WLP) analysis.

The synthesis is done pitch-synchronously by passing the reconstructed glottal source through the time-varying AR filter. The reconstructed glottal source is given as

u[n] = w[n]u l [n] + (1 -w[n])u l+1 u[n] (3.62)
where w[n] is a Hanning window whose length is twice the local period, and u l [n] and u l+1 [n] denote the short term glottal signals obtained from the l th and the (l + 1) th analysis instants, respectively.

Pitch and Time Scale Modifications

When modifications are applied, then two intermediate steps are involved: i) the determination of the sequence of the analysis frame indices given a stream of f 0 and time modification coefficients, and ii) the modification and synthesis of the selected speech frames. The first step is very similar to the PSOLA analysis scheme. For time scaling, a typical duplication or elimination of frames is performed, so the focus will be on pitch scaling. So, the LF waveform and the residual signal need to be modified, since AR parameters that describe the vocal tract remain untouched.

The modified LF waveform is related to the original one by

ûLF � t T0 � = u LF � t T 0 � (3.63)
which states that the spectrum of the modified glottal source is a stretched version of the original one with a duration of T0 .

Another way to modify the pitch is to change the shape parameters of the LF model:

Te = T e ⇔ Ôq = O q T 0 T0 (3.64) Ta = T a ⇔ Qa = Q a 1 -O q 1 -Ôq T 0 T0 (3.65)
where Ôq , Qa represent the modified open and return phase quotients, respectively. It should be noted that the shape of the spectral envelope of the LF waveform is preserved in this way.

Other Approaches

Parametric modelling has many other representatives and in this paragraph, some of them will be briefly discussed. In parallel to the work of McAulay and Quatieri on the Sinusoidal Model, Serra [Ser89] suggested a hybrid system for the analysis, transformation, and synthesis of sound based on a deterministic plus stochastic decomposition. This system is designed to obtain musically useful intermediate representations for sound transformations. The method is similar to the HNM described earlier in terms of decomposition, but the deterministic component is represented by a series of not-necessarily-harmonic sinusoids calculated by a STFT-based peak-picking method, as in SM. The stochastic component is represented by a series of magnitude-spectrum envelopes that work as a time-varying filter excited by white noise, similarly to HNM. This approach is able to create new sounds out of the representation of a particular sound. The deterministic signal is obtained by synthesizing a sinusoid from each trajectory. Then, the residual between the deterministic component and the original sound is modelled by a series of envelopes. Finally, the stochastic signal is generated by an inverse STFT. This system is very flexible and allows for transformations of the sound by manipulating each component separately.

George and Smith [START_REF] George | Speech analysis/synthesis and modification using an analysis-bysynthesis/overlap-add sinusoidal model[END_REF] proposed a novel speech analysis/synthesis system based on the combination of an overlapadd (OLA) sinusoidal model with an analysis-by-synthesis technique to determine the model parameters. An equivalent frequency-domain algorithm that takes advantage of the computational efficiency of the FFT is introduced and a refined OLA sinusoidal model is presented, which can offer shape-invariant speech modifications. The quality of the system is very high when the underlying frequencies are accurately estimated. Shape invariance and phase coherence are explicitly controlled in the modification process.

The Exponentially Damped Sinusoidal Model (EDSM) has also been proposed [NHD98a, JJH99, JHJ04] for modeling transient parts of speech or audio, along with more powerful parameter estimation schemes based on either Matching Pursuit or Subspace Methods. Subspace methods have good spectral properties and do not suffer from the time-frequency trade-off embedded in other methods. However, they are computationally intensive.

Furthermore, Degottex [START_REF] Degottex | Glottal source and vocal-tract separation[END_REF] supported that a model which is more dedicated to voice production better respects physiological or acoustic constraints. Towards this direction, the Separation of the Vocal tract with the Liljencrants -Fant (LF) model plus Noise (SVLN) is suggested. The Transformed LF (TLF) glottal model is used to represent the deterministic component of the source. Instead of using the standard, tri-parametric version of LF [START_REF] Fant | Notes on Glottal Flow Interaction[END_REF], a single parameter, named R d , is used to control the shape of the source. The reason for this is that, according to Fant [START_REF] Fant | The LF-model revisited. Transformations and frequency domain analysis[END_REF], the R d is the most effective parameter to describe voice qualities into a single value. Zero-mean Gaussian noise is used for representing the random component of the source. Amplitude modulation is applied on this noise during synthesis, to improve naturalness. Due to the different spectral properties of the deterministic and random components, the estimation of the Vocal Tract Filter (VTF) is adapted by taking into account this mixed source model. Transformations such as breathiness, time-scaling, and pitch transposition are available on this model, with very good results compared to the state of the art. However, as in almost all glottal-based models, there is a stability problem when estimating glottal parameters in frames that are in-between voiced and unvoiced, i.e. transient frames.

Finally, refinements of the proposed models using more powerful parameter estimation schemes have been presented. The Fan-Chirp Transform (FChT) [KW06, WK07] is a recently introduced transform that employs an adaptive analysis basis composed of quadratic chirps. A sinusoidal analysis of speech similar to [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] but with the FChT instead of the FFT has been conducted [DQ07, DQM09] with very satisfactory results. However, speech modifications are yet to be developed.

Conclusions and Discussion

The methods discussed so far yield high quality for speech resynthesis and moderate modifications. Parametric methods, such as SM or HNM, work well for well-estimated frequencies and under the assumption that speech is shortterm stationary. That is, sinusoids that represent voiced speech have constant amplitudes and constant frequencies for a short time analysis window, typically 20 -30 ms. It is already shown in [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] that this is not the case in speech, where there are rapid, non linear amplitude and frequency changes during short time intervals. It is essential for highquality speech analysis, synthesis, and modifications, to be able to capture these short-time fluctuations. Furthermore, parametric models usually represent speech in a two-fold process: they estimate the model parameters on a "deterministic part" of speech, and the "randomness" is then modelled differently. Separation of components has been proved practical and convenient for processing, synthesis, and manipulation of speech, under the assumption that the components are well-separated and accurately estimated. Finally, voice production-based models, such as ARX-LF and SVLN, although providing high-quality output, they are complicated and very sensitive in parameter estimation, especially in unvoiced or transient parts. Hence, it would be desirable to have a parametric speech model that is relatively simple, flexible, high-quality, and robust in resynthesis and modifications.

Towards this direction, hybrid systems of speech analysis based on eaQHM (extended adaptive Quasi-Harmonic + Noise Model -eaQHNM) and aHM (adaptive Harmonic + Noise Model -aHNM) will be presented in this thesis, with very satisfactory results in terms of perceptual quality. However, it will be shown that certain assumptions in component separation of hybrid systems are not necessary, and all speech sounds can be very accurately represented as AM-FM components only. Thus, speech can be uniformly represented very accurately as a sum of AM-FM sinusoids, providing compactness, uniformity, and simplicity of speech representation. Modifications will be applied on this representation of speech as well, and their performance will be discussed and compared with the corresponding hybrid systems, as well as with other state-of-the-art systems.

Regarding transformations, the models described have their limitations. Modifications based on non parametric frequency-domain methods introduce an undesirable reverberation effect, known as "chorusing", as well as other effects, such as transient smearing (loss of percussiveness) and phasiness (coloration of signal). Also, a number of them are computationally intensive. On the other hand, non parametric time-domain methods, while free of reverberation or chorusing artefacts and computationally efficient, they rely heavily on the quasi-periodic assumption of speech. Furthermore, parametric modelling is highly dependent on the performance of the analysis and synthesis algorithm. Hence, it is believed that since adaptivity provides a high quality analysis/synthesis scheme, prosodic modifications would be of superior quality compared to standard sinusoidal modeling techniques. However, certain aspects in speech modifications should be taken care of. An important aspect is the spectral and phase envelope estimation during pitch scaling. An accurate estimation of these envelopes is necessary in order to evaluate the magnitude and phase values of the shifted frequencies. While there is a variety of approaches for the magnitude envelope estimation, the phase envelope estimation is a difficult problem. In this thesis, phase modifications for time and pitch scaling is handled via very simple mathematical properties. Moreover, shape invariance is a property of analysis/synthesis systems that plays an important role. Shape invariance refers to the ability of a system to preserve the temporal structure of the speech waveform. The inability to maintain the shape of the waveform is caused by the so-called phase dispersion problem, which is due to that the reconstructed signal has the same frequency information as the original signal but the relationship of the phases between the different sinusoids has changed. This effect is audible and can be described as "chorusing". In this work, phase dispersion effects will be minimized using a very simple method that utilizes phase properties, and specifically, the notions of relative phase for the harmonic models and the relative phase delays for the quasi-harmonic models.

Chapter 4 Speech Analysis and Synthesis based on Adaptive Sinusoidal Models

In a previous chapter, the members of Adaptive Sinusoidal Models family have been analytically described. However, their application on running speech is not straightforward. In literature, there are two different approaches in speech analysis systems. The first one is based on hybrid systems, that is, analysis and synthesis systems that decompose speech into more then one components, usually a deterministic and a stochastic one. The second one is based on fullband Systems, that is, analysis and synthesis systems that treat all parts of speech the same way, as a sum of AM-FM components. In this chapter, speech analysis and synthesis systems based on the aSMs will be presented, and among them, the newly suggested eaQHM will be discussed and compared to the other aSMs and the state-of-the-art. Discussion and motivation on both hybrid and full-band approaches will be presented in this chapter as well.

Hybrid Systems

Hybrid systems are considered well suited for resynthesis and prosodic modifications, since a well-mastered separation of speech into a deterministic and a stochastic component leads to a better manipulation of them and that aids to an enhanced quality of speech synthesis and modifications. A typical flowchart of a hybrid system is shown in Figure 4.1. Let us briefly discuss the elements of a general hybrid system. First of all, in the analysis part, the pre-processing stage often includes actions such as pitch estimation, voiced/unvoiced decision, maximum voiced frequency estimation, filtering, enhancement, or noise cancellation. The deterministic analysis part is responsible of modelling the deterministic characteristics of speech, whereas the stochastic analysis models the random component of speech, such as friction noise, unvoiced speech, etc. Except for the usual deterministic and stochastic component, recent speech models often include a transient part [START_REF] Agiomyrgiannakis | ARX-LF-based source-filter methods for voice modification and transformation[END_REF][START_REF] Levine | Audio Representations for Data Compression and Compressed Domain Processing[END_REF], which captures (but not necessarily model) the transient parts of speech (vowelto-consonant frames, and vice versa, as well as stop sounds) and is handled differently than the other two components. However, the identification of a transient frame is not an easy task, and the most convenient choice is to account them in either the deterministic or the stochastic part.

When the analysis parameters for all speech components are estimated, they are passed to the synthesis step, where a pre-processing of the parameters is performed, as for example parameter interpolation or spectral envelope estimation, in case of speech modifications. Finally, each component is synthesized separately and all components are summed up to form the synthesized speech signal.

Typical examples of such systems include the Harmonic + Noise Model (HNM) [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], the STRAIGHT method [START_REF] Kawahara | Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited[END_REF], and the LF+ARX model [START_REF] Agiomyrgiannakis | ARX-LF-based source-filter methods for voice modification and transformation[END_REF]. In the following sections, a hybrid approach will be described based on a twocomponent paradigm: a deterministic and a stochastic component. The choice of such an approach is justified by its successful application in earlier models, and the convenience in manipulation for modification purposes. After that, drawbacks and misconceptions on hybrid sinusoidal-based systems will be discussed, and simple, full-band schemes of speech analysis, synthesis, and modifications will be proposed, where the term full-band refers to a uniform AM-FM decomposition of all parts of speech.

Pre-processing in hybrid systems

As discussed, hybrid systems consist of two components: a deterministic and a stochastic one. Thus, a preprocessing step is necessary to help the separation of components and the estimation of some crucial parameters. In most hybrid

) (t s ) ( ˆt d ) ( ) ( ) ( t d t s t r   ) ( ) ( ) ( t r t r t n  

Analysis Step

Synthesis Step

Pre-processing Deterministic Synthesis

Stochastic Synthesis

Transient Synthesis sinusoidal models in the literature, this preprocessing step allows the separation of speech in voiced/unvoiced/silence parts, in parallel to a f 0 estimation for voiced parts. We will now briefly describe these processes which apply to both hybrid systems presented in this chapter.

) ( ˆt s ) ( ˆt d ) ( ˆt r ) ( ˆt n + Variables Speech signal Deterministic part Stochastic part Transient part denotes estimate : ) ( : ) ( : ) ( : ) ( t n t r t d t s ) ( ˆt x

Voiced/Unvoiced/Silence Discrimination

Since the system in hand is a hybrid system, a voiced/unvoiced/silence (V/UV/S) discrimination algorithm needs to be run as a pre-processing step. Although any V/UV/S algorithm can be chosen, in our experiments the approach described in [START_REF] Pantazis | Adaptive AM-•FM Signal Decomposition With Application to Speech Analysis[END_REF] is selected. The V/UV detection is performed in a frame-by-frame procedure, with frame size of 30 ms and step size of 5 ms. The energy of each frame is computed and if it is above a threshold B E , then it is assigned as speech. Otherwise, a silence flag is assigned on this frame. For the separation of voiced and unvoiced speech, once a frame has been flagged as speech, two conditions should hold to declare it as voiced:

1. the energy of the speech frame minus the energy of a low-pass-filtered version of it should be below B d , and 2. the energy of the low-pass-filtered frame should be above B s As a final step, a median filter of order 5 is applied on the V/UV/S estimation in order to eliminate outliers.

Fundamental frequency estimation

For the f 0 estimation, any robust technique can be used, although the estimation is not critical for our systems due to the adaptation mechanisms. In the systems in hand, the recently proposed SWIPE pitch estimator is used. SWIPE estimates the pitch as the fundamental frequency of a sawtooth waveform whose spectrum best matches the spectrum of the input signal. The spectra are compared by computing a normalized inner product between the signal spectrum and a modified cosine. For details, please see [START_REF] Camacho | A sawtooth waveform inspired pitch estimator for speech and music[END_REF].

The eaQHNM analysis and synthesis system

The speech signal is decomposed into two parts, i.e.

s(t) = d(t) + n(t) (4.1)
where d(t) denotes the deterministic part and n(t) denotes the stochastic part. The separation of components is performed using the schemes presented in [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], i.e. an initial analysis is performed in order to discriminate speech into voiced and unvoiced regions, and then the pitch for the voiced regions is determined, as explained earlier. The following subsections present the deterministic and the stochastic part decompositions in detail.

Analysis of the Deterministic Part

Assuming a Hamming analysis window w(t) with support in [-T l , T l ], a frame of the deterministic part is initially modeled using QHM as:

d(t) = � L � k=-L (a k + tb k )e j2π fk t � w(t) (4.2)
where a k , b k are the complex amplitudes and the complex slopes of the k th component respectively, fk = k f0 are the analysis frequencies, f0 is an initial estimation of the fundamental frequency, and L is the number of quasi-harmonics, as specified by the maximum voiced frequency. The estimation of the model parameters is obtained via Least Squares, as described in [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. As it was mentioned in the previous section, QHM is able to correct, in the least square sense, frequency mismatches that are due to inaccurate estimation of the fundamental frequency. Let η k denote the frequency mismatch of the k th component:

η k = f k -fk (4.3)
where f k is the actual frequency and fk is an estimation of f k . It was shown in [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] that a projection of b k onto a k results in an estimate of the frequency mismatch, which is:

ηk = 1 2π �{a k }�{b k } -�{a k }�{b k } |a k | 2 (4.4)
where �{a k }, �{b k } and �{a k }, �{b k } are the real and imaginary parts of the complex amplitudes and complex slopes, respectively. Using Eq. (4.4), the analysis frequencies can be updated and the signal can be modelled again using QHM, but with a new set of analysis frequencies, fk = fk + ηk , and thus resulting in a more accurate signal representation. However, only accurate frequency estimation is addressed via QHM. The stationary model principle is still valid within an analysis frame. In order to confront this issue, the projection of the signal onto a set of time-varying basis functions is suggested in [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] and [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]:

d(t) = � L � k=-L � a k + tb k �� αk (t)e j φk (t) � � w(t) (4.5) with αk (t) = Âk (t + t i ) Â(t i ) (4.6) and φk (t) = φk (t i ) + 2π � t ti � fk (τ ) + c(τ ) � dτ, t ∈ [-T, T ] (4.7)
where Âk (t), fk (t), φk (t) are estimates of the instantaneous amplitudes, frequencies, and phases of the k th component, respectively, c(τ ) is the phase coherence term as explained in Section 2.3, Eq. (2.42), and t i is the center of the analysis window.

The adaptation is completed by using the frequency correction mechanism first introduced in [PRS08], and states that an estimate of the mismatch between the actual k th -frequency and the estimated one, termed

η k = f k -fk , is given by ηk = 1 2π �{a k }�{b k } -�{a k }�{b k } |a k | 2 (4.8)
Hence, at the first adaptation, for the analysis time instant t i , the instantaneous frequencies are fk

(t i ) = k f0 (t i ) + ηk (t i )
and the instantaneous phases become

φk (t) = φk (t i ) + 2π � t ti ( fk (τ ) + c(τ ))dτ (4.9)
Then, a Least Squares solution for the a k , b k using these refined frequencies (and phases) leads to a better estimate of the instantaneous amplitudes Âk (t) = |a k (t)| and the ηk terms. By iteratively adding the ηk term of the current adaptation on the k th -frequency track of the previous adaptation, the frequency tracks represent the underlying actual frequencies better. This adaptation mechanism stops according to a reconstruction criterion related to the Signal-to-Reconstruction-Error Ratio (SRER), redefined here for convenience:

SRER i-1 -SRER i SRER i-1 < � (4.10)
where SRER i is the Signal-to-Reconstruction-Error Ratio of the resynthesized signal in the i th adaptation, defined as

SRER = 20 log 10 σ x(t) σ x(t)-x(t) (4.11)
where σ x denotes the standard deviation of x(t), x(t) is the actual signal and x(t) is the reconstructed signal, and � is a threshold for convergence, typically set to 0.02. Finally, it is essential to describe the estimation of the time-varying parameters. The k th instantaneous amplitude track, Âk (t), is computed via linear interpolation of the successive estimates. Spline interpolation could be an alternative but it does not guarantee positiveness of the tracks. For that reason, linear interpolation is preferred. The k th instantaneous frequency track, f k (t), is computed via spline interpolation, because splines offer smooth transitions between frequency estimates. Also, it is worth noting that frequency matching is trivial since the analysis frequencies are integer multiples of a fundamental. As for the k th instantaneous phase track, φk (t), similar interpolation schemes are not suitable; thus, a non parametric approach is followed based on the integration of instantaneous frequency. In addition, phase coherence over frame boundaries is addressed via the addition of an extra term in order to guarantee phase continuation over frame boundaries as in Eq. (4.9). Finally, the deterministic part can be approximated by its time-varying components using:

d(t) = L � k=-L Âk (t)e j φk (t) .
(4.12)

Analysis of the Stochastic Part

The stochastic part, n(t), defined as the residual between the analyzed signal and the deterministic part, does not only account for the unvoiced parts of speech but also for friction noise in voiced parts. It is modelled as:

n(t) = e(t)(u G (t) * q(t)) (4.13)
where u G (t) denotes a Gaussian white noise component that is convolved by a time-varying auto-regressive (AR) filter with impulse response q(t), and e(t) denotes the time-domain envelope. Standard LPC analysis is used for the estimation of the AR filter, with an order of p = 18, for a sampling frequency of F s = 16 kHz. The time-domain envelope is a very important factor, since it is essential for efficient fusion between the deterministic and the stochastic part [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF].

In [START_REF] Pantazis | Improving the Modeling of the Noise Part in the Harmonic plus Noise Model of Speech[END_REF], it was shown that an energy-based time envelope is a good choice. The energy envelope is given by:

e(t) = T � u=-T |n(t + u)| (4.14)
where T equals to 1 ms. Since the energy envelope has a pitch-synchronous behaviour [START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF], it can be approximated within a frame using a sum of few sinusoids:

ê(t) = � M � k=-M A k e j2πf k t+φ k � w(t) (4.15)
where M is a small integer, typically 3 or 4, and f 0 is the fundamental frequency of the frame. Amplitude estimation is performed using peak picking on the short-time spectrum of the energy envelope signal. The latter is obtained using a 30 ms Hamming analysis window and a frame rate of 5 ms.

Synthesis

During synthesis, for the deterministic part, the k th instantaneous amplitude track, Âk (t), is computed via either linear or spline interpolation of the successive estimates from the last adaptation step. The k th instantaneous frequency track, f k (t), is also computed via spline interpolation. As for the k th instantaneous phase track, φk (t), the non parametric approach based on the integration of instantaneous frequency is followed, as it is shown in the adaptation steps of the analysis. Then, the deterministic part can be approximated as:

d(t) = L � k=-L Âk (t)e j φk (t) (4.16)
The stochastic part n(t), is obtained by a simple Overlap-Add (OLA) method, using the parameters from the analysis part. Finally, the synthetic part is given by ŝ(t) = d(t) + n(t) (4.17)

Examples

Two examples are shown in Figures 4.2, 4.3 for a male and female speaker, respectively.

The aHNM analysis and synthesis system

In this section, the analysis and synthesis scheme of aHNM is presented, along with an application in time-scaling of speech. While the original adaptive Harmonic Model has been developed by Degottex and Stylianou in 2013 [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], the aHNM has been developed during this thesis as a first step towards speech modifications based on adaptive models. Only the analysis and synthesis part will be described here.

Analysis

In the analysis part, the deterministic and the stochastic part are separated and modelled. In general, the former can be described as

s(t) = s d (t) + s s (t) (4.18)
where s(t) denotes the speech signal, and s d (t), s s (t) denote the deterministic and stochastic part, respectively. The deterministic part models the quasi-periodicities of voiced speech as a sum of time-varying harmonic components, thus

s d (t) = K � k=-K A k (t)e jkφ0(t) (4.19)
where

φ 0 (t) = � t 0 2πf 0 (u)du (4.20)
K is the number of components, and A k (t), kφ 0 (t) are the instantaneous amplitudes and the instantaneous phases of the k th component, respectively. Please note that the instantaneous phase of the k th component is an integer multiple of the instantaneous phase of the first harmonic, f 0 , and that the analysis in voiced speech is full-band.

Deterministic Part

In the analysis step for the deterministic part, a parametrization of the speech signal at each analysis time instant t i a is undertaken. At first, a sequence of the analysis time instants are created in the voiced parts of speech using the provided f 0 (t) track, such we have one analysis time instant per pitch period. Moreover, if the distance between t i a and t i+1 a is short enough, aHM can model the amplitude variations of the unvoiced signal (like in plosives). Thus, the upper limit of the size of the analysis window is 20 ms and the lower limit comes from the provided f 0 (t) track, and is therefore set to 50 Hz. Around each analysis time instant t i a , a Blackman window with a length of 3 local pitch periods is applied to the speech signal. The phase track φ 0 (t) is then computed by means of spline interpolation of f i 0 and using the integration formula in Eq.(4.20). that is called frequency mismatch, where f 0 is the actual fundamental frequency at a certain time instant and f0 is an estimate of the latter. Following the adaptive scheme presented in [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF], the amplitude a k (t) and fundamental frequency f 0 (t) values are obtained by a linear interpolation, respectively, of their values, a i k and f i 0 , at the analysis time instants, t i a . In order to have an estimate of these values, the adaptive Quasi-Harmonic Model -aQHM is used, that is given by the following equation: where φ 0 (t) is the same as in Eq.(4.20), a k and b k are the complex amplitude and the complex slope of the model, respectively, and K is again the number of the components. It has been shown in [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] that a k and b k , that are obtained via a Least Squares minimization, can be used to provide an estimate, η0 , for the frequency mismatch of Eq.(4.21). Thus, for the k th component in general, this can be computed as:

s(t) = K � k=-K (a k + tb k )e jkφ0(t)
ηk = 1 2π �{a k }�{b k } -�{a k }�{b k } |a k | 2 (4.23)
where �{a k }, �{b k } and �{a k }, �{b k } are the real and imaginary parts, respectively, of the complex amplitude and the complex slope of the model. Using this estimate, the fundamental frequency values f i 0 can be updated in an iterative manner. However, as it is shown in [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF], this term cannot be larger than the main lobe of the analysis window. In [START_REF] Degottex | A full-band adaptive harmonic representation of speech[END_REF], an iterative algorithm has been proposed to update the frequencies. Its main idea is discussed here. In a single analysis window, an arbitrary small number of harmonics K (e.g. 4) can be assumed. These harmonics are considered not to vary too much from their actual values, i.e. the mismatch η k is small. By computing the LS solution for Eq.(4.22), the correction term, η 0 , related to the fundamental frequency f 0 can be then estimated by the following equation:

η0 = 1 K K � k=1 ηk k (4.24)
This estimation can be furtherly used to update the number of harmonics, K. If η0 is small, this means that the current set of harmonics have converged very close enough to their actual values. Then, K can be further increased to add new harmonics in a new set of harmonics. If η0 is large, then the current set of harmonics have not converged to their actual values and further iterations are necessary to successively reduce η0 . The number of harmonics that are added in each iteration are given by the following equation:

K = � 1 2 N w | η0 | � (4.25)
where N w is given by N w = min{B w , f 0 }, where B w is the bandwidth of the main lobe of the analysis window. Using the LS solution of Eq.(4.22), the local parameters a i k , b i k are computed, along with the k th frequency mismatch, ηk , and the fundamental frequency correction, η0 . The number of harmonics, K i , is then updated using Eq.(4.25). As a last step, the process is repeated for all frames until the Nyquist frequency is reached for all frames. This approach is termed as the Adaptive Iterative Refinement -AIR and a pseudocode for it is given in [START_REF] Degottex | A full-band adaptive harmonic representation of speech[END_REF]. It should be noted that the estimated amplitude and phase values that are obtained at the analysis step correspond to the aQHM model and not aHM which is used for synthesis. Therefore, the aHM model is used in a last iteration step to ensure the consistency between the models used in the analysis and the synthesis.

Stochastic Part

The stochastic part is modeled exactly the same way as in eaQHNM (See Section 4.3.2).

Synthesis

In the synthesis step for the deterministic part, each harmonic is generated in separate, one after the other, without using any window. Each harmonic component is synthesized by its parameters, namely its amplitudes |a i k |, phases ∠a i k , and fundamental frequency f i 0 . First, the instantaneous amplitude, |a k (t)|, of the k th harmonic is simply obtained by linearly interpolating the estimated |a i k | on the analysis time instants t i a , on a logarithmic scale. The instantaneous phase ∠a i k cannot be interpolated directly across time to obtain a k (t) because of its rotation due to the time advance between analysis time instants. Therefore, it is proposed to remove this effect using the integral of f 0 (t) from the start of the signal, and obtain the relative phase -RP:

∠ã i k = ∠a i k -kφ 0 (t i a ) (4.26)
Thus, by assuming that the shape of the signal is changing smoothly, the phase values change also smoothly from one analysis time instant to the other. Then, the RP ∠ã i k can be interpolated to obtain its continuous counterpart, ∠ã k (t). Additionally, a spline or cubic interpolation is necessary such as its time derivative, the frequency, is still continuous. All along the iterative process, and since the harmonic numbers K i increase independently from one analysis time instant to the other, there are often missing components in the interpolations of amplitude and instantaneous phase. If this is the case, then the amplitude of the missing component is set to -300 dB and the corresponding phase ∠ã k (t) is set to zero. For the stochastic part, it is resynthesized using the OLA method. For each frame, white noise is passed through the AR filter to obtain the frequency modulation of the stochastic part. Then, the energy envelope is computed from Eq. (4.15) and its multiplication with the frequency-modulated noise provides the reconstructed stochastic frame.

Examples

Two examples are shown in Figures 4.4,4.5 for a male and female speaker, respectively.

An alternative for noise modeling

In [START_REF] Hanna | Adapting the Overlap-Add Method to the Synthesis of Noise[END_REF], it has been noted that overlap-add methods for unvoiced synthesis have certain drawbacks. Also, the modulated noise approach can model unvoiced speech well, but this is not the case for plosives. For this, another method is proposed in which unvoiced frames are synthesized using white noise as an input to a lattice implementation and a sample-by-sample interpolation of the reflection coefficients. Standard LPC techniques have been used to estimate the latter, with an AR filter of order 16, a Hanning window size of 20 ms length, and a step size of 10 ms. Then, one can use the time-envelope of the original unvoiced speech to modulate the output of the filter. Although many algorithms are available to compute the time-envelope of a signal, the one proposed in [START_REF] Agiomyrgiannakis | Towards Flexible Speech Coding for Speech Synthesis: an LF + Modulated Noise Vocoder[END_REF] is selected. This algorithm is the time-domain analogue of the True-Envelope estimator and can be described as follows:

Initialize: s(t) = |n(t)| 1. lowpass-filter(s(t), 500 Hz) 2. s(t) =max(s(t), |n(t)|)
3. Goto 1 and 2 for 50 iterations Finalize: lowpass-filter(s(t), 500 Hz) An example of application is given in Figure 4.6.

Discussion

In Sections 4.1-4.4, we presented a novel and a refined hybrid system for analysis and synthesis of speech based on the extended adaptive Quasi-Harmonic model and the adaptive Harmonic Model. The former decomposes voiced speech in AM-FM components that are quasi-harmonically related whereas the latter is inspired from the theory of adaptivity to accurately estimate an f 0 which is used to model voiced speech. The unvoiced parts of speech are represented by a stochastic component which is implemented as time and frequency modulated white Gaussian noise for both systems. Illustrative examples are given for each model that depict their performance in time and frequency domain.

Both models rely on a voiced/unvoiced (V/UV) estimator that separate the corresponding parts of speech. The importance of such an estimator is crucial for the performance of the systems. Although a very simple V/UV estimator is used in this work, no significant artefacts are present in the resynthesized speech waveforms. However, a binary decision on voicing in frames is often erroneous, since a frame can be a transient frame, that is, in certain cases it cannot be unequivocally categorized as either voiced or unvoiced. According to the decision of the estimator, the frame will be modeled by either the deterministic or the stochastic model. This may result in problems in the frame boundaries due to inappropriate fusion between different modeling of adjacent frames.

It could be suggested to drop the V/UV estimator, to both reduce the complexity of the overall system and to eliminate possible frame categorization errors that could influence system performance. Such a suggestion leads to full-band systems that perform AM-FM decompositions on the full-length of the waveform.

Full-band Systems

Although hybrid models have been proved to provide flexibility in manipulation, synthesis, and modifications of speech, in this section full band analysis and synthesis systems of speech are presented, using the adaptive sinusoidal models on the full length of a speech waveform. This means that the model describes both voiced and unvoiced parts of speech. A generalized full-band system is depicted in Figure 4.7.

Motivation

There are several reasons for suggesting such an approach: first of all, for voiced speech, a number of hybrid systems heavily rely on an accurate estimate of the so-called maximum voiced frequency -MVF, which divides the spectrum of voiced speech in a deterministic and a stochastic part. The efficient estimation of the MVF is critical for the performance of the system and its resulting modifications. Second, as it is described in [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], and supported by other researchers [DD97, DDH06] such a MVF is not necessary from a speech production point of view. In Figure 4.8, a series of glottal pulses is depicted on the upper panel, and its corresponding magnitude spectrum on the lower panel. As it can be observed, the spectrum of the glottal pulse does not abruptly stop at some frequency but continues to decrease up to the Nyquist frequency. Moreover, the stationarity assumption of speech models states that speech is relatively stationary in a 20 -30 ms interval. However, even in purely harmonic speech segments, the stationarity assumption holds for low-to-medium range frequencies but not for higher frequencies. The variation of higher frequencies is also higher, since any small fluctuation of the fundamental frequency is propagated to the higher harmonics proportionally to the harmonic number. Thus, the stationarity assumption does not hold for these frequencies and the use of stationary basis functions, as in DFT for example, does not fit well to the high frequency region of the spectrum. The latter can be further supported by the recently suggested Fan-Chirp transform -FChT [START_REF] Kepesi | Adaptive chirp-based time-frequency analysis of speech[END_REF][START_REF] Weruaga | The fan-chirp transform for non-stationary harmonic signals[END_REF]. The FChT uses a chirp related frequency basis adapted to the input signal. An example is depicted in Figure 4.9, where in the upper panel the speech signal in time domain is shown, in the middle panel the spectrogram obtained by the DFT is shown, and in the lower panel the corresponding spectrogram obtained from the FChT is illustrated. Black colored parts denote voiced speech, green colored parts denote unvoiced speech. Although the low voiced frequencies in the DFT-based spectrogram seem to have a regular structure, this is not true for the mid-and high-range frequencies, where the frequency content is blurred. This is exactly because of the non-stationary nature of the frequency content. On the contrary, the use of the FChT reveals a regularity in the frequency content across all frequencies of voiced parts.

To show this in a more illustrative way, in Transform in the same figure reveals that harmonic structure is present in the frequency range which is supposed to be modelled with stochastic components. This observation clearly shows that current speech analysis systems often overestimate the need of a MVF and a harmonic or sinusoidal representation could be applicable for all frequencies. Compared to the FChT, the aSMs provide more freedom in the instantaneous frequency curves, but the underlying principle is the same: local adaptivity.

However, it is questionable how and why sinusoids are appropriate when representing consonants, such as fricatives or voiceless/voiced stop sounds. A stop sound is produced with complete closure of the articulators involved, so that the stream of air can not escape through the mouth. Voiced stops are produced with vibrating vocal folds whereas in voiceless stops vocal folds are apart. In voiced stops, there are oscillations right before the burst, whereas in voiceless stops, there is no oscillation and, in many languages, there is aspiration after the burst. A fricative is produced with close approximation two articulators, so that the stream of air is partially obstructed and turbulent airflow is produced. It is well-known that conventional sinusoidal or harmonic models cannot efficiently tackle this problem, due to the highly non-stationary nature of these parts of speech and the stationarity assumption inside the analysis window of the models. The standard Sinusoidal Model [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] treats unvoiced parts, and hence stop sounds, the same way as voiced ones, based on the principle that periodogram peaks are close enough to satisfy the requirements imposed by the Karhunen-Loeve expansion [START_REF] Van Trees | Detection, Estimation, and Modulation Theory: Part I[END_REF]. However, the perceptual quality of the reconstructed speech is rather mediocre, especially under modifications. The Harmonic + Noise Model, and other systems as well, utilizes a stochastic component that models unvoiced speech as time-and frequency-modulated noise. Although this representation is perceptually closer and allows for better manipulation in case of modifications, still it does not attain the quality of the original speech. To this direction, let us examine more closely a consonant sample using the FFT and the FChT. In Figure 4.11, a fricative /s/ is depicted, along with its corresponding short time FFT and short time FChT obtained from a window centered in the middle of the sound, and the corresponding spectrograms based on the FFT and the FChT. Here, a similar conclusion can be drawn. Although there are not any prominent spectral peaks that can justify a sinusoidal model framework, intuitively, an adaptive decomposition of unvoiced speech should attempt to locate "optimal" frequency tracks that collectively minimize the mean-square error inside the frame. These "optimal" frequency tracks become more discernible in the FChT-based spectrogram, whereas in the DFT-based spectrogram severe blurring still exist.

In the next section, it will be shown how adaptivity can compensate the representation problem of fricatives and stop sounds, both voiceless and voiced. A complete and thorough study on the representation of unvoiced speech using adaptive speech models is beyond the scope of this thesis. For now, it is sufficient to show that adaptivity is capable of accurately representing stops and fricatives as AM-FM components.

Towards a uniform, adaptive, full-band AM-FM representation of speech

Let us roughly separate unvoiced speech into stop sounds (voiceless and voiced) and fricatives, and first deal with the former category. It has long been known that sinusoidal modelling is inefficient to model sounds well, since they are broadband signals and have noise-like frequency domain structure [START_REF] Macon | Speech Synthesis Based on Sinusoidal Modeling[END_REF]. Although sinusoidal models have been successfully applied for non-voiced speech, the nature of stops makes their modelling by a sum of stationary sinusoids inappropriate, because of the sudden change in amplitude during the release burst. An attempt to model the voiceless stops with a finite number of stationary sinusoids (i.e., one sinusoid every 80 -100 Hz) will manifest the Gibbs phenomenon just before the release time instant ( effect). This leads to an audible release energy smearing and therefore to a reconstructed signal with reduced intelligibility compared to the original signal. One could argue that an effort to model stops with a certain high amount of sinusoids would suffice; however, this is proved to be both insufficient and costly, since it requires a transient detection algorithm [START_REF] Levine | Audio Representations for Data Compression and Compressed Domain Processing[END_REF][Tho05] and some proper handling (i.e., transform coding [Lev99] [START_REF] Spanias | Speech Coding: A tutorial review[END_REF]). The use of short analysis windows when stop sounds are detected as in [START_REF] Levine | Audio Representations for Data Compression and Compressed Domain Processing[END_REF], does not alleviate the pre-echo effect as it will be also shown here. Other techniques such as multi-resolution sinusoidal analysis have failed to eliminate or alleviate the pre-echo effect [START_REF] Levine | Audio Representations for Data Compression and Compressed Domain Processing[END_REF]. Because of the aforementioned problems, copy strategies or are mostly used over the short time region of the attack onset in speech and audio synthesis state-of-the-art systems. However, SRER is commonly used as a global signal measure and thus small but pre-echo-related modelling errors at the abrupt part of the reconstructed signal may be buried into the global modelling error. So, additionally, a local SRER will be used in order to reveal the efficiency of the reconstruction around the pre-echo area. Experiments show that the provide a nearly pre-echo-free representation of , without the necessity of using very short analysis window lengths for these sounds, neither the use of a detector as in [START_REF] Levine | Audio Representations for Data Compression and Compressed Domain Processing[END_REF]. Also, it is shown that for the adaptive sinusoidal models the overall quality in modelling stops is high in terms of SRER. Since voiced stop sounds exhibit some oscillatory behaviour, and thus their modelling is not as difficult as for their voiceless counterparts, our main focus will be on the voiceless stop sounds.

Adaptive Sinusoidal Modelling of Stop Sounds

In this section, a comparison between the conventional sinusoidal model [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] and the adaptive sinusoidal models on a typical voiceless stop signal is presented. To this direction, a stop signal /t/ is extracted from a clear speech recording and is analyzed using the SM and the adaptive models. Since stops are broadband signals, attention should be paid in setting the parameters of the models. Both SM and adaptive models perform well under quasi-periodicity assumption, but this is not the case of this sound. SM performs peak picking on the spectrum of the input signal, so it does not need any initial frequency parameter values. On the other hand, adaptive models solve a least squares minimization problem, which requires a set of initial frequencies {f k }, (i.e., harmonic frequencies for a voice sound). It is suggested that for a sampling frequency of F s = 16 kHz, a low initial frequency value such as 80 Hz, which results in frequency values of 80k Hz, k = -100, • • • , 100, is enough to span the frequency spectrum, i.e. it is a full band analysis. The QHM frequency mismatch correction mechanism will fine-tune the frequencies around the maxima of the spectrum, and thus the highest energy components will be modelled.

For all models, the Hamming window is used and it is set to 3 times the larger pitch period (1/80 s). A 2048-point FFT is computed for the analysis frame and a maximum of 100 spectral peaks are allowed for the SM. The number of components is also set to 100 and five adaptations are allowed at most for the adaptive models. The frame rate is 1 sample for all models. Global as well as local SRER measures are computed. Local SRER focuses only before the release (burst) time and is computed over an interval of Nw 2 samples right before the onset of the waveform, where N w is half the analysis window length. Figure 4.12 shows the reconstructed signals for each case, with the aforementioned parameters, while Table 4.1 shows the global and local SRER evolution for all models. Based on the performance of the models in terms of local SRER, it is worth noticing that both adaptive models outperform the conventional sinusoidal model. Specifically, the eaQHM performs better than the aQHM, and both outperform SM in terms of reconstruction quality. Comparing the two adaptive sinusoidal models, the pre-echo effect is highly reduced for the aQHM, while it is mainly eliminated for the eaQHM. Moreover, the results from the global SRER show that both adaptive models produce high quality reconstruction of the stop sound compared with the conventional sinusoidal model. Experiments in manipulating the window length, the number of components, or both, did not provide any significant improvement for the SM representation. Therefore, it seems that the adaptation process is the key for accurate modelling of stops using long analysis windows. Moreover, it was observed that SM is unable to detect frequency components at the pre-echo area because of the stationary basis projection. This is not the case for the adaptive models, and it can be justified by the fact that adaptive modelling is a non parametric representation, taking into account local fre- quency (and amplitude, for the eaQHM) variation, which is pertinent for voiceless stop sounds. As a conclusion, adaptive sinusoidal modelling can represent highly non-stationary speech segments, like the voiceless stops, by projecting them on a set of also non-stationary basis functions that can capture the local characteristics of the signal. Thus, the pre-echo effect can be highly alleviated and sometimes eliminated, while a very high reconstruction performance is attained.

Database Validation for Stop Sounds

The next step is to strengthen the conclusions of the previous section using two databases of stop sounds.

Small Scale Validation

A small database of French speakers with male and female speakers is used for our purpose. Different voiceless stops corresponding to phonemes /p/, /t/, and /k/ are manually extracted from clean speech and are analyzed using the conventional sinusoidal models and the adaptive models, along with their voiced counterparts, for comparison purposes. The exact location of the burst release is manually identified, so as to compute local SRER accurately. The same metrics and parameters used in the previous section are also used here, i.e. a frame rate of 1 sample and an analysis window of 3 pitch periods. The sounds are categorized into classes of phonemes (20 waveforms for each class) and Table 4.2 shows mean value results for both global and local SRER. Apparently, adaptive modelling maintains its high SRER levels throughout different types of voiceless stops.

Large Scale Validation

A large scale validation is presented here. For convenience, only the eaQHM will be used in this validation, since it outperforms the aQHM and provides both amplitude and phase adaptation. A large database of both male and female French speakers is used. Phonetic labeling and manual segmentation is available in this database and thus stops can be easily extracted. In this experiment, 1000 stop sounds are considered. For such an amount of test signals, the exact burst locations are not available and consequently, the local SRER is not computed. Moreover, the frame rate of 1 sample used in the previous section, although providing remain the same as in the previous sections. The interpolation schemes used in this experiment are described in [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] and [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] (i.e., for SM, linear interpolation between amplitudes and cubic interpolation between phases). Table 4 As it can be observed from Table 4.3, the performance of the adaptive models sustains in high reconstruction levels, even with a frame rate up to 4 ms. The mean standard deviation per model is: 3 dB (SM) and 4.5 dB (eaQHM). No significant variations in standard deviation were observed across phonemes. Experiments with higher frame rates, such as 5 and 10 ms, showed an average decrease in performance of 3 and 7 dB respectively, compared to the 4 ms case, for all models and phonemes. Moreover, at higher frame rates the pre-echo effect was partially alleviated only for eaQHM modelling. Therefore it is suggested, as a rule of a thumb, the use of as low frame rate as possible. The average number of adaptations required for the convergence criterion in Eq.(2.52) is found to be 4.7 for the eaQHM, for all step sizes presented in Table 4.3.

Adaptive Sinusoidal Modelling of Fricative Sounds

As a reminder, fricatives are consonants produced by forcing air through a narrow passage made by placing two articulators close together. For modelling such sounds, a similar strategy as for stop sounds is followed for their analysis. A test case of a fricative /s/ is depicted in Figure 4.13. The signal is sampled at F s = 16 kHz, and a low initial frequency value such as 80 Hz, which results in frequency values of 80k Hz, k = -100, • • • , 100, which cover the full-band of the spectrum. The other experimental settings are exactly the same as in the stop sound example discussed earlier, except that aQHM has been omitted for convenience.

In Table 4.4, the SRER performance of adaptive models compared to the standard SM is presented for our test case, only here there is no local SRER calculation. Clearly, adaptivity is able to represent fricatives very accurately, compared to stationary models, such as SM.

Model

Global SRER (dB) SM 8.86 eaQHM 27.63 Table 4.4: Signal to Reconstruction Error Ratio values (dB) for all models on a fricative sound /s/.

Database Validation for Fricative Sounds

To validate our results, 485 voiced and voiceless fricatives have been automatically extracted from speech utterances. Voiced fricatives include /v/, /D/, /s/, and /S/, while unvoiced ones are /f/, /T/, /z/, and /Z/. The number of fricatives extracted from male speaker was almost the same with those from female speakers. As in the case of stop sounds, the frame rate of 1 sample used in the previous section is not realistic for applications. Hence, the same frame rates as previously are selected, namely 1 ms, 2 ms, and 4 ms. Parameters other than the frame rate remain the same as in the previous sections. As it can be observed from Table 4.5, the performance of the adaptive models sustains in high reconstruction levels, even with a frame rate up to 4 ms. The mean standard deviation per model is: 3.4 dB (SM) and 4.1 dB (eaQHM). No significant variations in standard deviation were observed across different fricatives. Experiments with higher frame rates were performed as well, such as 5 and 10 ms, that showed an average decrease in performance of 3.9 and 6.5 dB respectively, compared to the 4 ms case, for all models and fricatives. Therefore it is suggested, as a rule of a thumb, the use of as low frame rate as possible to attain a high enough perceptual and reconstruction quality. The average number of adaptations required for the convergence criterion in Eq.(2.52) is found to be 4.7 for the eaQHM, for all step sizes presented in Table 4.5.

Discussion

In this section, modelling of voiceless stop sounds and fricatives is presented and addressed via adaptive modelling. The well-known pre-echo effect of stop sounds in sinusoidal modelling is demonstrated and a solution is shown to be provided by the eaQHM. Pre-echo arises from the inability of sinusoidal models to represent highly non-stationary short time attacks, typically encountered in voiceless stop sounds. Using adaptive modelling, the pre-echo effect is greatly alleviated. The latter is demonstrated analytically using a characteristic example, where the limitations of sinusoidal modelling are also presented, and is validated on two different databases of stop sounds. Metrics such as global SRER for overall modelling and local SRER for a specific focus on the pre-echo effect are used and confirm the superiority of adaptive over stationary (conventional) sinusoidal modelling in representing highly non-stationary parts of speech. Moreover, fricatives are demonstrated to be represented very accurately using adaptive models. It is shown that local adaptation of the analysis parameters results in AM-FM components that are able to decompose and reconstruct fricative sounds effectively. SRER measures validate the latter for different fricative categories and different frame rates. Conclusively, it has been demonstrated that the adaptive models are capable of modelling not only voiced speech but also unvoiced parts as well with high accuracy. This is important to support the transition from hybrid systems to full-band systems that operate on the full-length of the speech signal, without any quality degradation, and thus providing a uniform representation of speech as AM-FM components.

The full-band eaQHM analysis and synthesis system

In the previous section, motivation towards full-band systems was presented in the context of analysis and synthesis of consonants. It is now apparent that the eaQHM can handle both categories of speech, voiced and unvoiced. However, care should be taken in the details of the implementation of such a system, since stability, robustness, and consistency are not only desirable for analysis and synthesis but also for modifications. The details of a full-band eaQHM-based analysis and synthesis system will be presented next.

The idea behind full-band eaQHM is that a first purely harmonic approximation of the speech signal is obtained, which successively -through adaptations and frequency corrections -converges to an adaptive, quasi-harmonic representation.

The full-band signal is described as an AM-FM decomposition

d(t) = K � k=-K A k (t)e jφ k (t) (4.27)
where A k (t) is the instantaneous amplitude and φ k (t) is the instantaneous phase of the k th component, respectively. The instantaneous phase term is given by

φ k (t) = φ k (t i ) + � t ti 2πf k (u)du (4.28)
where φ k (t i ) is the instantaneous phase value at the analysis time instant t i , f s is the sampling frequency, and f k (t) is the instantaneous frequency of the k th component.

Analysis

At first, an initial and continuous f 0 estimation for all frames is obtained, noted by f0 . Although there is no f 0 in unvoiced frames, a rough estimate can be useful for initialization. Then, the next step is to assume a full-band harmonicity to obtain a first estimate of the instantaneous amplitudes of all the harmonics. Using a Blackman analysis window w(t) centered at t i and with support in [t i -T, t i + T ], where 2T is of 3 local pitch periods length, a frame of the analyzed speech is initially modelled using a simple Harmonic Model as:

d(t) = � L � k=-L a k e j2π fk t � w(t) (4.29)
where a k is the complex amplitude of the k th harmonic, fk = k f0 are the analysis frequencies, and L is the number of harmonics that span the whole spectrum up to Nyquist frequency. The estimation of the model parameters is obtained via Least Squares, as described in [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. As opposed to [START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF], where the initial f 0 estimation is refined using an iterative QHM (iQHM), in our work no f 0 refinement is necessary, thus reducing the overall complexity of the algorithm, and a simple amplitude estimation for each component is performed. In [START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF], iQHM is operating as a means to refine the f 0 estimate for voiced frames. Since iQHM also holds the stationarity assumption, it was judged not to be crucial in frequency refinement, especially in unvoiced frames, where the refinement could lead to instability (e.g. abrupt jumps) of the estimated f 0 . Thus, the iQHM estimation was dropped in this system, allowing reduced complexity without loss of accuracy. As a final step, the overall signal can be synthesized by interpolating the |a k | and fk values over successive analysis time instants t i , thus obtaining

d(t) = L � k=-L Âk (t)

Adaptation

The above model is still harmonic and stationary within an analysis frame. Therefore, in order to converge to quasiharmonicity and to confront the stationarity issue, the projection of the signal onto a set of time-varying basis functions is suggested in [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF], by using the parameters a k and b k of the Quasi-Harmonic Model (QHM) [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF]. This yields the eaQHM model:

d(t) = � L � k=-L � a k + tb k �� Âk (t)e j φk (t) � � w(t) (4.34) with Âk (t) = Âk (t + t i ) Âk (t i ) (4.
�{a k }�{b k } -�{a k }�{b k } |a k | 2 (4.36)
Hence, at the first adaptation, for the analysis time instant t i , the instantaneous frequencies are fk (t i ) = k f0 (t i ) + ηk (t i ) and the instantaneous phases become

φk (t) = φk (t i ) + � t ti (2π fk (u) + c(u))du (4.37)
Then, a Least Squares solution for the a k , b k using these refined frequencies (and phases) leads to a better estimate of the instantaneous amplitudes Âk (t) = |a k (t)| and the ηk terms. By iteratively adding the ηk term of the current adaptation on the k th -frequency track of the previous adaptation, the frequency tracks deviate from strict harmonicity and represent the underlying actual frequencies better. Additionally, and on the contrary to previous works [START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF][START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF], where the frequency correction estimation ηk on each adaptation should be less than f 0 /2, in this approach it is supposed that after each adaptation the estimated frequencies become more and more localized to the actual frequencies, so the frequency correction for a given analysis time instant t i is constrained as in

|η k (t i )| ≤ f0 (t i ) m + 1 (4.38)
where m ∈ {1, • • • , M } is the current adaptation number and M is the maximum number of allowed adaptations (in our experiment, M = 6). This way, any relatively large frequency correction value -which often leads to audible artefacts -that might be obtained in a higher adaptation step will be suppressed. This constraint is also motivated by the fact that unvoiced parts should be handled by the same model. This way, the AM-FM components are kept more "tight" in their variability in noise representation. Finally, this adaptation scheme continues until a convergence criterion is met, which is related to the overall Signal-to-Reconstruction-Error Ratio (SRER), that is, when the SRER stops increasing after each adaptation, then the algorithm is considered to have converged.

Synthesis

In the synthesis stage, the k th instantaneous amplitude track, Âk (t), is computed via either linear or spline interpolation of the successive estimates from the last adaptation step. The k th instantaneous frequency track, f k (t), is also computed via spline interpolation. Also, it is worth noting that a frequency matching mechanism is trivial, since the analysis frequencies are integer multiples of a fundamental and the number of components is constant. As for the k th instantaneous phase track, φk (t), the non parametric approach based on the integration of instantaneous frequency is followed, as it is shown in the adaptation steps of the analysis. Finally, the speech signal can be approximated by its time-varying components using:

d(t) = L � k=-L Âk (t)e j φk (t) (4.39)
A block diagram of the algorithm is depicted in Figure 4.16.

Examples

Two examples are shown in Figures 4.14, 4.15 for a male and a female speaker, respectively.

The full-band aHM analysis and synthesis system

As discussed in Section 4.4, the adaptive Harmonic Model [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF] was developed by Degottex in parallel to this thesis, and its original form suggested an analysis and synthesis in the full-band. For completeness and convenience, a very brief review of the the analysis and synthesis schemes of the full-band adaptive Harmonic Model (aHM) is presented in this section. The aHM is actually nothing more than the aHNM discussed earlier, but with no noise component.

The adaptive Harmonic Model can be mathematically described as:

s(t) = K � k=-K a k (t)e jkφ0(t) (4.40)
where a k (t) is a complex function that copes with the amplitude and the instantaneous phase of the k th harmonic component, while K is the number of the components, and φ 0 (t) is a real function defined as the integral of the fundamental

) ( ) ( ˆ0 0 t t A k k  Windowing Basis functions LS Frequency Correction Parameter Interpolation ) ( ˆt d ) (t d ) ( ˆ) ( ˆt t A i k i k  0 kf k f ˆk a Parameter Interpolation Adaptive Basis functions LS k k b a Converge? No Yes ) ( ˆ) ( ˆt t A i k i k  Analysis Step ) ( 0 t f Synthesis Step Sinusoidal Synthesis 1   i i ) ( ) ( ˆt t A k k  )) ( êxp( ) ( ˆt j t A k L L k k    
φ 0 (t) = � t 0 2πf 0 (u)du (4.41)

Analysis

In the analysis step, a parametrization of the speech signal at each analysis time instant t i a is undertaken. At first, a sequence of the analysis time instants are created in the voiced parts of speech using the provided f 0 (t) track, such we have one analysis time instant per pitch period. In unvoiced segments, even though the estimated f 0 (t) is meaningless, it can be used to generate the corresponding analysis time instants. Moreover, if the distance between t i a and t i+1 a is short enough, aHM can model the amplitude variations of the unvoiced signal (like in plosives). Thus, the upper limit of the size of the analysis window is 20ms and the lower limit comes from the provided f 0 (t) track, and is therefore set to 50Hz. Around each analysis time instant t i a , a Blackman window with a length of 3 local pitch periods is applied to the speech signal. The phase track φ 0 (t) is then computed by means of spline interpolation of f i 0 and using the integration formula in Eq.(4.41).

Synthesis

In the synthesis step, each harmonic is generated in separate, one after the other, without using any window. Each harmonic component is synthesized by its parameters, namely its amplitudes |a i k |, phases ∠a i k , and fundamental frequency f i 0 . First, the instantaneous amplitude, |a k (t)|, of the k th harmonic is simply obtained by linearly interpolating the estimated |a i k | on the analysis time instants t i a , on a logarithmic scale. The instantaneous phase ∠a i k cannot be interpolated directly across time to obtain a k (t) because of its rotation due to the time advance between analysis time instants. Therefore, it is proposed to remove this effect using the integral of f 0 (t) from the start of the signal, and obtain the relative phase -RP:

∠ã i k = ∠a i k -kφ 0 (t i a ) (4.42)
Thus, by assuming that the shape of the signal is changing smoothly, the phase values change also smoothly from one analysis time instant to the other. Then, the RP ∠ã i k can be interpolated to obtain its continuous counterpart, ∠ã k (t). Additionally, a spline or cubic interpolation is necessary such as its time derivative, the frequency, is still continuous. All along the iterative process, and since the harmonic numbers K i increase independently from one analysis time instant to the other, there are often missing components in the interpolations of amplitude and instantaneous phase. If this is the case, then the amplitude of the missing component is set to -300 dB and the corresponding phase ∠ã k (t) is set to zero.

Examples

Two examples are shown in Figures 4.17

Evaluation and Results

In this section, results will be shown on the resynthesis of the speech signal. Both objective and subjective measures will be presented. A comparison to the state-of-the-art will be discussed as well. Due to their availability, the following models will be selected for comparison to the adaptive models: Sinusoidal Model (SM), Harmonic + Noise Model (HNM), and STRAIGHT. In these experiments, a database of 32 speech utterances was used, including 16 male and 16 female speakers from 16 different languages: Greek, French, English, Spanish, Finnish, Chinese, Portuguese, Basque, Japanese, Italian, German, Korean, Russian, Arabic, Indonesian, and Turkish. All waveforms were sampled at 16 kHz.

In objective evaluation, the SRER is computed for the whole waveform, serving as an estimate of the total residual energy "missed" by each model. The higher the SRER value, the more information is captured by the model used. Since only the full-band models attempt to accurately reconstruct the speech signals, only those will be considered in the evaluation.

In subjective evaluation, a formal listening test has been conducted in order to measure perceptual quality. In this evaluation, all models are included: SM, HNM, STRAIGHT, aHM, aHNM, eaQHNM, eaQHM. The listening test has the form of Figure 4.21, and in most of the times, it was available on-line.

The parameters for the models were the following: for both pitch estimators, the pitch was estimated every 1 ms and their f 0 estimation limits were [70, 220] Hz and [120, 350] Hz for males and females, respectively. For AIR-f 0 , which was used in the aHM model only, the analysis window is of Blackman type and its length is 3 local pitch periods, whereas the step size is pitch period synchronous. For the model parameter estimation, the analysis window is of Blackman type for aHM, and Hamming type for the eaQHM and SM. Their size is 3 times the local pitch period and the analysis step size was 2.5 ms, for all models. For the STRAIGHT method, the default parameters are used, and for the HNM, a synchronous analysis is considered, with a maximum voiced frequency of 5500 Hz.

Objective Evaluation

In objective analysis, the Signal-to-Reconstruction-Error Ratio (SRER) is chosen to measure the accuracy of the numerical representation between the original and the resynthesized speech. In Table 4.6, the mean and the standard deviation of the SRER for all utterances in our database are presented for both pitch estimators. It is clearly evident that quasi-harmonicity can capture more information of the underlying speech signal, with the same number of synthesis parameters. Figure 4.20 shows the first 16 frequency tracks in the analysis step for an utterance produced by Greek male speaker, the local SRER for a sliding window of 30 ms, and the corresponding speech waveform for the two adaptive models. It should be noted that the overall SRER for the eaQHM is 34.67 dB whereas for the aHM is 25.60 dB for this sample, which contains both voiced and unvoiced areas. Intuitively, the eaQHM components in unvoiced speech attempt to locate "optimal" frequency tracks that collectively minimize the Mean-Square Error inside a frame. In this figure, it is obvious that in AIR-aHM all components are purely harmonic, and any slight fluctuation of the f 0 propagates in the higher harmonics. In the eaQHM however, the upper frequency components deviate from the multiples of the f 0 and their structure seems smoother. Based on the lower panel (time-varying SRER), it seems that the representation suggested by the eaQHM (middle panel) is more accurate compared to that one obtained by aHM (upper panel). Also, it should be mentioned that in our experiments, no manual refinement of the estimated f 0 is performed.

Subjective Evaluation

For perceptual quality evaluation, a formal listening test was designed. A part of it is currently available on-line1 . The listeners were asked to evaluate the perceptual quality of the resynthesized speech compared to the original one, for all different models. An 1 -5 scale was used in the evaluation according to the recommendation ITU-R BS [START_REF]The ITU Radiocommunication Assembly[END_REF], with each scale being (1) "Very bad", (2) "Bad". (3) "Good", (4) "Very good", (5) "Perfect". The results from 34 listeners are depicted in Figures 4.22 and 4.23. In the same plot we show the 95% confidence interval. This shows that the obtained results are statistically significant. Please note that among these listeners, only 10 were familiar with signal processing and listening tests.

According to the listeners, the overall quality of all adaptive models is much higher than the state of the art. Moreover, perceptual differences between the adaptive models were not easy to find, and it was clearly stated that these differences are mostly present in the unvoiced parts, and especially in transients and sharp onsets of voiceless stop sounds (for example, in an aspirated velar /k/ in the utterance of Figure 4.19 by a Korean female). In general, it is acknowledged that the hybrid adaptive models (eaQHNM and aHNM) differ from the original in the unvoiced parts, where the modulated noise representation does not attain the quality of the original signal. However, their quality is superior than the state-ofthe-art (STRAIGHT, HNM, SM). Moreover, by looking at the objective measures, it is interesting that although AIR-aHM performs significantly lower in terms of reconstruction, this does not translate to a respective quality degradation, as in the SM, where there is a substantial degradation, compared to the other two models. Finally, it is interesting that although the pitch estimators behave differently, both the adaptive models appear to be very stable in the reconstruction of output speech, as Table 4.6 shows.

Conclusions

In this chapter, we presented two hybrid and two full-band systems of analysis and synthesis of speech. The two hybrid systems have a deterministic and a stochastic component. The deterministic component is modeled either by the extended adaptive Quasi-Harmonic Model (eaQHM) or by the adaptive Harmonic Model (aHM). The former models speech as sum of AM-FM components that are quasi-harmonically related whereas the latter estimates its parameters using the theory of adaptivity but the resynthesis is purely harmonic, by iteratively refining a fundamental frequency estimate. The unvoiced parts of speech are modeled as time and frequency modulated Gaussian noise.

In addition, motivation for applying the models in the full-band of speech is proposed. These include (a) the questionable nature of the so-called maximum voiced frequency (MVF). More powerful analysis tools such as the Fan-Chirp Transform (FChT) have shown that there is structure in voiced speech segment up to the Nyquist frequency, and thus no noise component is necessary for high frequency representation, and (b) that fact that continuous, adaptive quasiharmonic tracks have been proved to represent very accurately voiced and voiceless stop sounds as well as fricatives, provided that the frequency tracks span the spectrum adequately. The transient and stochastic nature of these sounds are well represented by quasi-harmonic tracks due to the frequency correction mechanism of QHM and the adaptation process. Suitable experiments for both stop and fricative sounds have confirmed this proposal, for different analysis step sizes.

Based on these observations, the full-band eaQHM has been developed, in parallel to the already developed aHM. The full-band eaQHM models all parts of speech as AM-FM sinusoids. Compared to the deterministic part of the eaQHNM, the full-band eaQHM starts from a strictly harmonic representation of speech and successively, through the frequency correction and adaptation mechanism, converges to quasi-harmonicity. The full-band aHM is identical to the deterministic part of the aHNM.

A comparison between available full-band models (SM, aHM, eaQHM) in terms of signal reconstruction is undertaken. The Signal-to-Reconstruction-Error Ratio (SRER) is a measure of closeness between the original and the reconstructed signal. It is shown that the eaQHM outperforms both aHM and SM in terms of SRER. From a perceptual point of view, a formal listening test revealed the superiority of the adaptive models (hybrid and full-band) compared to the state-of-the-art. Among all models, the eaQHM provides a transparent quality, indistinguishable from the original speech, whereas the aHM performs similarly well.

Chapter 5

Speech Modifications based on Adaptive Sinusoidal Models

Having analytically discussed the approaches, methods, and properties of the hybrid and full-band adaptive Sinusoidal Models, this chapter proposes methods for prosodic modifications of speech. First, modifications based on hybrid systems will be presented, following a similar approach as in milestone works [START_REF] Serra | A System for Sound Analysis, Transformation, Synthsis based on a Determistic plus Stochastic Decomposition[END_REF][START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. This means that modifications must be applied on both components (deterministic and stochastic). Then, we focus more on modifications on the full-band systems which are proposed next, since the reconstruction performance of these models outperforms the corresponding of the hybrid models. Due to the purely deterministic (sinusoidal) representation of the full-band models, the most challenging part in modifications is how to modify the non-voiced parts of speech to attain a high perceptual quality.

A general flowchart for modifying speech using hybrid and full-band systems is given in Figure 5.1. We can observe that in full-band systems, the manipulation of a single component is performed. On the other hand, in hybrid systems both components are to be differently manipulated. Before going into the details, let us remind the purpose of time and pitch scaling in technical terms. The purpose of time-scale modification is to maintain the perceptual quality of the original speech signal while changing the apparent rate of articulation. The purpose of pitch-scale modification is to change the pitch contour of the original speech signal while maintaining the apparent rate of articulation. The pitch contour (and thus the harmonics) should be shifted in frequency, and the formant structure should not be changed at a different rate than the rate of the input speech.

Preprocessing

Time Scaling

In time scaling harmonically related sinusoids, the interpolation of instantaneous parameters is necessary to change the articulation rate. For instantaneous amplitudes and frequencies, this is performed via linear and spline interpolation, respectively. However, for the phase, the process is not straightforward because it is rotation due to the time advance across time instants. Therefore, it is proposed to remove this effect using the integral of kf 0 from the start of the signal, and obtain the relative phase -RP [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF][START_REF] Kafentzis | Time-scale Modifications based on an Adaptive Harmonic Model[END_REF]. Thus, by assuming that the shape of the signal is changing smoothly, the phase values change also smoothly from one analysis time instant to the other. Then, the RP can be interpolated to obtain its continuous counterpart. Additionally, a spline or cubic interpolation is necessary such as its time derivative, the frequency, is still continuous. This way, the time scaled waveform is shape invariant.

Relative Phase

A theoretical basis for the relative phase notion follows next. Specifically, the linear phase term is sought to be removed during the resampling process in time scaling. Related work on linear phase removal has been suggested in other speech processing applications, such as concatenative speech synthesis [START_REF] Stylianou | Removing linear phase mismatches in concatenative speech synthesis[END_REF], speech transformations [START_REF] Federico | Waveform preserving time stretching and pitch shifting for sinusoidal models of sound[END_REF], and speaker verification [LPY + 12]. Let us consider a sinusoid

x 0 (t) = cos � 2π � t 0 f 0 (u)du + θ 0 � (5.1)
which we will consider as the reference sinusoid, and another sinusoid,

x k (t) = cos � 2πk � t 0 f 0 (u)du + θ k � , k ∈ Z + (5.2)
The instantaneous phases of the two sinusoids are

φ 0 (t) = 2π � t 0 f 0 (u)du + θ 0 , φ k (t) = 2πk � t 0 f 0 (u)du + θ k (5.3)
respectively. Let us consider that θ 0 = 0, meaning that the time origin is set as the point where φ 0 (0) = 0. If we choose any analysis time instant t i a , the instantaneous phases become

φ 0 (t i a ) = 2π � t i a 0 f 0 (u)du , φ k (t i a ) = 2πk � t i a 0 f 0 (u)du + θ k (5.4)
respectively. By changing variables, we get

θ k (t i a ) = φ k (t i a ) -kφ 0 (t i a )
(5.5)

The last equation describes the relative phase in analysis time instants t i a . For time scaling, one would simply interpolate and time-scale θ k (t i a ) in successive analysis time instants to obtain θ � k (t � ). Then, the instantaneous phase for the time scaled fundamental frequency would be

φ � 0 (t � ) = 2π � t � 0 f � 0 (u)du (5.6)
and thus, the new instantaneous phase for the k th harmonic would be

φ � k (t � ) = θ � k (t � ) + kφ � 0 (t � ) (5.7)

Relative Phase Delay

For time scaling the instantaneous phase of quasi-harmonically related sinusoids, one would suggest to remove the integral of f k (t) (instead of kf 0 (t)) from the k th instantaneous phase track, also from the start of the signal, thus obtaining the relative phase which owns most of the randomness and all of the glottal pulse shape, and still changes smoothly from one time instant to the other. However, this approach from the well-known problem of phase dispersion, that is, while the scaled signal has the same frequency content, the phases between the components change, resulting in a different wave shape. The phase dispersion problem has been addressed before in [START_REF] Quatieri | Shape-Invariant Time-Scale and Pitch Modifications of Speech[END_REF][START_REF] Federico | Waveform preserving time stretching and pitch shifting for sinusoidal models of sound[END_REF], among others. A similar strategy will be followed here, using the concept of relative phase delay, first proposed in [START_REF] Federico | Waveform preserving time stretching and pitch shifting for sinusoidal models of sound[END_REF].

The phase delay of the k th sinusoid in the i th analysis frame is defined as

τ i k = φ k (t i a ) ω k (t i a ) = φ k (t i a ) 2πf k (t i a )
(5.8) where φ k (t i a ) is the phase value of the k th sinusoid at analysis time instant t i a . The relative phase delay is defined as the difference between the phase lag of the k th sinusoid and that of the first one, which corresponds to the fundamental frequency,

∆τ i k = τ i k -τ i 0 (5.9)
The signal is modified so as to preserve the relative phase delay. In a similar way, we define the phase delay at the synthesis time instants t j s as

τ j k = φk (t j s ) ωk (t j s ) = φk (t j s ) 2πf k (t j s )
(5.10)

In order to ensure shape invariance, equal relative phase delays at analysis and synthesis times are imposed:

∆τ i k = ∆τ j k
(5.11)

τ i k -τ i 0 = τ j k -τ j 0 (5.12) φj k = (τ j 0 + (τ i k -τ i 0 ))2π f j k (5.13)
The instantaneous phase φ k (t j s ) of the fundamental frequency at synthesis time instants is computed using the formula

φj 0 = φj-1 0 + β(φ i 0 -φ i-1 0 ) (5.14)
where β is the time scale factor, and φ i k is the unwrapped phase value at time instant i. Having the instantaneous phase values for all frequencies at the analysis time instants, the same strategy as in plain resynthesis is used, that utilizes frequency integration.

In hybrid systems, the above discussion is applied for the deterministic part, whereas for the stochastic part, a simple time stretching of the parametric noise envelope is sufficient. For sample-by-sample lattice filtering representation of noise, a simple interpolation of the reflection coefficients should be performed.

Pitch Scaling

In pitch-scaling, the estimation of a new set of amplitude, frequency, and phase values is necessary due to pitch shifting. These values can be obtained by estimating the so-called amplitude and phase envelopes in the spectral domain. Spectral estimation is a field of study that has received increased attention because of the variety of its applications (voice conversion [START_REF] Godoy | Voice conversion using dynamic frequency warping with amplitude scaling, for parallel or nonparallel corpora[END_REF], word recognition [START_REF] Magi J. Pohjalainen | Stabilised Weighted Linear Prediction[END_REF], speech recognition [START_REF] Wolfel | Minimum variance distortionless response spectral estimation[END_REF], speaker verification [HKS + 12], speaker identification [START_REF] Reynolds | Robust text-independent speaker identification using gaussian mixture speaker models[END_REF], to name a few), and many algorithms are available to achieve it in a robust manner, such as cepstrum-based techniques [GR90, CM96], AR models [EJM91, TKMI94, MG76], and multi-frame analysis [START_REF] Shiga | Estimating the spectral envelope of voiced speech using multi-frame analysis[END_REF][START_REF] Toda | Statistical approach to vocal tract transfer function estimation based on factor analyzed trajectory hmm[END_REF].

Since pitch scaling requires the estimation of amplitudes in the new, shifted frequencies, the Discrete All-Pole method [START_REF] El-Jaroudi | Discrete All-Pole Modeling[END_REF] is used in this work for both models. For the phase, a simple approach is suggested for the aHMbased systems, which involves the computation and the interpolation of the relative phases, as mentioned in time scaling sections. For the eaQHM-based systems, the notion of relative phase delays is also employed, in order to minimize phase dispersion.

Amplitude Estimation

Amplitude estimation is performed via an all-pole technique, called the Discrete All-Pole method (DAP). This method utilizes a discrete version of the Itakura-Saito (IS) distortion measure as its error criterion, instead of a time-domain criterion that most of other all-pole models use. The IS error measure is given by

E IS = 1 N N � m=1 X(ω m ) X(ω m ) -log X(ω m ) X(ω m ) -1 (5.15)
where X(ω m ) is the given discrete spectrum defined at N frequency points, and X(ω m ) is the all-pole model spectrum evaluated at the frequencies ω m ∈ [0, f s /2], where f s is the sampling frequency. This method manages to overcome the well-known limitations of linear prediction [START_REF] Makhoul | Linear Prediction: A Tutorial Review[END_REF] and produces better fitting of spectra that are represented with a small set of discrete values, such as the case of sinusoidal models.

The DAP method works iteratively to solve a nonlinear set of equations, in order to converge to a global minimum. The order, P , of the method does not differ from the empirical choice that is employed in most all-pole methods, that is

P = f s 1000 + 2 (5.16)
where f s is in Hertz. The DAP method is used for spectral envelope estimation for all models. More details on DAP can be found in [START_REF] El-Jaroudi | Discrete All-Pole Modeling[END_REF].

Phase Estimation

For the aHM-based systems, the relative phase is once again used. After estimating and interpolating the relative phases, the integral of the shifted frequencies is added back, to obtain the instantaneous phases of each harmonic. Mathematically, each frequency track is modified as

kf 0 (t) → ρkf 0 (t)
(5.17

)
where ρ is the pitch scale factor. Next, the relative phases are computed as in Eq. (5.5) and interpolated over time. Finally, the pitch-scaled frequencies are integrated and added back to the continuous relative phases, yielding the instantaneous phases of the pitch-scaled frequencies

φ � k (t) = θ k (t) + 2π � t 0 ρkf 0 (u)du (5.18)
The relative phase values allow the reconstruction of the shape of the signal, using the reference phase φ 0 (t i a ) in a synchronous reconstruction. For the purposes of pitch-scale modification, the f 0 track can be changed without any recomputation of the phase, because if the RPs are kept constant, the waveform will stretch or shrink accordingly without any other change.

For the eaQHM-based systems, an extension of the time scaling algorithm presented earlier is suggested. Provided that the pitch scaling factor ρ is constant over the duration of a frame, the phase variation induced by pitch-scaling is equivalent to that produced by time scaling using the same factor. Thus, Eq. (5.14) is changed into

φj 0 = φj-1 0 + ρ(φ i 0 -φ i-1 0 ) (5.19)
and based on this phase track, the rest of the instantaneous phase values at synthesis time instants t j s (which can be the same as the analysis ones, or different, if both time and pitch scale are applied) are generated using the relative phase delays as

φj k = (τ j 0 + (τ i k -τ i 0 ))2πρ f j k (5.20)
The instantaneous phases are computed once again using the integration scheme of the analysis. In hybrid systems, the above discussion is applied for the deterministic part, whereas for the stochastic part, no modification is performed.

For the modifications part, we will focus only to the full-band systems, since in hybrid systems, only the stochastic part is different, and its modification methods are well-known [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF].

Speech Modifications based on the aHM system

Time-Scale Modification Scheme

The purpose of time-scale modification is to maintain the perceptual quality of the original speech signal while changing the apparent rate of articulation. In this system, deterministic and stochastic parts are scaled separately and in a different manner.

The pitch contour (and thus the harmonics) should be stretched or compressed in time, and the formant structure should be changed at a slower or faster rate than the rate of the input speech, but otherwise not modified. For an arbitrary time-scale modification, the time t in the original signal is mapped to a time t � in the modified signal. For that, a mapping function referred to as the time-scale warping function is defined:

D(t) = � t 0 β(τ )dτ (5.21)
where β(τ ) > 0 is the time-varying time-scaling rate. When β(τ ) > 1, then the articulation rate is slowed down, whereas the opposite happens when β(τ ) < 1. Note that for a fixed β(τ ) = β, then the time-scale warping function is reduced to a linear function of time, i.e. D(t) = βt.

In the adaptive Harmonic model context, the parameters should be transformed in the way described next. Let us remind that in an analysis window centered at t i a , the instantaneous components {a i k , f i 0 }, are known. From these, we can compute their continuous counterparts, which are the instantaneous amplitudes A k (t) = |a k (t)| and frequencies f 0 (t), obtained by interpolating a i k and f i 0 , respectively. Then, the time-scaled waveform, ŝT S (t � ) is given by:

ŝT S (t � ) = K � k=-K A � k (t � )e jφ � k (t � ) (5.22)
where A � k (t � ) and φ � k (t � ) are computed using the following way:

1. The instantaneous amplitudes are time-scaled:

A � k (t � ) = A k (D -1 (t)) (5.23) 2.
In order to compute φ � k (t � ), it is first necessary to compute the time-scaled frequencies. The instantaneous frequencies in the modified signal at time t � correspond to the instantaneous frequency in the original signal at time

D -1 (t � ): kf � 0 (t � ) = kf 0 (D -1 (t � )) (5.24)
where D -1 (t) is the inverse time-scale warping function.

3. Finally, to obtain a shape-preserving waveform, the relative phase (RP) values of the analysis need to be timescaled. Therefore, we first compute the continuous time-scaled RPs, ∠ã k (t), from the corresponding values, ∠ã i k . For this, the RP is first computed by extracting the integral of the frequency from the phase information at analysis time instant t i a , as in Eq.(5.25):

∠ã i k = ∠a i k -kφ 0 (t i a ) (5.25)
Then, the RP values are interpolated, thus obtaining ∠ã � k (t � ), as:

∠ã � k (t � ) = ∠ã k (D -1 (t � )) (5.26)
and finally, the integrated time-scaled frequency is added back to the interpolated RP values:

φ� k (t � ) = ∠ã � k (t � ) + � t � 0 2πkf � 0 (u)du (5.27)
This way the waveform retains its shape for voiced parts, i.e. the time-scaling is shape-invariant.

The time-scaling algorithm for aHM is given in Algorithm 2.

Algorithm 2 Time-scaling using aHM

Require: A set of parameters per analysis time instant (frame):

{A k (t i a ), f 0 (t i a ), ∠a k (t i a )} Interpolate successive t i a to obtain t � Interpolate A k (t) to obtain A � k (t � ) using Eq.
(5.23) Interpolate kf 0 (t) to obtain kf � 0 (t � ) using Eq. (5.24) Estimate relative phases ∠ã i k using Eq. (5.25) Interpolate relative phases to obtain ∠ã � k (t � ) using Eq. (5.26) Estimate instantaneous phase φ� k (t � ) using Eq. (5.27)

Time scaled speech

Synthesize ŝT S (t � ) using harmonic synthesis

Example of Application

In Figure 5.2 the shape invariance property of the aHM is clearly demonstrated for different time-scale factors, 0.5, 1.5, and 2.5. A male speaker waveform is presented. In Figure 5.3 the corresponding waveforms of Figure 5.2 is depicted in the frequency domain.

Pitch-Scale Modification Scheme

For an arbitrary pitch-scale modification, the input f 0 (t) contour is mapped to a different one, f � 0 (t) = ρ(t)f 0 (t) in the modified signal, where ρ(t) is the pitch-scale factor function. When ρ(t) > 1, then the pitch increases, whereas the opposite happens when ρ(t) < 1. Note that for a fixed ρ(t) = ρ, the pitch modification is constant throughout the whole waveform.

In the adaptive Harmonic model context, the parameters should be transformed in the way described next. Let us remind that in an analysis window centered at t i a , the instantaneous components {a i k , f i 0 }, are known. From these, we can compute their continuous counterparts, which are the instantaneous amplitudes A k (t) = |a k (t)| and frequencies f 0 (t), obtained by interpolating a i k and f i 0 , respectively. Then, the pitch-scaled waveform, s P S (t), for a constant pitch-scale factor is given by:

ŝP S (t) = K � k=-K A � k (t)e jφ � k (t)
(5.28)

where A � k (t) and φ � k (t) are computed using the following way:

1. In order to compute φ � k (t), it is first necessary to compute the pitch-scaled frequencies. Thus the new frequencies are given by: kf 0 (t) ← ρkf 0 (t) (5.29)

2. The instantaneous amplitudes at analysis time instants t i a , A � k (t i a ), are computed from sampling the spectral envelope at the corresponding frequencies ρkf 0 :

A � k (t i a ) = DAP (t i a , ρkf 0 ) (5.30)
where DAP (t i a , f ) is the Discrete All-Pole-based envelope constructed around time instant t i a . Then, the k th instantaneous amplitude is linearly interpolated across successive time instants.

3. Then, the instantaneous phase should be re-computed. For this, the RP is first computed by extracting the integral of the initial fundamental frequency from the phase information at analysis time instant t i a , as in Eq. (4.42). Then, the RP values are interpolated, thus obtaining ∠ã k (t), and finally, the integrated pitch-scaled frequency is added back to the interpolated RP values:

φ� k (t) = ∠ã k (t) + 2π f s � t 0 ρkf 0 (u)du (5.31)
To make the latter steps more algorithmic, please note Algorithm 3:

Algorithm 3 Pitch-scaling using aHM

Require: A set of parameters per analysis time instant (frame): {A k (t i a ), f 0 (t i a ), ∠a k (t i a )} Compute pitch-scaled frequencies using Eq. (5.29) Compute the spectral envelope DAP (t i a ) around time instant t i a . Sample the spectral envelope at the corresponding frequencies ρkf 0 using Eq. (5.30) Interpolate instantaneous amplitudes over successive time instants t a i to obtain A � k (t) Estimate relative phases ∠ã i k using Eq. (5.25) Interpolate relative phases to obtain ∠ã � k (t) using Eq. (5.26) Estimate instantaneous phase φ� k (t) using Eq. (5.31)

Pitch scaled speech

Synthesize ŝP S (t) using harmonic synthesis

Example of Application

In Figure5.4, the aHM pitch shifting is demonstrated for different pitch-scale factors, 0.5, 1.5, and 2.0. A female speaker waveform is presented. In Figure 5.5 the corresponding waveforms of Figure 5.4 are depicted in the frequency domain.

Speech Modifications based on the eaQHM system

Time-Scale Modification Scheme

In this case, the pitch contour (and thus the quasi-harmonics) should be stretched or compressed in time, and the formant structure should be changed at a slower or faster rate than the rate of the input speech, but otherwise not modified. For an arbitrary time-scale modification, the time t in the original signal is mapped to a time t � in the modified signal. For that, a mapping function referred to as the time-scale warping function is defined:

D(t) = � t 0 β(τ )dτ (5.32)
where β(τ ) > 0 is the time-varying time-scaling rate. When β(τ ) > 1, then the articulation rate is slowed down, whereas the opposite happens when β(τ ) < 1. Note that for a fixed β(τ ) = β, then the time-scale warping function is reduced to a linear function of time, i.e. D(t) = βt. In the adaptive sinusoidal model context, the parameters should be transformed in the way described next. Let us first assume that in an analysis window centered at t i , the instantaneous components {A k (t i a ), f k (t i a ), φ k (t i a ), are known, and the component trajectories have been computed, i.e. A k (t), f k (t), φ k (t), within this frame and up to the center of the next frame, t i+1 a . Then, time scaling requires the following steps to be performed. Let ŝ(t) denote the AM-FM decomposed signal:

ŝ(t) = L � k=-L Âk (t)e j φk (t)
(5.33)

1. The instantaneous amplitudes are time-scaled:

A � k (t � ) = A k (D -1 (t � )) (5.34)
2. The instantaneous frequencies in the modified signal at time t � correspond to the instantaneous frequency in the original signal at time D -1 (t � ): 3. The instantaneous phase values of the fundamental phase for the synthesis frames are computed:

f � k (t � ) = f k (D -1 (t � )) (5.35)
φ 0 (t j s ) = φ 0 (t j-1 s ) + β(φ 0 (t i a ) -φ 0 (t i-1 a )) (5.36) 4.
The instantaneous phase values of the k th frequency for the synthesis frames are computed:

φ k (t j s ) = (τ j 0 + (τ i k -τ i 0 ))2πf j k
(5.37) 5. The instantaneous phase curves φ k (t � ) are computed via instantaneous frequency integration:

φ � k (t � ) = φ k (t j s ) + � t � 0 (2πf � k (u) + c(u))du (5.38)
Thus, its time-scaled version, s T S (t � ), for a constant time-scale factor is given by:

ŝT S (t � ) = L � k=-L A � k (t � )e jφ � k (t � )
(5.39)

A pseudocode is presented below for time-scaling with eaQHM in Algorithm 4.

Algorithm 4 Time-scaling using eaQHM

Require: A set of parameters per analysis time instant (voiced frames):

{A k (t i a ), f 0 (t i a ), φ k (t i a )}
Interpolate successive t i a to obtain t � Interpolate A k (t) to obtain A � k (t � ) using Eq. (5.34) Interpolate f k (t) to obtain f � k (t � ) using Eq. (5.35) Estimate φ 0 (t j s ) using Eq. (5.36) Estimate φ k (t j s ) using τ j 0 and Eq. (5.37) Estimate φ � k (t � ) using Eq. (5.38)

Time scaled speech

Synthesize ŝT S (t � ) using sinusoidal synthesis

Example of Application

In Figure 5.6, the eaQHM-based time-scaling is demonstrated for different time-scale factors, 0.5, 1.5, and 2.5. A male speaker waveform is presented. The shape invariance property is clearly demonstrated. In Figure 5.7 the corresponding waveforms of Figure 5.6 is depicted in the frequency domain.

Pitch-Scale Modification Scheme

For an arbitrary pitch-scale modification, the input f k (t) contours are mapped to different ones, f � k (t) = ρ(t)f k (t) in the modified signal, where ρ(t) is the pitch-scale factor function. When ρ(t) > 1, then the pitch increases, whereas the opposite happens when ρ(t) < 1. Note that for a fixed ρ(t) = ρ, the pitch modification is constant throughout the whole waveform. For that, a mapping function referred to as the pitch-scale warping function is defined:

D(t) = � t 0 ρ(τ )dτ
(5.40)

where ρ(τ ) > 0 is the time-varying pitch scaling rate. When ρ(t) > 1, then the pitch increases, whereas the opposite happens when ρ(t) < 1. Note that for a fixed ρ(t) = ρ, the pitch modification is constant throughout the whole waveform.

In an adaptive sinusoidal model context, the parameters should be transformed in the way described next. Let us first assume that in an analysis window centered at t i a , the instantaneous components {A k (t i a ), f k (t i a ), φ k (t i a ), are known, and the component trajectories have been computed, i.e. A k (t), f k (t), φ k (t), within this frame and up to the center of the next frame, t i+1 a . Then, pitch scaling requires the following steps to be performed.

1. In order to compute φ � k (t), it is first necessary to compute the pitch-scaled frequencies. Thus the new frequencies are given by:

f k (t) ← ρf k (t) (5.41)
2. The instantaneous amplitudes at analysis time instants t i a , A � k (t i a ), are computed from sampling the spectral envelope at the corresponding frequencies ρf k :

A � k (t i a ) = DAP (t i a , ρf k ) (5.42)
where DAP (t i a , f ) is the Discrete All-Pole-based envelope constructed around time instant t i a . Then, the k th instantaneous amplitude is linearly interpolated across successive time instants.

3. The instantaneous phase values of the fundamental phase for the synthesis frames are computed:

φ 0 (t j s ) = φ 0 (t j-1 s ) + ρ(φ 0 (t i a ) -φ 0 (t i-1 a )) (5.43) 4.
The instantaneous phase values of the k th frequency for the synthesis frames are computed:

φ k (t j s ) = (τ j 0 + (τ i k -τ i 0 ))2πρf j k (5.44)

The instantaneous phase curves φ �

k (t) are computed via instantaneous frequency integration:

φ � k (t) = φ k (t j s ) + � t 0 (2πρf k (u) + c(u))du (5.45)
Thus, its pitch-scaled version, s P S (t), for a constant pitch-scale factor is given by:

ŝP S (t) = L � k=-L A � k (t)e jφ � k (t)
(5.46)

In algorithmic steps, Algorithm 5 is provided.

Example of Application

In Figure 5.8, the eaQHM pitch shifting is demonstrated for different waveforms and different pitch-scale factors, 0.5, 1.5, and 2.0. A female speaker waveform is presented. In Figure 5.9, the corresponding waveforms of Figures 5.8 are depicted in the frequency domain.

Evaluation and Results

In this section, subjective evaluations on speech modifications using the adaptive models will be presented and compared to the state-of-the-art. Due to their availability, the following methods will be selected for comparison to the aSMs: HNM, STRAIGHT, and WSOLA for time scaling, and HNM and STRAIGHT for pitch-scaling. In this experiment, a database of 32 speech utterances was used, including 16 male and 16 female speakers from 16 different languages: Greek, French, English, Spanish, Finnish, Chinese, Portuguese, Basque, Japanese, Italian, German, Ko-

Time-scaling

The time-scale modification factors were selected to be 0.5, 0.8, 1.2, 1.5, 2.0, and 2.5, which are typical values for moderate speech prosodic modifications. For the HNM, the maximum voiced frequency is fixed to 5500 Hz, and the analysis is pitch synchronous. The analysis window size is two local pitch periods. The order of the AR filter for the noise part is set to 20. The parameters of aHM and eaQHM are the ones described in the previous sections. For the WSOLA, an analysis window length of 15 ms is used. A tolerance variable ∆ (a tolerance factor on the desired timewarping function to ensure signal continuity at segment joins) of 7 ms is selected, which according to [START_REF] Verhelst | An Overlap-Add Technique Based on Waveform Similarity (WSOLA) for High Quality Time-Scale Modification of Speech[END_REF], usually produces high-quality time-scaled speech. For STRAIGHT, the default parameters were used. In general, the participants acknowledged the proposed methods natural. The aHM samples were considered of slightly higher quality than the eaQHM samples, and in general, both better than STRAIGHT. Also, common artefacts, such as "metallic" quality, chorusing, or musical noise do not appear more than in state-of-the-art methods. Although the models are simple, they are shown to perform similarly or even better than the -more complex -HNM or STRAIGHT methods, for time scale modifications, especially in voiced parts of speech, where the well-known problem of lack of presence present in the HNM is addressed. Please note that HNM decomposes speech into a deterministic and a stochastic component, and although it shares the harmonicity assumption in its deterministic component, it handles differently its stochastic part (modulated noise). In our WSOLA samples, a step effect in the amplitude of the time-scaled speech was observed, that led to audible artefacts. No such artefacts were present in the aHM or the eaQHM time-scaled samples. Finally, it should be noted that although WSOLA performs quite close to the adaptive models and is much faster, it is completely inappropriate of providing higher level representations of speech (i.e. spectral envelopes).

Pitch-scaling

For our tests, the pitch-scale modification factors were selected to be 0.5, 0.8, 1.2, 1.5, and 2.0, which are typical values for speech. For both genders of speakers, we have posed a minimum and maximum value for the pitch estimation: f 0(min,max) = (120, 300) Hz for females, and f 0(min,max) = (70, 200) Hz for males. For the HNM, the maximum voiced frequency is fixed to 5500 Hz, and the analysis is pitch synchronous. The analysis window size is set to two local pitch periods. The order of the AR filter for the noise part is set to 20. For the STRAIGHT, default parameters were used, as they are provided by the on-line version of the code. The parameters of the adaptive models are the ones described in the previous sections. In general, first informal listenings acknowledged that common artefacts, such as "metallic" quality, chorusing, or musical noise do not appear in adaptive sinusoidal models more than they do in the state-of-the-art methods in hand. Once again, the aHM samples were considered slightly better than the eaQHM ones. However, for large pitch scale factor and due to the sinusoidal nature of the representations, the spectral area between the distant successive sinusoids manifests a sense of tenseness in voice. This is apparent especially in unvoiced parts, where the number of sinusoids is not high enough to represent these parts well. It should be noted that both HNM and STRAIGHT use some kind of noise component, whereas adaptive models do not. The HNM uses time and frequency modulated noise to represent unvoiced parts and high-frequency components of voiced parts, whereas STRAIGHT uses all-pass filters to compensate for the buzz timbre of minimum-phase vocal tract filter.

Conclusions

In this chapter, speech modifications based on the adaptive Sinusoidal Models were presented. Modifications are governed by simple rules, making them highly attractive. For the aHM, each harmonic track is modified separately, using the notion of relative phase. Time scaling is achieved by interpolating the instantaneous amplitude, frequency, and relative phase tracks over time, and the instantaneous phase is computed by adding the integral of the time scaled instantaneous frequency from the start of the signal back to the interpolated relative phase track. Shape invariant waveforms are generated using this approach. For the eaQHM, since the model is quasi-harmonic, the notion of relative phase delays is employed to produce shape invariant waveforms. The instantaneous amplitude and frequency tracks are interpolated over time and for the instantaneous phase, the relative phase delays at the analysis time instants are forced to be valid at the synthesis time instants. This way, the well-known phase dispersion problem is minimized. In pitch scaling, the same principles are followed, only that the frequency tracks are resampled in new frequency values, and the corresponding amplitudes are obtained from a Discrete-All-Pole based spectral envelope. The instantaneous phases are computed using the relative phase and the relative phase delay for the aHM and the eaQHM, respectively in order to maintain waveform shape. Algorithms and examples are given for each model. Adaptive Sinusoidal Modelling of Musical Instrument Sounds

Introduction

Musical instrument sounds are notoriously challenging to represent with a single model because different instruments may feature radically different characteristics, such as sharp attacks, quasi-periodic (or quasi-stationary) oscillations, noise, and inharmonicity [START_REF] Fletcher | The Physics of Musical Instruments[END_REF]. The attack is the most salient perceptual feature of musical instrument sounds that listeners use in dissimilarity judgments [GG77, SC64, Kru89]. It is well known that much of the characteristic quality of many musical sounds derives from the attack [START_REF] Iverson | Isolating the dynamic attributes of musical timbre[END_REF], although the harmonic structure of the quasi-stationary oscillations (when there is a steady state) is also clearly important [START_REF] Handel | Timbre perception and auditory object identification[END_REF]. Percussive sounds produced by plucking strings (such as harpsichords, harps, and the pizzicato playing technique) or striking percussion instruments (such as drums, idiophones, or the piano) feature a sharp onset with highly nonstationary oscillations that die out very quickly, called transients [START_REF] Bader | Modeling of musical instruments[END_REF]. The reed in woodwind instruments presents a highly nonlinear behavior that also results in attack transients [START_REF] Fletcher | The nonlinear physics of musical instruments[END_REF], while the stiffness of piano strings results in a slightly inharmonic spectrum [START_REF] Bader | Modeling of musical instruments[END_REF].

Ideally, the representation of musical instrument sounds should rely on a few parameters that capture most information in an intuitive way. Additionally, high fidelity is essential in synthesis [START_REF] Quatieri | Audio signal processing based on sinusoidal analysis/synthesis[END_REF], transformation [VZA06, SB98, CR12], model conversion [RVR07, VRR07], and estimation of various features, be they perceptual such as pitch [START_REF] De Cheveigne | YIN, a fundamental frequency estimator for speech and music[END_REF][START_REF] Camacho | A sawtooth waveform inspired pitch estimator for speech and music[END_REF] or acoustical such as size, material, fingering, among others [MIT + 10, Abe13, TS01, MIK + 12]. The musical instrument sound model can be used in a variety of related problems, from onset detection [BDA + 05] and segmentation [PGV97, CR10, PV00] to musical instrument classification [START_REF] Marques | A study of musical instrument classification using gaussian mixture models and support vector machines[END_REF][START_REF] Herrera | Automatic classification of musical instrument sounds[END_REF], recognition [EK00, ERD06], identification [Bro99, KM95], and conversion [START_REF] Giotis | Instrument timbre transformation using gaussian mixture models[END_REF]. Parametric representations typically compare the signal to be represented with a template, such as wavelets [START_REF] Kronland-Martinet | The wavelet transform for analysis, synthesis, and processing of speech and music sounds[END_REF], sinusoids [MQ86, SS90], exponentially damped sinusoids [HVL + 05], or a dictionary of atoms [START_REF] Mallat | Matching pursuits with time-frequency dictionaries[END_REF][START_REF] Sturm | Dark energy in sparse atomic decompositions[END_REF]. In general, sinusoids render a compact representation of quasi-stationary oscillations that is perceptually close to the original recording and whose parameters encode intuitive information. However, the quality of the representation depends on the parameter estimation techniques and on how the underlying model represents temporal variation in the parameters over the course of the sound.

There have been several proposals about how to estimate the parameters of sinusoidal models. The earliest models [MQ86, SS90] relied on straightforward estimation techniques, such as peak-picking and parabolic interpolation. Over the years, researchers have proposed improvements in parameter estimation [START_REF] Nunes | Recursive least-squares estimation of the evolution of partials in sinusoidal analysis[END_REF], partial tracking [LMR07, DGR93], and time-frequency resolution [AF95, [START_REF] Kodera | Analysis of time-varying signals with small bt values. Acoustics, Speech and Signal Processing[END_REF][START_REF] Fulop | Algorithms for computing the time-corrected instantaneous frequency (reassigned) spectrogram, with applications[END_REF]. Nonetheless, there are some intrinsic limitations to the representation of musical instrument sounds by quasi-stationary sinusoids, such as poor noise and transient modeling, smearing of the sharpness of attack, and limited temporal resolution. The standard sinusoidal model (SM) suffers from the time-frequency uncertainty principle when estimating the parameters because long analysis windows blur the temporal resolution to improve the frequency resolution and vice-versa. Moreover, the SM uses constant amplitude and frequency values inside the analysis window because the SM assumes that the sound is relatively stable locally. Consequently, the representation notoriously fails to capture short-time temporal variations such as transients or sharp percussive onsets [START_REF] Caetano | Adaptive sinusoidal modeling of percussive musical instrument sounds[END_REF]. Therefore, the SM tends to use a separate model for noise [SS90, CKD + 13] and for transients [START_REF] Verma | Extending spectral modeling synthesis with transient modeling synthesis[END_REF][START_REF] Daudet | Transients modelling by pruned wavelet trees[END_REF][START_REF] Duxbury | Separation on transient information in musical audio using multiresolution analysis techniques[END_REF][START_REF] Daudet | A review on techniques for the extraction of transients in musical signals[END_REF].

Exponentially Damped Sinusoidal Models (EDS) have been gaining popularity as the template (or basis functions) to represent many types of signals [START_REF] Kumaresan | Estimating the parameters of exponentially damped sinusoids and pole-zero modeling in noise[END_REF][START_REF] Umesh | Estimation of parameters of exponentially damped sinusoids using fast maximum likelihood estimation with application to nmr spectroscopy data[END_REF], including audio and musical instrument sounds [NHD98b, BBD02, HVL + 05] and more recently for audio coding [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF]. Essentially, the EDS uses stationary sinusoids modulated by a real exponential function inside the analysis window to represent the partials. There are parameter estimation algorithms based on iterative analysis-by-synthesis [START_REF] Goodwin | Matching pursuit with damped sinusoids[END_REF] and subspace-based methods [RK89, VHPR96], sometimes referred to as "high-resolution methods" because they do not suffer from the time-frequency uncertainty of the FFT. Proponents of the EDSM to model musical instrument sounds claim that the EDS outperforms the SM [START_REF] Jensen | A comparison of sinusoidal model variants for speech and audio representation[END_REF] not only because the parameter estimation techniques are more powerful and robust, but also because the temporal envelope of the sinusoids is suitable to represent percussive sounds. However, the frequency of the partials in the EDS is still constant inside each analysis frame, and the beginning of the frame has to coincide with the onsets to take advantage of the shape of the temporal envelope. Synchronization of the frame boundaries with percussive onsets requires additional steps prior to modeling, namely onset detection and classification into percussive or not. The EDS notoriously requires additional partials [START_REF] Nieuwenhuijse | Robust exponential modeling of audio signals[END_REF] when the onsets are not synchronized. Another important drawback is the constant frequency of each partial inside the analysis frame.

Adaptive sinusoidal models have been applied in speech [KPRS12, PRS11] and musical instrument sounds [R 06, CKMS13] to address some of the issues with traditional sinusoidal models. Adaptation of sinusoidal partials inside the analysis window frees the algorithm from the inherent temporal limitation of the STFT and quasi-stationary sinusoids, allowing representation of changes in a temporal scale smaller than the hop size. In a preliminary study [START_REF] Caetano | Adaptive sinusoidal modeling of percussive musical instrument sounds[END_REF], we used an adaptive sinusoidal model dubbed the "extended adaptive Quasi-Harmonic Model" (eaQHM) [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF] to represent percussive musical instrument sounds. We showed that the eaQHM outperformed the representation obtained with sinusoidal models that use stationary partials. Previously, the eaQHM had been used to model the speech counterparts of percussive audio sounds, namely stop sounds, outperforming quasi-stationary sinusoidal models [START_REF] Kafentzis | On the Modeling of Voiceless Stop Sounds of Speech using Adaptive Quasi-Harmonic Models[END_REF]. In the eaQHM, adaptation results from the iterative projection of the original waveform onto nonstationary basis functions that are locally adapted to its time-varying characteristics, rendering a flexible model capable of representing sudden changes such as transients or sharp onsets.

In this work, we model a large number of musical instrument sounds from different families with sinusoidal modeling algorithms and compare their modeling accuracy, defined as how much information the parametric model captures quantitatively. We use the "signal to reconstruction error rate" (SRER) as a measure of modeling accuracy. We compare both the local SRER, measured over a window just before the onset, and the global SRER, comprising the whole duration of the sound. The algorithms considered are the standard sinusoidal modeling algorithm [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] (SM), exponentially damped sinusoids [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF] (EDS) using ESPRIT [START_REF] Roy | Esprit-estimation of signal parameters via rotational invariance techniques[END_REF] for parameter estimation, and the eaQHM. First we show that adaptation significantly improves both the local and global SRER. Then we compare the modeling accuracy varying the size of the analysis window and the number of sinusoidal partials. We show that the eaQHM outperforms both the state of the art (EDS) and the baseline model (SM) for most instrumental families in all experiments with with the same number of analysis parameters as EDS and the same number of synthesis parameters as the SM.

In the next section, we describe the analysis and synthesis stages of eaQHM, focusing on adaptation. Then, we present the experimental setup, describe the musical instrument sound database used in this work and the analysis parameters. Next, we explain the experiments we performed, we present the results and evaluate the performance of the SM, EDS, and eaQHM in modeling musical instrument sounds. Finally, we discuss the results and present conclusions and perspectives for future work. The raw data will be available for reproducibility of the results along with Matlab code to obtain the figures and tables presented here. Sound examples and further information can be found at http://www.csd.uoc. gr/ ˜kafentz/listest/pmwiki.php?n=Main.AdaptiveSinMus.

Experimental Setup

The aim of the experiment is to compare the modeling accuracy of the SM, EDS, and eaQHM for a broad range of musical instrument sounds, including percussive and nonpercussive. In this work, modeling accuracy is measured by the local and global SRER, calculated using Eq. (6.1).

SRER = 20 log 10 σ x(t) σ x(t)-x(t) (6.1)
The local SRER is measured over a window just before the onset to evaluate the smearing of the attack also known as pre-echo, a very common artifact among sinusoidal models that use quasi-stationary sinusoids. The global SRER measures the overall modeling accuracy, taking the whole sound into account.

The modeling accuracy depends on the number of partials K and the window size L for the sinusoidal algorithms. Traditionally, partials are not supposed to vary much inside the analysis window and thus are modeled with quasistationary sinusoids whose parameters are averaged over the window, changing between windows according to the step size. On the other hand, supposing that each sinusoid captures one partial, there is a minimum number of sinusoids required to represent the oscillatory energy in musical instrument sounds. Thus we will present a comparison of the SRER as a function of K and L for the SM, EDS, and eaQHM. First we describe the musical instrument sounds modeled and the selection of parameter values for the algorithms.

The Musical Instrument Sounds Used

In total, 90 musical instrument sounds were used in this work1 . Table 6.1 lists the musical instruments divided in six classes, Brass, Woodwinds, Strings, Percussion, Popular, and Keyboard. The recordings were chosen to represent the range of musical instruments commonly found in traditional Western orchestras and in popular recordings. Some instruments feature different registers (alto, baritone, bass, etc) or different keys (pitched in C or B�). All sounds used belong to the same pitch class (C), ranging in pitch height from C3 (f 0 � 131 Hz) to C6 (f 0 � 1046 Hz), but most are C3 or C4. The dynamics of all sounds is forte, while the duration was kept under 2 s. All sound files were edited so the first sample corresponds to the onset. Normal attack ("na") and no vibrato ("nv") were chosen whenever available. Presence of vibrato is indicated ("vib"), as well as different playing modes such as staccato ("stacc"), sforzando ("sforz"), and pizzicato ("pz"), achieved by plucking string instruments. Extended techniques were also included, such as tongue ram ("tr") for the flute and bowing ("bow") idiophones (vibraphone, xylophone, etc). Different materials such as metal, plastic and wood are also indicated (respectively by "me", "pl", and "wo".)

Brass

Analysis Parameters

The parameter estimation for the SM follows [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] with phase interpolation via cubic splines. The estimation of parameters for EDS used here is described in detail elsewhere [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF], while the estimation of the optimum number of poles (sinusoids) [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF] is used for comparison. In all experiments, the threshold for SRER convergence is set to 0.01, the size of the FFT is N = 4096 samples, and the sampling frequency for all sounds is F s = 16kHz. The step size for the SM and eaQHM was H = 1ms (which corresponds to 16 samples). EDS estimates the parameters separately for non-overlapping frames (rectangular window) of size L. For each parameter estimation step, EDS breaks down the frame at L/2 and uses a hop size H = 1 sample when building the Inte matrix [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF]. Prior to modeling with the SM, EDS, and the eaQHM, the fundamental frequency f 0 of all sounds was estimated using SWIPE [START_REF] Camacho | A sawtooth waveform inspired pitch estimator for speech and music[END_REF] because in this work the window size L and the maximum number of sinusoidal partials K max supposing harmonicity depend on f 0 .

The number of (sinusoidal) partials K is an important input parameter which directly affects the modeling accuracy for the SM, EDS, and the eaQHM. In the SM, K dictates how many local maxima (harmonically related or not) the peak picking algorithm will retain. The parameter estimation algorithm for EDS [START_REF] Roy | Esprit-estimation of signal parameters via rotational invariance techniques[END_REF] uses K to determine the separation between the dimension of the signal space and the background noise. In turn, the eaQHM initializes the template signal for QHM (see Section 2.4) with K harmonically related partials. For all the algorithms, we suppose the musical instrument sounds under investigation can be well represented as nearly harmonic, so we set the maximum number of partials K max to the highest harmonic number below Nyquist frequency or equivalently the highest integer K that satisfies Kf 0 ≤ F s /2.

The window size L also directly affects the modeling accuracy of the SM, EDS, and the eaQHM. In the STFT, L determines the well known trade-off between temporal and spectral resolution which, in turn, directly affects the performance of the peak picking algorithm that the SM uses for parameter estimation. Moreover, the parameters estimated are averaged across the length L of the window and used to represent the center of the window. EDS estimates stationary (damped) sinusoids inside these frames, thus L limits the temporal modeling accuracy. Finally, the eaQHM uses SM-like frames and captures variations inside the analysis window. In this case, varying L does not directly affect temporal or spectral resolution because the eaQHM uses least squares (QHM) in the time domain to estimate the parameters. However, L will impact the modeling accuracy of the amplitude and frequency modulations inside the window. In the literature [START_REF] Rabiner | Digital Processing of Speech Signals[END_REF], L = 3T 0 (where T 0 = 1/f 0 is the fundamental period) is considered a reasonable value for speech and audio when using the SM. However, we are unaware of a systematic investigation of how L affects modeling accuracy for EDS. Next, we present the investigation on modeling accuracy (local and global SRER) as a function of the number of adaptations, K and L.

Adaptation Cycles

Figure 6.1 shows the global and local SRER as a function of the number of adaptation cycles (iterations). Each plot was averaged across the sounds indicated, while the plot "all instruments" is an average of the previously shown. Notice how the SRER increases quickly after a few iterations, slowly converging to a final value several orders of magnitude higher than before adaptation.

Number of Partials K

We studied the impact of K in the modeling accuracy of the SM, EDS, and eaQHM. We ran each algorithm with different numbers of partials as input parameter (the window size was kept at L = 3T 0 ) and recorded the resulting local and global SRER values. We started from K max and decreased K by 2 partials each run. We expected the SM to quickly converge to a maximum value and stabilize because of the parameter selection algorithm. The literature on EDS [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF] suggests that there is an optimum number of partials for audio, thus we expected EDS to render a curve that would reach a maximum around that point and then decrease. Finally, adaptation allows the eaQHM to represent small temporal variations such as transients accurately as modulations of amplitude and frequency of the partials (no matter the number of partials). Therefore, we expected the eaQHM to yield higher values of SRER as the number of partials increased. Figures a andb shows the local and global SRER as a function of K for the SM, EDS, and the eaQHM. All curves are averaged across the sounds from the musical instruments indicated. Note that sounds with different f 0 values have different maximum number of partials K max .

Window Size L

We investigated the impact of L in modeling accuracy for the SM, EDS, and the eaQHM. We ran each algorithm varying L from 3T 0 to 8T 0 with constant number of partials K max and measured the resulting local and global SRER. We expected L to negatively impact all three algorithms differently. We expected L to have a greater impact on the SM because both parameter estimation and temporal resolution depend on L. We expected L to have a smaller impact on modeling accuracy for EDS because of the time-varying amplitude of the locally stationary sinusoids (despite the constant frequency value inside the window.) Finally, we conjectured that L will have a minor effect on the eaQHM because L mostly affects the eaQHM's ability to capture amplitude and frequency modulations inside the window. Figures c andd illustrate the results, showing the SRER as a function of L expressed as times T 0 , so sounds with different f 0 values have different window size L in samples. All curves are averaged across the sounds from the musical instruments indicated.

However, Figure 6.2 is not enough evidence that the eaQHM outperforms the SM and EDS in average for all musical instrument sounds investigated. In what follows, we will compare the average modeling accuracy of the SM, EDS, and eaQHM using the curves from Figure 6.2 and defining the "mean SRER difference." For each musical instrument sound, we subtracted point by point the SRER values (in dB) corresponding to the SM and EDS from that of the eaQHM and averaged the result (across L or K). A positive "mean SRER difference" represents how much eaQHM outperforms the other method in average for that particular musical instrument, while a negative value means eaQHM was outperformed.

Analysis of Results

This section presents a systematic analysis of the experiments introduced in the previous section consisting of a comparison across musical instruments for all methods. Next, we present these mean SRER differences for all musical instrument sounds clustered by musical instrument family in table 6.2 and table 6.3. only for the maximum number of partials. The Local column is especially important to evaluate the attack since the SRER does not include temporal information, the Global column assesses overall performance across K. The algorithms perform best at K max , so this column will be used as reference measure of modeling accuracy per musical instrument family. The Total row will be used as reference measure of modeling accuracy per sinusoidal modeling algorithm. Hence the "mean SRER difference" in the K max column and Total row gives the final modeling accuracy performance.

Variation Across

K Holding L = 3T 0 Table 6.
Locally, the eaQHM is only outperformed by EDS for Bowed Percussion, and globally for Struck Percussion and Popular. In turn, the SM only outperforms the eaQHM globally for Popular. Finally, the eaQHM presents a higher modeling accuracy than both SM and EDS with K max . Both locally and globally, the eaQHM achieves the highest performance for Plucked Strings, outperforming EDS by over 10dB and SM by over 20dB. However, the column K max reveals that the eaQHM outperforms EDS by more than 20dB for Bowed Strings and Bowed Percussion, while outperforming EDS by around 15dB for Plucked Strings. The row Total reveals that the eaQHM outperforms EDS and SM locally, globally, and for K max in average when all musical instruments are clustered together.

Variation Across L Holding K = K max

Similarly to the previous section, table 6.3 shows the mean SRER difference between eaQHM and EDS and eaQHM and SM. The Local and Global columns present the mean SRER difference across L, while the 3T 0 column shows the difference in global SRER for the window size that gives the best modeling accuracy for all algorithms. The local SRER is used to evaluate onset modeling accuracy and global SRER evaluates general performance. The column 3T 0 will be used as reference measure per musical instrument family and the Total row as reference per model.

Under variation of L, the eaQHM only outperforms EDS locally for Bowed Strings and Struck Percussion. However, the eaQHM is only outperformed globally by EDS for Plucked Strings and Keyboard. Once again the eaQHM presented a consistently higher performance than EDS for 3T 0 . Note that the eaQHM outperforms the SM for every musical instrument cluster considered locally, globally, and for 3T 0 . When compared against EDS, eaQHM achieves the highest performance locally for Struck Percussion, globally for Brass, and using window size 3T 0 for Bowed Strings. However, when we compare the performance with the SM, Plucked Strings is the highest performing cluster locally, globally, and for 3T 0 . Notice that the comparison across L with EDS is not consistent, while the comparison with the SM is much more consistent. This phenomenon will be analyzed in the following section.

SRER(eaQHM-EDS) SRER(eaQHM-SM) Local Global

Discussion

Figure 6.1 shows the impact of adaptation on modeling accuracy for the eaQHM, revealing that adaptation does lead to a significant improvement in modeling accuracy as measured by both local and global SRER. Figure 6.2 illustrates the behavior of the SM, EDS, and eaQHM under variation of the number of partials K and window size L, respectively. In general, there is no significant difference in local and global behavior for the curves, while the global SRER presents higher absolute values. The tendency for higher absolute global SRER seems natural for Figure 6.2 because the global SRER uses information from the whole sound duration, while the local SRER reflects the fit of the models right before the beginning of the sound. Figures 6.2a and 6.2b show that the SM behaves as expected when varying the number of partials, not showing significant improvement in modeling accuracy after a certain number of partials. shows that the global values are higher for the same sounds, and the addition of partials tends to increase the global SRER more than the local SRER. This trend confirms that the SM has a tendency to fit onsets poorly and stable partials significantly better. Interestingly, EDS does not behave as predicted theoretically in the literature [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF]. Instead of reaching a maximum performance for a certain number of partials and decreasing after that, EDS presents a general trend to continuously improve the modeling accuracy when the number of partials increases. In fact, the optimum number of partials obtained with ESTER [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF] was much higher than the maximum number of partials K max for all musical instrument sounds. Moreover, the difference in absolute value between local and global SRER for EDS is less significant than for the SM or the eaQHM, revealing tha EDS does not improve the fit with the presence of more information. Finally, the eaQHM behaved as expected, improving the modeling accuracy with more partials. In general, the absolute value for the global SRER is higher than for the local fit, but the difference between global and local is smaller than for the SM, indicating that the eaQHM presents a more consistent modeling performance throughout. In turn, Figures 6.2c and 6.2d show that the modeling accuracy decreases with L for all algorithms. Once again the local and global SRER curves show a consistent behavior for each model, with the same general tendency for higher global than local SRER values for the same sounds. Interestingly, the window size affects the SM much less than EDS or the eaQHM, suggesting that the SM is more robust to variations of L than EDS or the eaQHM. On the other hand, it is depicted that the SM seldom outperforms the others (apparently, only when the performance of EDS and the eaQHM is compromised by very large values of L.) Again, the SM presents a higher difference between global and local SRER than EDS or the eaQHM. Figures 6.2c and 6.2d reveal that variation across L affected the performance of EDS significantly less than that of the eaQHM and the SM because of the different impact that L has in the eaQHM and SM against EDS. L is the length of the non-overlapping slice that EDS models. Internally, the effective length used by EDS to fit the parameters of the model is L/2 with a step size of 1 sample. Here we should notice that L more greatly affects the difference between global and local SRER for EDS than the number of partials. Not surprisingly, the modeling accuracy of the eaQHM suffers when L increases. In some cases, the general performance is still superior throughout (see Fig. (6.2c) Brass, Woodwinds, Bowed strings, and Popular). The difference between global and local SRER in absolute value for L is more significant than for K regarding the eaQHM. In the eaQHM, L affects frequency correction and interpolation mechanisms. Long windows have an averaging effect because the parameters are iteratively fit in the time domain. Frequency correction is applied an the center of the analysis window and the eaQHM uses interpolation to capture frequency modulations between windows. Thus, adaptation improves the fit more slowly for longer L, generally reaching a lower "roof" SRER value in fewer iterations.

Analysis and Synthesis Complexity

Here, complexity is considered as the number of parameters (or degrees of freedom) required to estimate and to represent each sinusoid. Table 6.4 shows a comparison for the SM, EDS, and eaQHM using real numbers because some parameters are complex. As is widely known, the SM requires the estimation of three real parameters for each sinusoid, namely the amplitude a k , the phase φ k , and the frequency f k . In turn, EDS requires the estimation of two complex parameters [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF], amplitudes a k , and poles z k . These are converted into the four real parameters amplitude a k , phase φ k , frequency f k , and damping coefficient δ k (see [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF] for details). Finally, the eaQHM estimates two complex amplitude parameters a k and b k . The frequencies f k are initialized as integer multiples of a fundamental frequency f 0 and later corrected by the estimations a k and b k , so f k is not directly estimated by the eaQHM. Thus the eaQHM and EDS present a higher analysis complexity than the SM. Table 6.4 also shows the number of parameters for the synthesis stage. Notice that the eaQHM has the same synthesis complexity as the SM, while both the SM and EDS need all analysis parameters also in the synthesis stage. The synthesis complexity of the eaQHM is lower than the analysis stage because the synthesis stage is essentially the same as the SM.

Real numbers per sinusoid per frame

SM EDS eaQHM Analysis a k , φ k , f k �{a k }, �{a k }, �{z k }, �{z k } �{a k }, �{a k }, �{b k }, �{b k } Synthesis a k , φ k , f k |a k |, φ k , f k , δ k |a k |, φ k , f k

Modeling Accuracy and SRER

In this work, modeling accuracy is the ability of a model to capture information from a signal. The SRER defined in Eq. ( 6.1) is the ratio in dB of the energy in the original signal x (t) and the modeling error or residual x (t). As such, the aim of maximizing the SRER is equivalent to minimizing the residual energy and therefore minimizing the information missed by the model. Here, we consider that some of the noise might be intrinsic to the musical instrument sound being modeled, such as breathing noise, and should be captured as well. Therefore, technically, the SRER does not measure the same as the more commonly known signal-to-noise ratio (SNR) because the SNR usually considers additive noise from an external source whose statistical properties differ from the signal's, such as background noise from quantization or transmission. Thus the aim of minimizing the residual energy in sinusoidal modeling can be interpreted as capturing as much signal energy as possible with sinusoids. Ideally, the sinusoids should capture all oscillatory energy (including transients) and leave a noisy residual with a flat spectrum, indicating that the residual is indeed statistically independent from the model.

An important feature of the SRER is that it uses the waveforms to estimate the energy, comparing x (t) and x (t) directly. Consequently, x (t) will only have low energy when x (t) follows x (t) closely. However, the SRER is blind to where the differences lie in the waveform. For discrete waveforms, if x (t) as an identical copy of x (t) except for a single sample, the energy in x (t) results in a particular value of SRER. However, a different waveform x2 (t) created by adding a small perturbation to each sample in x (t) could have the same SRER. Consequently, the SRER alone does not show that x1 (t) and x2 (t) are different waveforms. So, when comparing two models x1 (t) and x2 (t), it is only safe to say that a higher SRER indicates better modeling accuracy, which in turn suggests a better quality representation.

Percussive Musical Instruments

The musical instrument clusters that contain percussive sounds in this work are Plucked strings, Struck percussion, and Keyboard. The literature [NHD98b, BBD02, HVL + 05] proposes that EDS is particularly suitable to model percussive sounds because of the exponential temporal envelope, commonly claiming that the parameter estimation technique [START_REF] Roy | Esprit-estimation of signal parameters via rotational invariance techniques[END_REF] uses "high-resolution methods" that outperform traditional estimation methods based on Fourier analysis. However, the ability to adapt the amplitude of the sinusoidal partials to the local characteristics of the waveform makes the eaQHM extremely flexible to fit both percussive and nonpercussive musical instrument sounds. For example, table 6.2 shows that the eaQHM outperformed EDS the most for Plucked Strings, a percussive sound that EDS should supposedly capture well due to the temporal envelope. Upon close examination, table 6.2 reveals that the eaQHM in general outperforms EDS in local, global, and K max for the percussive instruments. Table 6.3, on the other hand, shows better performance for the eaQHM than EDS only for 3T 0 for these particular clusters.

In general terms, the experiment revealed that the eaQHM has a better modeling accuracy than EDS and SM in average across K, with a robust performance throughout. Table 6.2 shows that the eaQHM consistently achieved better modeling accuracy than EDS and SM for K max . In some cases, the eaQHM significantly outperforms EDS, while the total performance was 14.14 dB. Table 6.3 also reveals that the eaQHM outperforms EDS for 3T 0 for all clusters of musical instruments with some significant differences of more than 20 dB, while comparison with the SM shows significant differences across all clusters. The total row indicates that the eaQHM outperforms the EDS in average by 16.55 dB and the SM by 31.66 dB.

Conclusion and Perspectives

Musical instrument sounds are challenging to represent accurately because different musical instruments may feature radically different characteristics, such as sharp onsets, attack transients, inharmonicity, or mechanical noise. The quality of the representation depends not only on accurate parameter estimation, but also on how the underlying model uses this information to capture and represent temporal variations of the model parameters. This work proposed to use an adaptive Sinusoidal Model dubbed eaQHM to represent percussive and nonpercussive musical instrument sounds as sinusoids modulated in amplitude and frequency. In general, the eaQHM renders a compact yet high-quality representation with intuitive parameters. The model represents well sharp onsets with attack transients, inharmonic spectra, and even mechanical noise.

We showed that adaptation of the sinusoids inside the analysis window allows the eaQHM to significantly increase the accuracy of representation without the need to increase model complexity. We used the signal to reconstruction error ratio (SRER) to compare the modeling accuracy of the eaQHM with exponentially damped sinusoids (EDS), considered here to be the state of the art, and the standard sinusoidal model (SM) as the baseline representation of a database of 90 percussive and nonpercussive musical instruments sounds. The experiments measured the local and global SRER as a function of the number of partials and size of the analysis window for the the SM, EDS, and eaQHM. The local SRER is measured just before the onset to capture potential artifacts in attack transient modeling, and the global SRER measures the fit for the whole waveform to evaluate general modeling performance. The results showed that the eaQHM outperforms EDS and SM in average across both variations in all cases except local SRER for the EDS under variation of window size. Considering only the window size and number of partials for which the algorithms perform their best, the eaQHM consistently outperformed EDS by more than 10dB and SM by over 25dB in average.

Adaptive sinusoidal modeling can be used in parametric audio coding with low bit rates because the very high modeling accuracy potentially gives high fidelity with the same model complexity as the standard sinusoidal model. However, the eaQHM currently only handles monophonic sounds (speech or musical instruments). An important perspective of this work is to develop an adaptive algorithm for polyphonic audio. Therefore, future work should focus on improving the robustness of parameter estimation first. Presently, the use of least squares is costly and unstable, failing whenever the frequencies of two partials are closer together than a threshold value. We envision total least squares or singular value decomposition as good candidates to improve the robustness in parameter estimation and allow modeling of polyphonic audio.

Residual Analysis

Introduction

In essence, sinusoidal analysis models each partial with a time-varying sinusoid, capturing temporal variations in amplitude, frequency and phase (the parameters of the model). Sinusoidal modeling is considered to represent musical instrument sounds well because most musical instruments are designed to present very clear modes of vibration. However, there is noise present in virtually all musical instrument sounds, such as breathing noise in woodwinds or mechanical noise like the hammer striking the piano strings.

There have been improvements in sinusoidal modelling to address issues such as partial tracking [NMB07, LMR07, DGR93], transient modelling [START_REF] Verma | Extending spectral modeling synthesis with transient modeling synthesis[END_REF][START_REF] Levine | A Sines+Transients+Noise Audio Representation for Data Compression and Time/Pitch Scale Modications[END_REF], to augment the accuracy of parameter estimation as well as the temporal resolution by adapting partials trajectories inside the analysis window [START_REF] Degottex | A full-band adaptive harmonic representation of speech[END_REF][START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]. Nevertheless, the lack of noise is perceptually noticeable in the sinusoidal representation of musical instrument sounds [START_REF] Goodwin | Residual modeling in music analysis-synthesis[END_REF][START_REF] Ding | Processing of musical tones using a combined quadratic polynomial-phase sinusoid and residual (quasar) signal model[END_REF]. Serra [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF] proposed to subtract the sinusoidal component (i.e., the result of sinusoidal analysis) from the original recording to estimate a "residual component". This residual is, by definition, whatever is left from sinusoidal modeling, and therefore, commonly assumed to be noise not captured by the sinusoidal model (usually because sinusoids are not a compact representation of noise). Considerably less effort has been made in residual modeling. It has become standard practice [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF][START_REF] Goodwin | Residual modeling in music analysis-synthesis[END_REF][START_REF] Ding | Processing of musical tones using a combined quadratic polynomial-phase sinusoid and residual (quasar) signal model[END_REF] to model the residual component by filtering white noise with a time-varying filter that emulates the spectral characteristics of the residual signal. Naturally, there are different ways to model the spectral distribution of energy of the residual component. The basic assumption is that the residual signal does not contain perceptually relevant information in the phase spectrum, only in magnitude. Therefore, "psychoacoustic" filter banks are usually found in residual modelling [START_REF] Levine | A Sines+Transients+Noise Audio Representation for Data Compression and Time/Pitch Scale Modications[END_REF][START_REF] Goodwin | Residual modeling in music analysis-synthesis[END_REF]. Goodwin [Goo96] uses the short-time energy in equivalent rectangular bands (ERBs) of the magnitude spectrum for both the analysis and synthesis stage, and justifies stating that the ear is insensitive to energy distributions within each ERB. Levine [START_REF] Levine | A Sines+Transients+Noise Audio Representation for Data Compression and Time/Pitch Scale Modications[END_REF] uses Bark bands instead. Resynthesis commonly uses a piece-wise constant spectrum with magnitudes from the ERB (or Bark bands) energy and random phase. Goodwin remarks that temporal phase correlations can control the texture of the modelled residual, which has been studied further to synthesize environmental sounds (e.g., running water or crackling fire) [START_REF] Athineos | Sound texture modelling with linear prediction in both time and frequency domains[END_REF]. Ding [START_REF] Ding | Processing of musical tones using a combined quadratic polynomial-phase sinusoid and residual (quasar) signal model[END_REF] proposes to use multi-pulse excitation linear prediction (MPLP) to keep phase coherence with the sinusoidal component.

There have been no formal investigations on the filtered white noise model for residual from sinusoidal modelling of musical instrument sounds. In this work we present a systematic evaluation of how well filtered white noise models the residual from sinusoidal modelling of musical instrument sounds for different sinusoidal modelling algorithms. Each algorithm captures oscillatory behaviour differently and, consequently, leaves (perceptually) different residuals. We performed a subjective listening test to evaluate the perceptual similarity between filtered white noise and the residual of each sinusoidal algorithm. Then we use an objective measure of similarity to compare with the perceptual assessments. The next section briefly reviews the sinusoidal modelling algorithms used in this investigation. Next, we describe the framework used to decompose the musical instrument sounds into the blocks used in the evaluation, which is followed by a discussion and the conclusions and future perspectives.

Sinusoidal Modelling

Conceptually, traditional sinusoidal modelling supposes that the musical instrument sounds being modelled can be decomposed into quasi-harmonic oscillations and additive noise. In practice, the musical instrument sound y (t) is separated into a sinusoidal component y s (t) plus a residual component y r (t), where y r (t) is obtained by subtraction of the purely sinusoidal component y s (t) from the original sound y (t). The sinusoidal component is further represented as

y s (t) = � K � k=0 α k e j2πtf k � w (t) (6.2)
where α k and φ k (t) = 2πtf k are respectively the amplitude and phase of the k th sinusoid inside the analysis window w (t), and K is the number of sinusoids. The model assumes that the sinusoids describe stable partials of the sound so their parameters do not vary significantly inside the analysis window. Traditionally [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF], the parameters of the model α k and φ k (t) are estimated for each frame of the short-time Fourier transform, limiting the temporal resolution of the model to that of the STFT. In the rest of the text, SM stands for a sinusoidal model that imposes no restrictions on the frequencies of the partials [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF]. For most musical instrument sounds, a model where the sinusoids are harmonically related is a good approximation, giving rise to the Harmonic Model (HM) [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], which uses sinusoids whose frequencies are multiple integers k of a fundamental frequency f 0 as φ k (t) = 2πtkf 0 .

There have been proposals to improve the temporal resolution of the sinusoidal model by adapting the estimation of the parameters of the sinusoids inside the analysis window, resulting in adaptive sinusoidal models. In particular, the adaptive Harmonic Model (aHM) [START_REF] Degottex | A full-band adaptive harmonic representation of speech[END_REF] used in this work modulates the frequency of each sinusoid inside the analysis window upon resynthesis. Recently, the extended adaptive Quasi-Harmonic Model (eaQHM) was developed [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]. The eaQHM algorithm adapts both the amplitudes and frequencies of the sinusoidal partials inside the analysis window, therefore it can be considered a full AM/FM model, as shown below

y s (t) = � K � k=0 α k (t) e jφ k (t) � w (t) , (6.3)
where α k (t) denotes the time-varying amplitude and φ k (t) denotes the instantaneous phase function of the k th component inside the analysis window w (t). Table 6.5 summarizes the temporal representation of frequencies for the analysis and synthesis stages for the sinusoidal algorithms used.

Analysis

Residual Modelling

The residual component y r (t) is modeled as

ŷr (t) = � t 0 a (t -τ ) u (τ ) dτ (6.4)
where ŷr (t) is the modeled residual component, u (τ ) is white noise and a (t, τ ) is the response of a time-varying filter.

Serra [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF] wrote that "a stochastic, or noise, signal is fully described by its power spectral density which gives the expected signal power versus frequency. When a signal is assumed stochastic, it is not necessary to preserve either the instantaneous phase or the exact magnitude details of individual FFT frames," justifying the assumption that the residual component can be modeled as filtered white noise. There have been different proposals to estimate the filter a (τ ) [START_REF] Serra | Musical sound modeling with sinusoids plus noise[END_REF][START_REF] Goodwin | Residual modeling in music analysis-synthesis[END_REF][START_REF] Ding | Processing of musical tones using a combined quadratic polynomial-phase sinusoid and residual (quasar) signal model[END_REF]. In this work, we estimate the spectral envelope of each frame of the STFT of the residual component y r (t) using linear prediction (LPC) [START_REF] Makhoul | Linear Prediction: A Tutorial Review[END_REF] and use it as the time-varying filter coefficients, as has been previously proposed for speech [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. LPC is adequate for spectral envelope estimation of y r (t) because it tends to follow the average energy of noisy spectra rather than the peaks. Using Eq. ( 6.4) the model suposes that if we inverse filter y r (t), we should obtain white noise (a signal with flat magnitude and no temporal phase coherence or random phase). In this work, we investigate if filtered white noise is perceptually close to the original residual signals with a listening test and further investigate if the inverse filtered residual component presents the characteristics of white noise with an objective measure based on the autocorrelation function.

Experimental Framework

Figure 6.3 illustrates the steps of the experimental framework. Each musical instrument sound y (t) is decomposed into sinusoidal y s (t) and residual y r (t) using the SM, the HM, the aHM, and the eaQHM. Each component, y s (t) and y r (t), is modeled with linear prediction, resulting in a time-varying spectral envelope A s (z) and A s (z) and an inverse filtered (whitened) signal ȳs (t) and ȳr (t), which are the prediction errors [START_REF] Makhoul | Linear Prediction: A Tutorial Review[END_REF]. In the listening test, we use white noise filtered with A r (z). The objective similarity measure compares ȳr (t) with ȳs (t) and u (t). 6.6 lists the 14 musical instruments used. The pitch of all sounds is C3 � 131 Hz, the dynamics is forte, and the duration is under 2s. All sinusoidal algorithms used a window size equivalent to 3 times the period of the fundamental frequency f 0 � 131 Hz, 50% overlap, and size of the FFT 4 times the window size. The linear prediction order used was 50 for both y s (t) and y r (t) to avoid smearing possible oscillatory energy left in y r (t) (missed by the sinusoidal model).

Strings

Evaluation

The evaluation consists of a listening test and an objective measure based on the autocorrelation function. However, firstly we estimate the residual energy to compare how well each sinusoidal algorithm models the musical instrument sounds. The less residual energy, the better the algorithm captured the oscillatory behavior. The signal to reconstruction error ratio (SRER) shown in Eq. (6.5) measures the ratio between the total energy and the energy in the residual component y r (t). The higher the ratio, the less residual energy there is in y r (t). SRER = 20 log 10 σ y(t) σ yr(t) (

where y (t) is the original signal, σ( ) is the standard deviation operator, and y r (t) is the residual component. Table 6.7 shows the average SRER in dB across musical instrument sounds for each method, revealing that the eaQHM has a higher SRER than all other methods by roughly 15 dB.

SRER (dB)

Listening Test

The purpose of the listening test is to evaluate the perceptual similarity between the residual signal y r (t) and its filtered-white-noise counterpart ŷr (t) for the the 14 musical instrument sounds listed in Table 6.6 modeled with the four sinusoidal algorithms shown in Table 6.5. For each participant, the listening test presented a subset of 16 pairs of sounds corresponding to y r (t) and ŷr (t) from 4 musical instruments (times 4 algorithms) in random order to minimize cross comparison among methods. All sounds were normalized at -16dB RMS. The listener is instructed to listen to each pair as many time as they want and rate their perceptual similarity in a scale from 1 to 5 labeled with the terms 1) Very different, 2) Different, 3) Fairly similar, 4) Very similar, 5) Identical. The test can be found on-line2 . Figure 6.4 shows the result for 51 participants aged between 22 and 67, depicting the mean opinion score (MOS) and 95% confidence interval. In average, the eaQHM results in a residual signal that was considered between fairly similar and very similar to its filtered white noise counterpart. The other 3 algorithms (SM, HM, and aHM) produced residuals whose filtered white noise counterparts were considered practically different.

Objective Measure

The result of the listening test indicates that, in general, filtered white noise was not considered a perceptually similar representation of y r (t). However, the listening test gives no further evidence to help explain why. Ideally, we would like to identify what remains in the residual signal that departs from the conceptual filtered-white-noise hypothesis. In the listening test, the perceptual effect of the LPC spectral envelope of y r (t) is present in ŷr (t). Thus we assume that the differences lie elsewhere, in the spectral fine structure or in the phase spectrum. To evaluate the importance of the fine structure between y r (t) and ŷr (t), we compare the whitened residual component ȳr (t) with the whitened sinusoidal component ȳs (t) and with the model (i.e., white noise u (t)) with an objective similarity measure. We use the autocorrelation functions, shown in (6.6), which should provide a unique representation of both the white noise (zero except at zero lag) and the sinusoidal component (peaks at multiple integers of the fundamental frequency).

R (i) = N -1-i � n=0 y (n) y (n -i) (6.6)
The similarity measure is then the dot product between the autocorrelation functions, given by cos (Θ {ȳ r , u}) = R ȳr (i) • R u (i) and cos (Θ {ȳ r , ȳs }) = R ȳr (i) • R ȳs (i). The dot (or inner) product can be interpreted as the projection of R ȳr (i) onto R ȳs (i) and R u (i). Thus Θ is the angle between the autocorrelation functions interpreted as vectors, and it varies from 0 (identical) to 90 • (orthogonal). Table 6.8 shows the average of these values across all musical instruments 6.8: Average angle in degrees across musical instrument sounds for each algorithm.

Discussion

The extended adaptive Quasi-Harmonic Model (eaQHM) is tested to confront the notorious pre-echo effect in sinusoidal modelling and it is shown that highly accurate, pre-echo-free representations of percussive sounds are possible using the adaptive approach. Results on a database of percussive sounds such as plucked strings and percussion instruments show that, on average, the eaQHM improves by over 30 dB the Signal to Reconstruction Error Ratio (SRER) obtained by the standard sinusoidal model. A listening test showed that the percussive sounds modelled by the eaQHM are perceptually closer to the original recordings than the same sounds represented by a traditional sinusoidal model for more than 80% of the listeners in all cases.

We also notice that each sinusoidal modelling algorithm resulted in a different perceptual similarity, revealing that different algorithms leave different undesired information in the residual signal y r . Therefore we suspect that there might be some oscillatory behaviour left in y r . In other words, some sinusoidal modelling algorithms fail to capture all oscillatory energy such as frequency modulations or transients. The models that use slowly varying sinusoids (stable oscillations) plus additive noise might oversimplify the complexity of musical sounds. It has already been remarked [START_REF] Levine | A Sines+Transients+Noise Audio Representation for Data Compression and Time/Pitch Scale Modications[END_REF] that sinusoids plus noise plus transients might be a more realistic representation for musical instrument sounds. However, transients are characteristically present mostly during the attack, but there is no indication that the participants used the attack as perceptual cue. The listening test shows that the AM/FM modelling of the eaQHM captures most oscillatory energy, including transients.

On the other hand, Table 6.8 reveals no significant difference across algorithms. The angles Θ do indicate that ȳr is closer to u (white noise) than to ȳs (sinusoidal) for all algorithms. But the similarities measured by Θ do no explain the results of the listening test. Our interpretation of this result is that the perceptual differences found in the listening test cannot be explained by fine spectral structure, rather, by phase coherence or transients.

Interestingly, one of the participants of the listening test remarked that, for each pair, one of them always sounded brighter. Indeed, ŷr has more energy in high frequencies because pure white noise has a flat spectrum where energy is not equal per octave (let alone per ERB or Bark band). A possible course of investigation would be to use different types of noise (prior to applying the time-varying spectral envelope) to correctly balance the spectral energy, such as pink noise. Future perspectives also include using the eaQHM in transient detection and transient modelling for musical instrument recognition, segmentation, and sound transformations such as timbral variations, perceptually coherent time stretching and pitch shifting.

Conclusions and Future Perspectives

We presented a systematic investigation of the filtered white noise model for the residual from sinusoidal modelling of musical instrument sounds. Four different sinusoidal modelling algorithms were evaluated. We conducted a listening test and we developed an objective measure of spectral similarity. The listening test assessed the perceptual similarity between filtered white noise and the residual component for each sinusoidal algorithm. The results indicate that, in general, filtered white noise was considered different from the residual component. However, we determined that the eaQHM leaves a residual that is fairly similar to the filtered white noise counterpart. The objective measure compared the residual with both the sinusoidal component and their modelled counterpart across algorithms using the autocorrelation functions. The objective evaluation aimed to investigate the reason for the result of the listening test, trying to indicate whether there was "sinusoidal" energy left in the poorly modelled residuals, for example. The objective similarity measure did not indicate that the perceptual differences found can be explained by comparing spectral fine structure. However, the autocorrelation function only includes information from the power spectral density. Thus we suspect that the differences lie in the phase spectrum (possibly due to temporal phase coherence) or transients in the residual, confirming the conclusion of previous studies [START_REF] Goodwin | Residual modeling in music analysis-synthesis[END_REF][START_REF] Levine | A Sines+Transients+Noise Audio Representation for Data Compression and Time/Pitch Scale Modications[END_REF].

Future perspectives include using the eaQHM in transient detection and transient modelling for musical instrument recognition, segmentation, and sound transformations such as timbral variations, perceptually coherent time stretching and pitch shifting. Moreover, we should focus on determining the reason for the difference between the conceptual model of filtered white noise and what current sinusoidal modelling algorithms fail to model. The EDS model might be useful in such an analysis, and is planned to be included in the tests. Perspectives include using "colored" noise to correct the high-frequency energy content perceived as brightness (or some other more sophisticated psychoacoustic model). Further investigation on the temporal phase coherence should develop a measure for analysis and comparison with the sinusoidal component. Attack transients might account for some of the perceptual difference we found for most sinusoidal algorithms.

Chapter 7

Expressive Speech Analysis and Classification

Introduction

Emotional (or stressed) speech can be defined as the speech style produced by an emotionally charged speaker. Such speech styles can be characterized as happy, sad, angry, neutral and fearful speech, among others. Analysis of emotional speech could provide information about the emotional state of the speaker, which can be useful in applications such as health care and emergency conditions, and is a necessary pre-processing step in applications such as recognition and classification. Also, speaker recognition and verification systems could benefit from such an analysis, as well as speech synthesis applications, like unit selection based text-to-speech synthesis or HMM-based speech synthesis.

Numerous approaches have been suggested in the literature in order to show the variation of speech characteristics among different emotion conditions. These variations can form features that are exploited to identify and/or classify different emotional speech styles [START_REF] Bou-Ghazale | A comparative study of traditional and newly proposed features for recognitionof speech under stress[END_REF]. Womack and Hansen discussed the use of Linear Prediction (LP) coefficients and cepstral features in analyzing and classifying stressed speech [DH99, HW96, HWA94, WH95]. Zhou et al [START_REF] Zhou | Nonlinear feature based classification of speech under stress[END_REF] have shown that the Teager operator can be used to obtain better results compared to LP-based features in classification of stressed speech. Moreover, it has been suggested that features related to the pitch mean and variance, as well as intensity features, are useful for discrimination among speaking styles [START_REF] Amir | Toward an automatic classification of emotions in speech[END_REF][START_REF] Bulut | On the robustness of overall f0-only modifications to the perception of emotions in speech[END_REF]. Cummings et al [START_REF] Cummings | Estimation and comparison of the glottal source waveform across stress styles using glottal inverse filtering[END_REF] have shown that the glottal pulse shape varies with different stressed conditions. Ruiz et al [RAH + 96] discussed time and frequency related variabilities in stressed speech, whereas Castellanos et al [START_REF] Castellanos | An analysis of general acoustic -phonetic features for spanish speech produced with lombard effect[END_REF] provided an analysis of general acousticphonetic features in Lombard speech. Scherer [START_REF] Scherer | Vocal communication of emotion: A review of research paradigms[END_REF] investigated the intensity, duration, and spectral envelopes in stressed speech for speech and speaker recognition, whereas Bosch [START_REF] Bosch | Emotions, speech, and the ASR framework[END_REF] has discussed the importance of prosody for emotion recognition in speech. Ramamohan and Dandapat [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF] suggested the use of a sinusoidal model (SM) to distinguish between different speaking styles, using its parameters (amplitude, frequency, phase) as features. For the recognition and/or classification of emotional speech, several classifiers have been suggested, such as Hidden Markov Models (HMM) [DH99, RD06, CH94, NMBM01, KCHL03, NFS03], Neural Networks (NN) [BGH00, HW96, NTN00, BWG04], Gaussian Mixture Models [LNHS05, AKK07], and Vector Quantization (VQ) [RD06, KK11] using a variety of feature vectors.

In spite of its wide range of applications [START_REF] Macon | Applications of sinusoidal modeling to speech and audio signal processing[END_REF], the Sinusoidal Model (SM) [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] has not been thoroughly engaged in analysis and/or classification of stressed speech until recently [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF][START_REF] Drioli | Emotions and voice quality: Experiments with sinusoidal modeling[END_REF]. In these approaches, the parameters of sinusoids (amplitude, frequency, and phase) over time are suggested as features for classification or conversion of speech using Hidden Markov Models, Vector Quantization, and Gaussian Mixture Model-based techniques. Although the use of amplitude and frequency contours was straightforward, the phase contours are either disregarded or could not be directly used in the analysis. Furthermore, the parameters obtained from sinusoidal analysis have a significant constraint; they are extracted under the assumption of local stationarity, that is, the speech signal is considered as stationary inside the analysis window. However, this is not the case for speech styles characterized as "fast" or "angry". Recently, the adaptive Sinusoidal Models (aSMs) [PRS11, KPRS12, DS13] have managed to cope with this problem by projecting the signal onto a set of amplitude-and frequency-varying basis functions inside the analysis window. This way, the parameters represent the underlying signal more closely as an AM-FM decomposition. In brief, the adaptive Quasi-Harmonic Model (aQHM) [START_REF] Pantazis | Adaptive AM-•FM Signal Decomposition With Application to Speech Analysis[END_REF] adapts the phase of the basis function to the local characteristics of the signal, whereas the extended adaptive Quasi-Harmonic Model (eaQHM) [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF] performs both amplitude and phase adaptation. More recently, the adaptive Harmonic Model (aHM) [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF] assumes full-band harmonicity and iteratively adapts the fundamental frequency f 0 to localize harmonics up to the Nyquist frequency. All models have demonstrated their ability to model adequately and accurately speech signals from different languages and different speakers. However, they have not been tested in emotional speech, where it is assumed that the AM-FM components of the speech signal behave differently compared to neutral or conversational speech.

In this work, the extended adaptive Quasi-Harmonic Model (eaQHM) is utilized to demonstrate its ability to analyze, resynthesize, and classify emotional speech. The speech corpus for the analysis and resynthesis is a high-quality, wideband database containing emotional running speech. Subjective listening tests have been conducted to prove the transparency of the resynthesized speech. It is also shown that eaQHM can efficiently model all styles of emotional speech in this database with high precision, and this is demonstrated via Signal-to-Reconstruction-Error Ratio (SRER) values, compared to the standard SM. Moreover, an emotion classification task is presented using the well-known Speech Under Simulated and Actual Stress (SUSAS) [START_REF] Hansen | Getting started with SUSAS: A speech under simulated and actual stress database[END_REF] database, in which there are 11 pre-labelled emotional speech corpora. Details on the database are discussed in Section 3. Results show that the sinusoidal features of the eaQHM yield higher classification scores than those of the SM.

The rest of the work is organized as follows. In Section 7.2.1 presents the analysis parameters and the evaluation, both objective and subjective, of the eaQHM compared to SM. Section 7.3 describes the VQ-based classification experiment, and Section 7.4 discusses future perspectives. Finally, Section 7.5 concludes the work.

Analysis and Evaluation

In this section, the evaluation procedure is described, along with the dataset selection and the parameter estimation.

Objective Evaluation

At first, it is important to show that eaQHM can decompose high-quality running expressive speech signals into AM-FM components that represent the signal closer than SM. For this, a custom, small database of acted speech is used. This database consists of one male and one female subject, acting in four different speaking styles (angry, sad, happy, neutral), in a recording studio. A total number of 20 waveforms sampled at 16000 Hz are analyzed. All speech files in the database have been analyzed and resynthesized from their AM-FM components, and the corresponding SRER has been computed for each speech utterance. For this analysis, the window size was 30 ms for the SM and 3 local pitch periods for the eaQHM, both of Hamming type. A step size of 2.5 ms was selected for both models. The results are depicted in Table 7 However, this database is not appropriate for classification purposes, since the containing data is too few. Another database will be used, named SUSAS (Speech Under Simulated and Actual Stress). The SUSAS database was developed in the 1990s and was the first emotional speech database ever created. It contains both actual and simulated stressed speech. In the simulated part, 9 U.S. English male speakers, of three main dialects (general USA, New England/Boston, and New York City accent), under different simulated stress conditions (angry, clear, fast, lombard, loud, neutral, question, slow, soft, and two conditions where the speaker was recorded during medium and light activity) have been recorded. Each speaking style corpus has 70 speech files per speaker, which consist of isolated, short communication words, such as "hello", "break", "go", and "destination". This amounts to about 1190 tokens per speaker, with a considerable subset of them being acoustically similar, such as (six, fix) and (white, wide). The simulated data in SUSAS database were sampled using a 16-bit A/D converter with sample rate of 8 kHz. Table 7.2 shows the mean and the standard deviation of SRER for all speakers, for most common speaking styles.

This clearly demonstrates the quality and the performance stability of the adaptive model compared to the SM on a large database of isolated words of different expressive speaking styles. It is interesting to note that both models appear to be very stable around a mean of about 16.6 and 32.5 dB, for the SM and the eaQHM respectively. Although the distribution of SRERs is wider in eaQHM-analysis, the mean is high enough to show that in almost all cases the eaQHM manages to compactly capture most of the information present in the speech signal, for all speaking styles. Conclusively, it is evident that the adaptive model can handle word-isolated (i.e. SUSAS) and running expressive speech equally well.

Subjective Evaluation

For our subjective evaluation, a formal, on-line listening test was designed1 using the small, high-quality database of emotional running speech. The listeners were asked to evaluate the overall quality of the resynthesized speech based on the two models. A total of 32 listeners participated in this test, and the results are depicted in Figure 7.1 along with the 95% confidence intervals. Please note that only 5 of them are familiar with signal processing. According to the preference test, almost all listeners noted eaQHM as being almost indistinguishable to the original one. It should be noted that the SUSAS database was judged to perform poorly from a perceptual point of view due to the recording noise and the low sampling frequency. Informal listening tests showed that the eaQHM-based resynthesized speech samples were indistinguishable from the original ones, but this was the case for most samples obtained from the standard Sinusoidal Model as well. After careful listening, only a minority of waveforms demonstrated perceptual differences between the models but they were not enough in quantity to justify a listening test with this database. However, due to its pre-labelled data and its parallel corpora for each speaking style, this database was characterized as suitable for the classification task.

VQ-based Emotion Classification

As already discussed, a discrimination between different emotional speaking styles is of great interest. Considering a sinusoidal analysis, it has been reported that amplitude and frequency values of the sinusoidal components can be used successfully to characterize the different expressive classes (emotions) in a speech signal [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF]. Since the eaQHM can compute these parameters more accurately, it is not surprising that their discrimination properties among different speaking styles are similar or better than those reported in the literature for the standard SM. An example is presented in Figure 7.2, where the parameters of two speech samples (of the same word: "No") from the SUSAS database pronounced with different emotional content (angry, neutral) are depicted. Clearly, the amplitudes and frequencies of the fundamental are different in each case, and this is the case for other sinusoidal components as well. Another example of a single word ("point") in four different emotions is depicted in Fig. 7.3, along with the corresponding spectrograms that partly reveal their differences. The signals are aligned according to the stop consonant /p/. It can be seen that these differences appear in amplitude strength, frequency variations, energy distributions, formant positioning, timings, duration of vowels and consonants, etc. Sinusoidal modeling can capture some of these differences in the form of AM-FM components [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF]. Due to its adaptive processing, we propose that eaQHM can provide parameters that are highly accurate, which makes them more suitable for an emotion classification task than the same parameters obtained from a standard SM.

Feature Extraction

To evaluate our suggestion, a classification task based on a 128-bit Vector Quantizer (VQ) was designed using a subset corpus of the SUSAS, labelled as Angry, Neutral, Soft, and Question. A total number of 2520 waveforms (630 per emotion) were used. A number of 756 waveforms were kept for testing (189 per emotion), while the rest were used for training. All discrete-time waveforms were normalized to unit energy, as in

x[n] = x[n] � � L-1 n=0 x 2 [n] (7.1)
where L is the signal length in samples. Both models used an analysis frame rate of 2.5 ms. The 10 strongest components of the magnitude spectrum of the FFT and the 10 highest sinusoidal amplitudes provided by the LS, along with their corresponding frequencies, were extracted from each analysis frame. The analysis window was set at 30 ms for the SM, and at 3 local pitch periods for the eaQHM. No distinction between voiced and unvoiced parts of speech was made in this work.

Classification -Single Feature

At first, two classification tasks were set, each one using different features (amplitudes and frequencies). Having M spectral vectors x i containing the selected features (amplitudes or frequencies), the data matrix X is created as

X = [x 1 x 2 • • • x M ] (7.2)
The codebooks are then designed based on the minimization of the Average Distortion (AD) between the training vectors and the codebook vectors in matrix Y, where

Y = [y 1 y 2 • • • y C ] (7.3)
and C is the codebook size. The AD is defined as

AD = 1 C C � k=1 min y i ∈Y d 2 (x k , y i ) (7.4)
where d(x, y) is the Euclidean Distance (ED) between vectors x and y. For each of the four emotions mentioned earlier, a codebook was designed using the LBG algorithm [START_REF] Linde | An algorithm for vector quantizer design[END_REF]. The emotion is recognized by the minimum average distortion. The Confusion Matrix for the amplitude-based classification is given in Table 7.3, whereas for the corresponding frequency-based one is given in

Table 7.4: Classification score (%) for four emotions of the SUSAS database, using frequency features extracted from eaQHM and SM (in parenthesis).

In general, the parameters obtained from the eaQHM lead to better classification scores in all cases. Furthermore, the angry speaking style has the highest correct classification percentage for both models and both sets of features. The question speaking style is the most difficult one to correctly classify when the frequencies are used as features, and we can see that it is mostly confused with the neutral speaking style. On the other hand, the soft speaking style has the lowest classification score when the amplitudes are used as features.

Classification -Combined Features

Since single-feature based classification leads to low classification scores, a combined classification scheme is suggested. The ADs obtained from amplitude and frequency based VQs are normalized by the highest corresponding AD. Then, the ADs of the corresponding emotions are added. Finally, the emotion with the minimum sum of ADs is selected as the recognized emotion. This way, when the VQs have decided differently, the VQ which is more "confident" in its decision (the minimum AD is far less than other ADs) can influence the final outcome. Table 7.5 presents the corresponding classification scores for eaQHM and SM using the proposed scheme. Using this scheme, on average, the eaQHM correctly classifies 65% of the utterances in the database, whereas the SM reaches 54%. Apparently, not all speaking styles were favoured by this combined scheme. Mostly the angry and the question speaking style achieved significant increase of their classification rates in both models. While the angry speaking style already had a relatively high percentage, the question speaking style has interestingly increased its correct classification score. However, the soft and neutral speaking style did not significantly change their percentages. This suggests that a weighted sum of the ADs before ranking may be more appropriate.

Discussion and Perspectives

In this work, we attempted to perform emotion classification from speech signals using instantaneous parameters of sinusoidal models. Although the database in hand contains short, isolated words with similar perceptual content, and this makes recognition and classification results rather difficult, results are encouraging. However there is room for improvement.

First of all, the use of phase information could be exploited in combination with amplitudes and frequencies. In [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF], the number of phase reversals is suggested as a feature. However, a more intuitive measure could be suggested. In [SHE + 09], the notion of relative phase shift (RPS) is revisited and a phase structure is shown to be revealed through RPS. It would be interesting to examine if there are different patterns in RPS structures that can help discriminate emotional content in speech, combined with the standard amplitude and frequency features.

Secondly, sinusoidal amplitudes provide an implicit information about the spectral envelope, and they have been shown to be important in emotion recognition [DH99, HW96, HWA94]. Nevertheless, when considering only a part of the full-band, such as the 10 highest spectral peaks, a significant part of the spectrum is not taken into account. The inclusion of that part may contribute to better recognition percentages. Moreover, higher frequency components were suggested to be disregarded in sinusoidal model-based emotion classification as inappropriate for the task [START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF]. However, the aSMs are able to follow the dynamics of speech in the upper bands, and thus to reveal the spectral details that are blurred due to the time-frequency trade-off of the FFT-based estimation.

Furthermore, vowels have received increasing attention when it comes to emotion recognition, however consonants are shown to be important as well (see for example [START_REF] Bitouk | Class-level spectral features for emotion recognition[END_REF]). Since our model is full-band and models both voiced and unvoiced parts of speech using AM-FM components, it would be interesting to examine whether there is any useful information embedded in the sinusoidal representation of consonants that is able to distinguish emotions. Finally, different classifiers can be used, such as HMMs, SVMs, or GMMs, for a more efficient classification.

Conclusions

In this chapter, we presented an application of an adaptive sinusoidal model, named eaQHM, on the problem of emotional speech analysis and classification and compared it to the standard Sinusoidal Model. It was shown that different emotional speech styles can be effectively represented by the adaptivity mechanism of eaQHM, yielding very accurate AM-FM decomposition. This was demonstrated through resynthesis of the original speech signal from its AM-FM components and by evaluating the Signal-to-Reconstruction Error (SRER). A formal listening test was designed to evaluate the perceptual quality of the resynthesized speech and showed that eaQHM-resynthesized emotional speech is indistinguishable from the original. The instantaneous amplitude and frequency were used as features for the classification. Results showed that a Vector Quantization classification based on eaQHM achieves higher classification scores for a subset of the SUSAS database, both on single-feature classification based on the sinusoidal parameters and on their combination. Future work will focus on different classifiers, phase parameter exploitation, and transforming neutral speech into emotional.

Chapter 8

Conclusions and Future Work

Overview

In this work, we have presented the adaptive Sinusoidal Models (aSMs), with applications in speech modifications, speech classification, and musical instrument sound analysis. The focus of the thesis has been on the extended adaptive Quasi-Harmonic Model (eaQHM), which was introduced, thoroughly described, and evaluated for the aforementioned applications. The eaQHM has been shown to provide transparent speech quality and high Signal-to-Reconstruction Error Ratios (SRERs) by decomposing speech into AM-FM components.

First, the performance of the eaQHM was tested on analysis and resynthesis of speech. Hybrid and full-band systems were presented based on the eaQHM and the recently proposed adaptive Harmonic Model (aHM). According to this distinction, hybrid systems included the adaptive Harmonic + Noise Model -aHNM and the extended adaptive Quasi-Harmonic + Noise Model -eaQHNM, whereas full-band systems included the adaptive Harmonic Model -aHM and the extended adaptive Quasi-Harmonic Model -eaQHM, i.e. there was no noise component to model stochastic parts of speech. Full-band systems do not require voicing decision, which is often a difficult task, thus alleviating the overall complexity of the system. Results showed that the full-band eaQHM provides transparent speech quality and stands among the competition.

Hybrid systems based on the aSMs showed to be efficient in speech modifications, such as time and pitch scaling. The separate manipulation of components (deterministic, stochastic) provided convenience and flexibility, and modified speech turned out to be of high quality, given that voicing decision is well estimated. On the other hand, full-band systems were shown to provide equally high quality. Compared to the state-of-the art, the aSMs have performed better than well known methods, such as the Harmonic+Noise Model (HNM) and the Waveform Similarity Overlap-Add (WSOLA), and are comparable to the mostly used high quality method for speech modifications, named STRAIGHT. Moreover, the advantages of the aSMs include flexibility, simplicity, and compactness of representation.

The application of the aSMs in modelling musical instrument sounds was presented in this thesis. It was proved that eaQHM can outperform the standard Sinusoidal Model (SM) and the recently suggested Exponentially Damped Sinusoidal Model (EDSM) in terms of analysis and representation of the oscillatory, transient, and sustain behaviour of musical instrument sounds. Preliminary residual analysis has also shown that the "filtered white noise" approach in residual modelling of sounds is not attained by stationary sinusoidal models, such as the SM. On the contrary, the eaQHM was shown to leave a residual that when represented by filtered white noise, it is perceptually closer to the original residual signal, thus showing there is no oscillatory information left in it.

Finally, the eaQHM has been applied on the task of emotional speech analysis and classification. Although sinusoidal models have not been used much in the emotion classification literature, we first showed that the eaQHM outperforms the standard SM in analyzing and resynthesizing emotional speech in terms of reconstruction and perceptual quality. Then, the instantaneous parameters obtained from the analysis were used for the classification task, yielding higher classification scores for the eaQHM than for the SM.

Future Research Directions

Regarding the adaptive models themselves, further improvements and applications can be considered for the future. Improvements may include the complexity reduction of the models. Due to the adaptivity process, the computational time for all models is increased. Methods and techniques can be applied to increase speed. Also, other approaches for instantaneous phase estimation can be used and evaluated instead of frequency integration. Such an approach is cubic polynomials. Furthermore, the current convergence criterion for the eaQHM is based on the rate of increment of the Signal-to-Reconstruction-Error Ratio (SRER) computed on the whole waveform, over successive adaptations. This way, some frames may not be optimally reconstructed in terms of their local SRER, that is, they need further adaptations to locally converge to their optimal SRER. To solve this, a frame-based convergence criterion should be imposed in a computationally efficient way. Finally, the Least-Squares-based parameter estimation scheme limits the application of the models to monophonic speech, an extension to polyphonic speech can be attempted, using more robust estimation schemes, such as Total Least Squares.

From an applications perspective, Text-to-Speech (TTS) synthesis using Unit-Selection from sinusoidal representations has been successfully suggested. A sinusoidal model which provides more accurate signal representation such as eaQHM can lead to more naturally sounding speech units. Moreover, sinusoidal models in statistical TTS are less common in the literature. HMM-based speech synthesis has gained increased attention over the last years, especially when low footprint and general speech domain are required. High resolution models such as eaQHM can provide parameters that can be proved useful. Another application is Voice Conversion (VC). VC is similar to speech modification but the modifications are with respect to a target speaker. Statistical methods such as GMMs have been successfully applied in VC using parameters from sinusoidal models. Furthermore, speech coding can be applied in the aSMs, and especially in the aHM, which provides almost transparent speech quality with less parameters than the eaQHM.

On speech modifications, there is room for improvement on pitch shifting. It is suggested that during pitch shifting, there is some sort of interaction between the vocal tract and the glottal source. In all state-of-the-art systems, this is not taken into account for. That is, the vocal tract remains unaffected during pitch shifting. Further research can be made on how the vocal tract changes according to different pitch values, and correlate it with formant frequencies and bandwidths. This will lead to perceptually better modification schemes. Additionally, some parts of speech need to be protected from time or pitch scaling both in full-band and in hybrid systems, such as stop sounds for time-scaling and unvoiced speech for pitch-scaling. A time-varying modification factor based on the characteristics of speech would improve naturalness even more. Finally, spectral envelopes other than DAP (which is computationally intensive due to the iteration process) can be tested and evaluated.

On musical instrument sounds analysis, the modification schemes discussed in this thesis can be applied to produce artificially time-stretched or pitch-shifted counterparts of the musical instrument sounds. Another very important application in the audio processing domain is audio coding. Since the eaQHM yields highly accurate estimates of the instantaneous components of a signal, then coding these parameters will probably result in better quality with the same bitrates compared to the state-of-the-art.

On emotional speech analysis, recognition, and classification, there is vast room for improvements. There is considerable uncertainty as to the best feature set for classifying emotional data. At first, more comparisons should be made, especially with classifications schemes that use MFCCs, which are the most common set of features when it comes to speech recognition. The advantage of sinusoidal models is that the instantaneous parameters -which are used as features for the recognition -are jointly "optimal" in the sense that they accurately represent the dynamics of speech over time and frequency using a reconstruction criterion. Thus, a sinusoidal approach offers a set of features that come from a single estimator, rather than many different ones, as is the case in the literature (MFCCs, energy, pitch, speaking rate, and statistics on them). Moreover, there is still a debate on which classifiers to use for the classification: Support Vector Machines (SVM), Vector Quantizers (VQ), Gaussian Mixture Models (GMMs), Hidden Markov Models (HMMs), and Neural Networks (NN) are the most dominant candidates. However, the most interesting research direction is using the aSMs for statistical emotional speech synthesis and transformations between different emotions.

Part IV

Appendices

Annexe A

Résumé détaillé en franc ¸ais A.1 Introduction Générale

Sans aucun doute, la parole est le plus important et le moyen le plus naturel de communication entre les êtres humains. Pour cela, le traitement numérique des signaux de parole a été l'un des domaines les plus passionnants de traitement du signal. Dans les dernières décennies, la recherche de la parole a conduit les scientifiques et les ingénieurs dans la découverte de plusieurs outils qui ont encore énorme impact sur la société. Communication et de stockage de voix ont été réalisés et efficace grâce aux progrès de codage de la parole [RD01, HJA02] algorithmes et technologies. L'interaction homme-ordinateur est devenu plus pratique en raison de de reconnaissance vocale [START_REF] Huang | An overview of modern speech recognition[END_REF] techniques qui ont pu rendre les ordinateurs comprennent non seulement la parole humaine , mais aussi les langues humaines. Systèmes interactifs ont augmenté leur efficacité en correspondant à l'homme d'une voix naturelle : synthèse vocale [ZTB09] algorithmes sont responsables de cela. Modèles et principes d'analyse des parole [START_REF] Quatieri | Discrete-Time Speech Signal Processing[END_REF] ont réussi à fournir une connaissance plus approfondie sur le système de production de la parole humaine, aider les médecins vers plus rapide et une détection fiable des pathologies et des anomalies de la parole. Algorithmes pour l'amélioration de la parole [START_REF] Benesty | Speech Enhancement. Signals and Communication Technology[END_REF] dans des conditions bruyantes faites terrestre et communications par satellite plus robuste. L'industrie du divertissement a bénéficier de pointe de transformation de la parole [START_REF] Stylianou | Voice transformation[END_REF] techniques pour fournir des voix artificielles dans les jouets, des films et des jeux vidéo. La liste est sans fin et devient de plus en plus peuplé de jour en jour.

Apparemment, traitement de la parole a de nombreuses applications et probablement aura plus à l'avenir, lorsque la convergence de l'informatique, les communications et l'Internet se renforce.

Dans cette thèse, l'accent est mis sur Modèles adaptives sinusoïdaux de parole avec des applications sur parole modifications et analyse audio.

Modèles sinusoïdaux adaptatifs (aSMs), bien que pas entièrement développé pour cette thèse, sont présentés dans une perspective commune, comme des modèles sinusoïdaux qui s'adaptent aux caractéristiques locales du signal de parole sous-jacent. En d'autres termes, ils ne pas partager l'hypothèse de stationnarité des modèles mentionnés ci-dessus font. Le premier aSM a été développé en 2010 par Pantazis [START_REF] Pantazis | Adaptive AM-•FM Signal Decomposition With Application to Speech Analysis[END_REF], suivant le modèle proposé par Laroche [START_REF] Laroche | A new analysis/synthesis system of musical signals using Prony's method. Application to heavily damped percussive sounds[END_REF], et d'affiner successivement il [PRS08, PRS09b, PRS09a, PRS10] jusqu'à atteindre un temps où le modèle est non-paramétrique adaptatif et pour les caractéristiques de phase du signal analysé. Ce modèle est appelé le modèle quasi-harmonique adaptatif (aQHM) [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF]. Deux ans plus tard, Degottex et Stylianou proposé une pleine bande Modèle Harmonique adaptatif (aHM) [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], où l'analyse utilise un schéma quasi-harmonique pour un précis f 0 estimation, mais la synthèse est strictement harmonique. Un peu après, le modèle qui occupe la majeure partie de cette thèse, le étendu modèle quasiharmonique adaptatif a été suggéré [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF][START_REF] Kafentzis | Robust full-band adaptive sinusoidal analysis and synthesis of speech[END_REF], où l'adaptation comprend à la fois l'amplitude et la phase. Le principal avantage de aSMs est l'exactitude du processus d'estimation des paramètres, due soit à une très précis f 0 estimation (aHM) ou une amplitude très précis, la fréquence et la phase d'estimation (aQHM, eaQHM). Plus tard, une description détaillée des modèles sera présenté, les avantages de chaque modèle seront mis en évidence, et une comparaison sur l'analyse de la parole et resynthèse aura lieu.

En outre, les transformations de la parole se réfèrent aux différentes modifications qui peuvent être appliquées au signal de parole. Il couvre un large domaine de la recherche de la modélisation de la production de la parole et de la compréhension de la perception de la parole, et de traitement du langage naturel, la modélisation et le contrôle du style de parler, à la reconnaissance de formes et traitement statistique du signal.

Les transformations de la parole (les termes "transformations" et "modifications" sont utilisés indifféremment dans le reste de ce texte) ont de nombreuses applications potentielles dans des domaines tels que la synthèse de la parole (par exemple pour les systèmes interactifs de réponse vocale, les systèmes de dialogue, systèmes text-to-speech), le divertissement, l'industrie de la musique et le film, les jouets, les bavardoirs et les jeux, les aides auditives haut de gamme, la pathologie vocale et la restauration de la voix. L'objectif de la recherche est de proposer et mettre en oeuvre des algorithmes de modification de la parole sur la base des aSMs. Plus précisément, deux objectifs principaux doivent être abordés : le temps et les modifications pas à l'échelle. Donnons maintenant une définition formelle de ces deux termes :

• Dans la modification d'échelle de temps, le taux d'articulation est changée sans affecter la qualité de perception de la parole d'origine.

• En modification de pitch-échelle, la fréquence fondamentale est modifiée tout en préservant l'enveloppe de courte durée (conduit vocal) caractéristiques ainsi que la durée du discours original.

Bien que d'autres types de transformations existent, telles que la conversion de la voix (où la parole d'un source hautparleur est transformé pour correspondre aussi étroitement que possible le discours d'un cible haut-parleur), ce travail se concentre uniquement sur la modifications sans locuteur cible spécifique. Il est intéressant de noter que les modifications comprennent des termes tels que taux d'articulation et enveloppe de courte durée (conduit vocal), ce qui implique qu'il ya une combinaison de la théorie sinusoïdale modèle et source-filtre afin de fournir des modifications de la parole de haute qualité [START_REF] Quatieri | Speech transformations based on a sinusoidal representation[END_REF][START_REF] Quatieri | Shape-Invariant Time-Scale and Pitch Modifications of Speech[END_REF][START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF].

En outre, une variété d'applications intéressantes des SMO sont présentés dans cette thèse. Il s'agit notamment de la Modélisation des Instruments de Musique Sonne et le analyse et de classification de la parole expressive en utilisant les modèles sinusoïdaux adaptatif (et la plupart du temps, la eaQHM). Pour les premiers, la eaQHM est comparée à la modèle sinusoïdale exponentielle amortie (EDSM), qui est considéré comme l'état de l'art dans la modélisation audio, et la supériorité de la eaQHM est démontrée en termes d'instruments de musique des sons représentation. En ce qui concerne ce dernier, la eaQHM est comparée à la SM standard représentant de la parole expressive, et quelques résultats préliminaires concernant la reconnaissance et la classification sont présentés.

A.2 Modèles sinusoïdaux adaptatif

Le terme Modèles sinusoïdaux adaptatif -aSMs se réfère à la famille des modèles sinusoïdaux qui sont capables de s'adapter aux caractéristiques locales (phase et / ou amplitude) du signal de parole analysé. Il convient de souligner que la non-adaptatif des modèles sinusoïdaux (classiques) ne considèrent stationnarité locale d'un signal dans leur représentation.

Avant d'entrer dans les détails, les notions de stationnarité et adaptativité doivent être définis avec précision. Il convient de noter que la stationnarité est un terme général qui peut être utilisé pour caractériser un quelconque signal ; Cependant, l'adaptation sera définie dans un contexte de ce qui représente un signal avec un ensemble de fonctions de base exponentielles complexes. Maintenant nous allons définir le stationnaire et adaptation termes d'analyse sinusoïdale.

Soit un ensemble de fonctions de base exponentielles complexes qu'un signal est projeté sur.

• A exponentielle complexe est appelé stationnaire dans un intervalle de temps bien défini lors de sa fréquence et / ou son amplitude est constante.

• A exponentielle complexe est appelé adaptatif dans un intervalle de temps bien définie lorsque son amplitude et / ou la fréquence sont calculés en prenant en compte les caractéristiques locales du signal qui est projetée sur elle.

En général, les aSMs sont fondées sur le principe de la projection d'un segment de signal sur un ensemble de non-paramétrique, variant dans le temps, les fonctions non-stationnaire base exponentielles complexes l'intérieur d'une fenêtre d'analyse, alors que les modèles sinusoïdaux classiques considèrent que la parole caractéristiques demeurent relativement inchangées dans un niveau local, ainsi leurs fonctions de base sont stationnaires. La construction de cet ensemble de fonctions de base variables dans le temps dépend du modèle adaptatif, comme cela deviendra apparent dans ce chapitre.

Tout d'abord, le coeur de tous les aSMs est présenté, le soi-disant modèle quasi-harmonique -QHM. Le QHM n'est pas un modèle adaptatif lui-même, mais il fournit le mécanisme pour l'adaptation, c'est un textit mécanisme de correction de fréquence, qui donne une estimation de l'écart entre les fréquences réelles et estimées. Cette correction de fréquence est ajouté aux fréquences estimées pour permettre une représentation plus proche du signal sous-jacent. Une procédure itérative sur l'estimation des paramètres conduit à la modèle itératif quasi-harmonique -iQHM, qui met à jour successivement les fréquences jusqu'à ce qu'un critère de convergence soit satisfait. Toutefois, les deux QHM et iQHM détiennent l'hypothèse de stationnarité locale, qui est, tous les paramètres sont obtenus en projetant le signal sur un stationnaire ensemble de fonctions de base exponentielles.

Pour atténuer l'hypothèse de stationnarité locale, le modèle adaptatif quasi-harmonique -aQHM développé par Pantazis et al [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF] est présenté, puis, nous proposons une extension, le modèle étendu adaptatif quasi-harmonique -eaQHM [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]. Ces modèles vont encore plus loin et d'exploiter le mécanisme de correction de fréquence de QHM d'affiner leurs estimations de fréquence, ainsi que la construction itérative d'un variant dans le temps, non paramétrique, et ensemble non stationnaire de fonctions de base. Par conséquent, une définition de la adaptation terme est donnée comme suit : Soit un ensemble de fonctions de base exponentielles complexes qu'un signal est projeté sur. L'adaptation est une construction itérative de l'ensemble de fonctions de base exponentielles complexes en fonction des caractéristiques locales du signal sous-jacent, en parallèle à un affinement des composantes instantanées du signal, par exemple, l'instantané d'amplitude, de fréquence et de phase.

En outre, le QHM fonctionne comme un initialiseur pour aQHM et eaQHM, fournissant un ensemble bien-estimée de trajectoires de fréquence. Enfin, une autre approche récente, appelé le modèle harmonique adaptatif -aHM [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF] est brièvement présenté. Le aHM est un modèle purement harmonique, mais bénéficie de l'idée de fonctions de base non-stationnaire pour fournir une représentation plus proche de la parole par un raffinement itératif de la fréquence fondamentale.

A.2.1 Introduction

En général, un ASM peut être décrit comme

x(t) = � K � k=-K C k (t)ψ k (t) � w(t) (A.
ψ aQHM k (t) = 1 • e jφ k (t) , C aQHM k (t) = a k + tb k , (A.4) ψ eaQHM k (t) = α k (t) • e jφ k (t) , C eaQHM k (t) = a k + tb k (A.5) et ψ aHM k (t) = 1 • e jkφ0(t) , C aHM k (t) = a k + tb k (A.6)
où a k , b k sont l'amplitude complexe et de la pente de la k e composante du modèle, φ 0 (t) est la phase instantanée de la fréquence fondamentale, calculée comme l'intégrale de celui-ci. Sur la base de cette analyse, d'introduction, la description du prochain eaQHM suit. Le lecteur peut en savoir plus sur la aQHM et l'aHM dans [START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF][START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF].

A.2.2 La modèle étendue adaptatif quasi-harmonique -eaQHM

Dans cette thèse, nous proposons une extension de aQHM à intégrer l'adaptation amplitude, appelé la modèle étendue adaptatif quasi-harmonique (eaQHM) :

x(t) = � K � k=-K (a k + tb k ) αk (t)e j( φk (t+ti)-φk (ti)) � w(t), t ∈ [-T, T ] (A.7) où αk (t) = A k (t + t i ) A k (t i ) (A.
(E e0 ) n,k = α k (t n )e j(φ k (tn+ti)-φ k (ti)) (A.10) et (E e1 ) n,k = t n α k (t n )e j(φ k (tn+ti)-φ k (ti)) = t n (E e0 ) n,

A.2.3 Algorithme d'analyse sinusoïdale adaptatif utilisant eaQHM

Les modèles sinusoïdaux adaptatif nécessitent une étape d'initialisation, de sorte QHM [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF] seront utilisés à cette fin, bien que n'importe quel algorithme d'estimation de fréquence peut être utilisée. Ainsi, les initiales étapes sont les suivantes :

f 0 k (t i ) = f 0 k (t i-1 ) + ηk (A.19) Â0 k (t i ) = |a i k |, φ0 k (t i ) = ∠a i k (A.20)
où t i est le centre de l' i e du cadre d'analyse. L'algorithme de décomposition AM-FM à l'aide eaQHM est fourni dans l'algorithme 6. Le critère de convergence de l'algorithme a été choisie pour être la suivante :

SRER m-1 -SRER m SRER m-1 < � (A.
x(t) = K � k=-K Âk (t)e j φk (t)
Le modèle a été évalué dans des signaux synthétiques, et a été prouvé pour être robuste en présence de bruit additif [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF].

A.2.4 Validation sur parole vocale

La prochaine étape est de tester le modèle proposé sur la parole réelle, et en particulier, sur des signaux de parole voisés. L'algorithme de décomposition suggéré itératif AM-FM basée sur eaQHM peut être appliqué sur des signaux de parole voisés d'une manière simple. En fait, l'algorithme eaQHM peut être appliqué sur un grand segment de la parole voisée. En effet, en supposant que la parole vocale est quasi-périodique et que le contenu de la fréquence des signaux de parole voisés ne change pas très vite , alors nous avons seulement besoin de fournir la fréquence fondamentale de la première trame voisée au début du segment exprimé, f 0 (t 1 ), puis supposer f 0 k (t 1 ) = kf 0 (t 1 ) . L'application de l'analyse du C Port sur la première trame voisée, un ensemble actualisé de fk peut être obtenue pour cette image. L' ensemble mis à jour de fréquences peut ensuite être utilisé comme estimations initiales pour la trame voisée suivante. En continuant de cette manière, l'ensemble de la région voisée sera analysée en fournissant seulement la fréquence fondamentale de la première image du segment voisé. Il est à noter que la précision de l'estimateur de fréquence fondamentale n'est pas crucial pour aQHM, depuis décalages de fréquence sont facilement corrigées.

Pour notre propos, nous considérons un signal de parole voisée de la base de données CMU-ARCTIQUE avec une fréquence d'échantillonnage f s = 16 kHz et la durée d'environ 0.35 sec. Le aQHM est également inclus pour comparaison. Pour les deux algorithmes, le nombre d'harmoniques a été défini sur K = 40 et une estimation de la fréquence fondamentale du début du segment a été donnée à l'algorithme. Au plus 10 adaptations ont été autorisés à les modèles. La taille de la fenêtre d'analyse était de 2.5 périodes de hauteur et de la taille de l'étape d'analyse a 1 échantillon. Dans la suite, les signaux sont considérés jusqu'à un maximum fixe exprimé fréquence (5500 Hz). Pour comparer objectivement les performances des deux algorithmes, l'es srer définie dans l'équation (A.22) a été utilisé. L'es SRER était 41.2 dB pour aQHM et 45.2 dB pour eaQHM. Deux adaptations pour aQHM et trois adaptations pour eaQHM étaient nécessaires pour les modèles à converger.

Pour confirmer ces résultats, un critère objectif à grande échelle a été réalisée. Utilisation de trois tailles différentes de pas, soit 1 ms, 2 ms, et 4 ms, nous avons analysé et reconstruit environ 50 minutes de parole voisée de 3 haut-parleurs dans la base de données de l'Arctique. La fréquence d'échantillonnage des signaux de parole a été sous-échantillonné à 16 kHz. Une fenêtre de Hamming de longueur fixe a été utilisé ; 3 fois la période de la hauteur moyenne de l'orateur. La même fenêtre a été utilisé à la fois pour aQHM et eaQHM. Le nombre de composants a été fixé à K = 30. La moyenne et l'écart-type de la es srer (en dB) est fourni dans le tableau A.1, avec différents pas de temps.

A.2.5 Conclusions

Dans ce chapitre, la théorie des modèles sinusoïdaux adaptatif a été présenté. Un nouveau modèle sinusoïdal adaptatif (eaQHM) est proposé. Dans le eaQHM, l'amplitude, en même temps que la fréquence du signal, est incluse dans le processus d'adaptation, d'une manière simple. Des expériences sur la parole vocale utilisant la base de données ARCTIC ont montré que eaQHM surpasse aQHM en matière de reconstruction de signal.

A.3 Etat de l'art à l'analyse, de synthèse, et des modifications de la parole Par souci de clarté et de commodité, les travaux connexes est divisé en deux sections : les techniques paramétriques et non paramétriques. Une présentation des régimes les plus importants seront réalisés dans ce travail. Toutes les méthodes d'analyse de la parole, de modification et de synthèse seront décrites à partir des approches précédentes jusqu'à ce que les derniers en date, afin de montrer l'évolution du domaine scientifique à travers les années. En outre, de cette fac ¸on, il est plus facile de mettre en évidence d'importantes améliorations par rapport aux procédés. Toutefois, l'accent sera mis dans les approches paramétriques, car ils sont plus proches des modèles-en-main.

A.3.1 Les techniques non paramétriques

Les techniques non paramétriques sont principalement basées sur le court-temps transformée de Fourier (STFT), avec la soi-disant vocodeur de phase étant le représentant le plus connu. Phase Vocoder est presque totalement une technique de fréquence sur la base. D'autres approches incluent des techniques de temps de domaine, telles que la méthode Overlap-Add (OLA) et ses variantes, avec pitch-synchrone OLA (PSOLA) étant une étape importante. Les plus importantes celles de chaque point de vue sont discutés prochaine. L'idée sous-jacente est que l'analyse STFT une transformée de Fourier est effectuée sur un segment de parole à fenêtre, puis un décalage dans le temps est effectué, et la transformée de Fourier est appliquée une autre, et ainsi de suite.

Phase Vocodeur

Ainsi, la STFT peut être mathématiquement décrit comme

X(t a (k), ω) = ∞ � n=-∞ h u [n]x[t a (k) + n]e -

STRAIGHT Vocodeur

STRAIGHT est un vocodeur hight-qualité qui utilise l'analyse spectrale de pitch-adaptative associée à un procédé de reconstruction de la surface dans la région temps-fréquence, et une conception de source d'excitation sur la base de la manipulation de phase. Plus précisément, STRAIGHT décompose le signal de parole en trois termes • Un spectrogramme lisse, sans périodicité dans le temps et la fréquence • Un contour f 0 , et • Une cart de périodicité temps-fréquence, qui capte la forme spectrale du bruit et aussi son enveloppe temporelle D'un point de vue des modifications, l'étape à suivre est très simple :

• Modification d'échelle de temps réduit à la duplication / suppression de tranches ST du spectrogramme STRAIGHT et apériodicité carte.

• Modification de l'échelle de hauteur réduit à la modification du contour f 0

Les trois termes STRAIGHT peuvent être manipulées indépendamment, ce qui permet une plus grande flexibilité.

STRAIGHT permet de très haute facteur modifications prosodiques tout en conservant le naturel de la parole synthétisée. Le principal inconvénient de STRAIGHT est son intensité de calcul.

Comme pour d'autres approches, le travail de pionnier de Griffin et Lim [START_REF] Griffin | Multiband Excitation Vocoder[END_REF] sur le Vocoder Multiband Excitation (MBE) doit être mentionné. Amélioration par rapport au vocodeur de phase ont été proposées au fil des ans. Laroche et al [LD99] [Puc95] proposé une explication pour le problème de phasiness (souvent désigné comme la réverbération) et a suggéré une amélioration pour le calcul de phase qui réduit considérablement le problème. En outre, le coût de calcul a été réduite de plus d'un facteur de deux. D'autres améliorations ont été proposées dans [START_REF] Puckette | Phase-locked Vocoder[END_REF].

Overlap-Add (OLA) Méthodes

Pitch-Synchronous Overlap-Add (PS-OLA) est la base de la plupart des techniques dans le domaine temporel pour le temps et les modifications à l'échelle du terrain. L'étape d'analyse consiste à décomposer le signal de parole x[n] dans un flux de signaux d'analyse de courte durée, qui peut être notée x[t a (k), n], où t a (k) est l'indice du signal de courte durée. Une analyse fenêtre h[n] est appliqué sur le signal afin d'en extraire la forme d'onde de courte durée, c'est à dire

x[t a (k), n] = x[n]h[n -t a (k)] (A.24)
Le temps des instants t a (k) sont appelés analyse hauteur-marques, et ils sont mis de manière pitch-synchrone sur les segments voisés de la parole et de manière constante sur les non voisées. L'étape de la synthèse consiste à transformer le flux de signaux d'analyse à court temps dans un flux de signaux de synthèse à court temps. Ces signaux de synthèse sont synchronisés sur une nouvelle série d'instants de temps, appelée synthèse des instants de temps, dérivé des instants de temps d'analyse, en fonction de la modification désirée. Enfin, la production synthétique y[n] est obtenue en appliquant une procédure des moindres carrés pondérés OLA sur les signaux de synthèse court temps synchronisés. Les modifications peuvent être considérés comme une étape intermédiaire entre l'analyse et la synthèse. Dans un premier temps, les marques de synthèse hauteur doivent être générés en fonction de la hauteur désirée et / ou modification d'échelle de temps, et ensuite chaque synthèse pitch-marque est mis en correspondance avec un ou plusieurs analyse pitch-marque. Enfin, la synthèse est effectuée comme décrit dans le paragraphe précédent. D'autres variantes de PSOLA comprennent Fréquence domaine PSOLA (FD-PSOLA) [START_REF] Charpentier | Diphone synthesis using an overlap-add technique for speech waveforms concatenation[END_REF], Linear Predictive PSOLA (LP-PSOLA) [START_REF] Moulines | Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones[END_REF], et la forme d'onde de similarité Overlap-Add (WSOLA) [START_REF] Verhelst | An Overlap-Add Technique Based on Waveform Similarity (WSOLA) for High Quality Time-Scale Modification of Speech[END_REF].

A.3.2 Les techniques paramétriques

Techniques paramétriques font généralement référence à des méthodes qui reposent sur un modèle de production de la parole, dont les paramètres sont à estimer. Deux sont les principaux représentants de ces techniques : (1) un modèle avec plusieurs systèmes en série qui représentent les différentes étapes du système humain de production de la parole, à savoir le système d'excitation, le système de conduit vocal, et le système de lèvres, et (2) un modèle de la représentation des séries chronologiques, soit une somme de la fréquence et / ou modulés en amplitude cosinus.

La première approche a été la Linear Predictive Vocoder [START_REF] Atal | Speech analysis and synthesis by linear prediction of the speech wave[END_REF], qui a exprimé la parole est modélisée comme une convolution entre un train périodique d'impulsions et un filtre AR variant dans le temps. Bien que très populaire au début, il a été rapidement abandonnée en raison de son incapacité à fournir de haute qualité parole modifié. Plus tard, Almeida et Marques [START_REF] Almeida | Variable-frequency synthesis: an improved harmonic coding scheme[END_REF] [MA89] ont été les premiers à proposer l'utilisation de sinusoïdes d'analyse et de synthèse vocales. à la fin des années 80, Griffin [START_REF] Griffin | Multiband Excitation Vocoder[END_REF] et Serra et Smith [START_REF] Serra | A System for Sound Analysis, Transformation, Synthsis based on a Determistic plus Stochastic Decomposition[END_REF], avait proposé un modèle où les sinusoïdes sont en relation harmonique. Cependant, les étapes de la modélisation sinusoïdale étaient loeuvre de McAulay et Quatieri, la soi-disant modèle sinusoïdale, qui a été suivie de plusieurs variantes, comme [START_REF] George | Analysis-by-Synthesis Overlap-Add Sinusoidal Modeling Applied to the Synthesis of Musical Tones[END_REF], et que de Stylianou [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF] sur la modélisation harmonique, appelé le Harmonic plus Noise Model. Pour les transformations temps-échelle, les paramètres qui sont mises à l'échelle sont les amplitudes et les phases du système, M (ω, t), et ψ(ω, t), et l'amplitude d'excitation et de fréquences, a k (t) et ω k (t). La modification de ces paramètres correspondant à un déplacement plus lent ou plus rapide des articulateurs du conduit vocal et à l'étirement ou la compression de la trajectoire de fréquences, respectivement. Pour la modification de la hauteur, les fréquences d'excitation doivent être décalés à de nouvelles fréquences, selon une échelle de pitch facteur de modification β. Ainsi, la trajectoire de fréquence ω k (t) dans le signal original correspond à la trajectoire de fréquence βω k (t) Cependant, le modèle ne tient pas compte des changements dans les caractéristiques spectrales de l'appareil vocal. Pour cela, les amplitudes du système, M (ω, t), et les phases, ψ(ω, t), doivent être calculés dans les nouveaux endroits de la voie de fréquence βω(t).

A.3.3 Le modèle sinusoïdal (SM)

s(t) = N � k=1 A k (t) cos[θ k (t)] (A.30) où A k (t) = a k (t)M k (t) (A.31) et θ k (t) = Ω k (t) + ψ k (t) + φ k (A.

A.3.4 Le Modèle harmonique Plus bruit (HNM)

Au début et au milieu des années 90, Stylianou proposé un nouveau modèle, appelé le modèle harmoniques plus bruit (HNM) [START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. Dans ce modèle, le spectre de la parole est séparée en deux parties : une partie déterministe et stochastique a, délimité par un maximum variant dans le temps a exprimées fréquence. Dans la bande inférieure (partie déterministe), le signal est considéré comme harmonique. La partie stochastique, qui est le résidu du signal d'origine moins la partie déterministe, est modélisé par un modèle d'AR et de son comportement dans le domaine temporel est prélevée par une enveloppe dans le domaine temporel paramétrique. Dans ce modèle, à la fois l'analyse et la synthèse est effectuée d'une manière de pitch-synchrone, inspiré par PSOLA. Ainsi, on peut fournir des techniques souples pour mise à l'échelle du temps et de brai, dont il sera question plus brefs délais. Les comptes partiels harmoniques pour les phénomènes quasi-périodiques de la parole, tandis que les modèles de pièces composantes non périodiques bruit, qui comprennent généralement le bruit de frottement, vocal sourd, etc maximale variant dans le temps exprimé fréquence est utilisée pour déterminer la limite entre les deux parties. Dans la bande inférieure, le signal peut être modélisé comme une somme de sinusoïdes en relation harmonique avec variation lente des amplitudes et des fréquences : Pour la partie harmonique, les amplitudes et les phases sont estimés via LS, comme mentionné précédemment, et sont interpolées linéairement entre les trames successives. S'il vous plaît noter que les phases sont déballés avant d'appliquer l'interpolation. Cela se fait par une prédiction de la phase de la trame courante, en utilisant la phase de la précédente et de la fréquence instantanée moyenne. La partie du bruit est synthétisé en utilisant une procédure Overlap-Add (OLA), afin d'éviter des discontinuités aux frontières de trames. Enfin, pour les trames voisées, l'enveloppe de domaine temporel triangulaire est appliquée directement sur la partie de bruit synthétique. Le signal final de la parole synthétique est obtenue par l'addition des deux composants.

h(t) = K(t) � k=1 A k (t) cos(kθ(t) + φ k (t)) (A.33) où θ(t) = � t -∞ ω 0 (u)
h(t) = K � k=-K a k (t i )e jkω0t = K � k=1 A k (t i ) cos(kω 0 t + φ k (t i )), t i -N ≤ t ≤ t i + N (A .
Compte tenu des modifications, comme première étape, les nouveaux instants de temps de synthèse doivent être trouvées, selon l'analyse des instants de temps et la hauteur désirée et des modifications à l'échelle du temps. HNM suit un schéma de pitch-synchrone, inspiré par PSOLA, qui a été brièvement discuté à la section précédente. Une mise en correspondance entre les instants synthèse et d'analyse est déterminée en spécifiant une analyse instantanée qui doit être sélectionné pour une synthèse donnée une. Ceci est réalisé en fonction des contraintes suivantes : pour pas mise à l'échelle, l'évolution dans le temps doit être préservée, et mise à l'échelle de temps, le contour de hauteur doit être préservée. En cas de terrain mise à l'échelle, il faut calculer l'amplitude et la phase des harmoniques modifiés. Pour cela, une estimation de l'enveloppe spectrale et de la phase est nécessaire. Dans le contexte HNM, une technique de cepstre est utilisé régularisée [START_REF] O. Cappe | Regularized estimation of cepstrum envelope from discrete frequency points[END_REF], où les coefficients cepstraux sont calculées discrètes, avec un critère de LS de domaine de fréquence, associée à une méthode de régularisation pour augmenter la robustesse de l'estimation. Les amplitudes sont alors obtenues en échantillonnant l'enveloppe estimée aux nouvelles fréquences harmoniques. Pour l'enveloppe de phase, une autre approche est suivie. Considérons une trame vocale d'une partie vocale de la parole. La phase est déballé dans le domaine de fréquence en ajoutant des multiples entiers de 2π, afin de maintenir la variation de fréquence de pente, dφ k = Φ k+1 -φ k , aussi lisse que possible, où k est le k e harmonique. Dans la trame voisée prochaine, les phases sont déballés en utilisant les fréquences de ski de la trame précédente et pas la fréquence de ski de l'actuel. De cette fac ¸on, la phase poursuite est garantie à la fois dans le temps et dans le domaine fréquentiel. Enfin, les nouvelles phases sont obtenues par échantillonnage de l'enveloppe de phase aux harmoniques de pas modifiés. Ayant les nouvelles amplitudes et phases, la synthèse de parole modifié est effectuée de la même manière que dans la synthèse sans les modifications.

A.3.5 Le modèle LF + ARX

Une approche commune en traitement de la parole est de représenter le système de production de la parole comme un modèle source-filtre. Dans ces représentations, la source est désignée comme la dérivée du flux glottal (GFD) (comme la convolution de l'écoulement glottique et le rayonnement de la lèvre). Un modèle bien connu pour la représentation GFD est le modèle Liljencrants-Fant (modèle LF) [START_REF] Fant | A four-parameter model of glottal flow[END_REF], qui caractérise la GFD en utilisant un certain nombre de paramètres -généralement cinq : un pour l'emplacement de la source glottique, un pour l'amplitude, et trois pour la forme de l'écoulement glottique. Par conséquent, le signal de parole peut être représentée au moyen d'un modèle autorégressif avec une entrée exogène (modèle ARX) [START_REF] Ding | Simultaneous estimation of vocal tract and voice source parameters based on an ARX model[END_REF] :

s[n] = - p � k=1 a k [n]s[n -k] + b 0 u LF [n] + e[n] (

A.3.6 Discussion

Les systèmes hybrides de l'analyse des discours basés sur eaQHM (Modèle etendu adaptatif quasi-harmonique + bruit -eaQHNM) et aHM (de Modèle harmonique adaptative + bruit -aHNM) seront présentés dans cette thèse, avec des résultats très satisfaisants en termes de qualité perc ¸ue. Cependant, il sera démontré que certaines hypothèses de séparation des composants des systèmes hybrides ne sont pas nécessaires, et textit tous les sons du langage peut être représenté de fac ¸on très précise en tant que composants AM-FM seulement. Ainsi, la parole peut être uniformément représenté très précisément comme une somme de sinusoïdes AM-FM, offrant compacité, l'uniformité et la simplicité de la représentation de la parole. Modifications seront appliquées sur cette représentation de la parole ainsi, et leurs performances seront discutés et comparés avec les systèmes hybrides correspondants, ainsi que d'autres systèmes à la pointe de la technologie.

A.4 Analyse et synthèse de la parole utilisant des modèles sinusoïdaux adaptatifs

Dans cette section, nous présentons tout d'abord les systèmes hybrides de l'analyse et de la synthèse de la parole, suivis de leurs approches full-band.

A. 4

Analyse du partie déterministe

En supposant une analyse de Hamming fenêtre w(t) avec le soutien de [-T l , T l ], une image de la partie déterministe est d'abord modélisé à l'aide du QHM comme : Le k e trajectoire instantanée de fréquence, f k (t), est calculée par interpolation spline, parce cannelures offrent des transitions douces entre les estimations de fréquence. En outre, il est intéressant de noter que la fréquence correspondant est trivial puisque les fréquences d'analyse sont des multiples entiers d'une fondamentale. Quant à la k e trace de phase instantanée, φk (t), techniques d'interpolation similaires ne sont pas adaptés ; ainsi, une approche non paramétrique est suivi sur la base de l'intégration de fréquence instantanée. En outre, la cohérence de phase sur les limites de trame est adressée par l'ajout d'un terme supplémentaire afin de garantir la poursuite de phase sur les limites de trame que dans l'équation (A.46). Enfin, le composant déterministe peut être approximée par ses composantes variant dans le temps à l'aide :

d(t) = � L � k=-L (a k + tb k )e j2π
d(t) = L � k=-L Âk (t)

Partie déterministe

Dans l'étape d'analyse pour la partie déterministe, une paramétrisation du signal de parole à chaque fois d'analyse instant t i a est effectuée. Dans un premier temps, une séquence des instants d'analyse sont créés dans les parties vocales de la parole en utilisant la condition f 0 (t) trajectoire, comme nous avons un instant de temps d'analyse par période de hauteur. En outre, si la distance entre t i a et t i+1 a est assez courte, aHM peut modéliser les variations d'amplitude du signal non voisé (comme dans occlusives). Ainsi, la limite supérieure de la taille de la fenêtre d'analyse est de 20 ms et la limite inférieure est fournie à partir de la f 0 (t) trajectoire, et est ainsi fixé à 50 Hz. Autour de chaque temps d'analyse instant t i a , une fenêtre de Blackman avec une longueur de 3 périodes de hauteur locales est appliqué au signal de parole. La phase trajectoire φ 0 (t) est alors calculé au moyen d'une interpolation spline de f i 0 et en utilisant la formule d'intégration dans l'équation (A.54). Un algorithme dédié nommé Adaptive Itératif Raffinement (AIR) est conc ¸u afin de fournir une estimation très précise de la f 0 .

Partie stochastique

La partie stochastique est modélisé exactement de la même manière que dans eaQHNM (voir la section A.4.1).

Synthèse

Dans l'étape de synthèse de la partie déterministe de chaque harmonique est générée dans distinct, l'un après l'autre, sans utiliser de fenêtre. Chaque composante harmonique est synthétisé par ses paramètres, à savoir ses amplitudes |a i k |, phases ∠a i k , et la fréquence fondamentale f i 0 . Tout d'abord, l'amplitude instantanée, |a k (t)|, de la k e harmonique est simplement obtenue par interpolation linéaire estimé |a i k | sur la durée de l'analyse instants t i a , sur une échelle logarithmique. La phase instantanée ∠a i k pas peut être interpolée directement dans le temps pour obtenir a k (t) à cause de sa rotation en raison de l'avance de temps entre des instants de temps de l'analyse. Par conséquent, il est proposé de supprimer cet effet en utilisant l'intégrale de f 0 (t) depuis le début du signal, et obtenir le phase relative -RP :

∠ã i k = ∠a i k -kφ 0 (t i a ) (A.55)
Ainsi, en supposant que la forme du signal est en train de changer sans à-coup, les valeurs de phase changent également en douceur d'une analyse instant de temps à l'autre. Ensuite, le RP ∠ã i k peut être interpolée pour obtenir son homologue continue, ∠ã k (t). En outre, une cannelure ou interpolation cubique est nécessaire tel que sa dérivée dans le temps, la fréquence, est toujours constante. Tout au long du processus itératif, et depuis le nombre harmonique K i augmenter indépendamment l'un de l'analyse instantanée de temps à l'autre, il ya souvent des éléments manquants dans les interpolations d'amplitude et de phase instantanée. Si tel est le cas, alors l'amplitude de la composante manquante est réglée à -300 dB et l'angle correspondant de la phase ∠ã k (t) est mis à zéro.

Pour la partie stochastique, il est resynthétisé utilisant la méthode OLA. Pour chaque trame, le bruit blanc est passé à travers le filtre d'AR pour obtenir la modulation de fréquence de la partie stochastique. Ensuite, l'enveloppe énergétique est calculée à partir de l'équation (A.50) et sa multiplication par le bruit modulé en fréquence fournit la trame stochastique reconstruit.

A.5 La motivation des approches pleines bandes

Il ya plusieurs raisons pour proposer une approche complète de la bande : tout d'abord, pour la parole voisée, un certain nombre de systèmes hybrides s'appuient fortement sur une estimation précise de la soi-disant fréquence maximale voisée -MVF, qui divise le spectre de discours exprimé dans un déterministe et une partie stochastique. L'estimation de l'efficacité de la MVF est critique pour la performance du système et de ses modifications qui en résultent. Deuxièmement, comme il est décrit dans [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], et soutenu par d'autres chercheurs [START_REF] Doval | Spectral correlates of glottal waveform models : an analytic study[END_REF][START_REF] Doval | The Spectrum of Glottal Flow Models[END_REF] une telle MVF n'est pas nécessaire d'un point de vue de la production de la parole. En outre, bien que les fréquences basses exprimées dans un spectrogramme à base de DFT semblent avoir une structure régulière, ce n'est pas vrai pour les moyennes et des fréquences élevées, où la teneur en fréquence est floue. C'est précisément en raison de la nature non stationnaire du contenu en fréquence. Au contraire, l'utilisation de la Fan-Chirp Transform (FChT) révèle une régularité dans le contenu de fréquence de toutes les fréquences des parties voisées. Cependant, on peut se demander comment et pourquoi sinusoïdes sont appropriées lorsque représentant consonnes, comme fricatives ou sans voix / arrêt voisée sons. Un son de butée est réalisée avec une fermeture complète des articulateurs concernés, de sorte que le courant d'air ne peut pas s'échapper par la bouche. Sonores occlusives sont produites avec vibration des cordes vocales alors que dans sans voix s'arrête cordes vocales sont écartées. En occlusives sonores, il ya des oscillations juste avant l'éclatement, alors que dans occlusives sourdes, il n'y a pas d'oscillation et, dans de nombreuses langues, il est aspiration après l'éclatement. Un fricatif est produite avec des gros rapprochement de deux articulateurs, de sorte que le flux d'air est partiellement obstruée et la circulation d'air turbulent est produit.

Il est suggéré que, pour une fréquence d'échantillonnage de f s = 16 kHz, une valeur de fréquence initiale faible telle que 80 Hz, ce qui conduit à des valeurs de fréquence de 80k Hz, k = -100, • • • , 100, est suffisante pour couvrir le spectre de fréquence, c'est à dire qu'il s'agit d'une analyse complète de la bande. La fréquence décalage mécanisme de correction C Port affiner les fréquences autour de la maxima du spectre, et donc les composants de la plus haute énergie sera modélisé. Une FFT du point 2048 avec 100 pics spectraux a été considéré pour la partie SM. La longueur de la fenêtre est de 3 périodes de hauteur locale. Nous avons tenté de valider le modèle proposé sur une grande base de données de sons d'arrêt voisées et sourdes. Table A la parole voisée, non seulement, mais également des parties non voisées, ainsi avec une grande précision. Il est important de soutenir la transition des systèmes hybrides aux systèmes plein-bande qui fonctionnent sur la pleine longueur du signal de parole, sans dégradation de la qualité, et fournissant ainsi une représentation uniforme de la parole en tant que composants AM-FM.

A.6 Systèmes d'analyse et de synthèse sur la pleine bande

Il est maintenant évident que le eaQHM peut gérer deux catégories de discours, exprimé et non voisées. Cependant, il faut prendre soin dans les détails de la mise en oeuvre d'un tel système, puisque la stabilité, la robustesse et la cohérence ne sont pas seulement souhaitable pour l'analyse et la synthèse, mais aussi des modifications. Les détails d'une analyse basée sur eaQHM pleine bande et système de synthèse seront présentés prochaine. Le signal pleine bande est décrite comme une décomposition AM-FM :

d(t) = K � k=-K A k (t)e jφ k (

A.6.1 Analyse

Dans un premier temps, un continue f 0 estimation initiale et pour tous les cadres est obtenue, noté f0 . Bien qu'il n'y ait pas de f 0 dans des cadres non voisées, une estimation approximative peut être utile pour l'initialisation. Ensuite, l'étape suivante consiste à prendre une harmonicité pleine bande pour obtenir une première estimation des amplitudes instantanées de toutes les harmoniques. En utilisant une fenêtre d'analyse Blackman w(t) centrée à t i et avec le soutien de [t i -T, t i + T ], où 2T est de 3 locale longueur des périodes de hauteur, un cadre de l'analyse la parole est d'abord modélisé à l'aide d'une simple harmonique modèle comme :

d(t) = � L � k=-L a k e j2π

A.6.2 Adaptation

Le modèle ci-dessus est encore harmonique et fixe à l'intérieur d'un cadre d'analyse. Par conséquent, afin de converger vers quasi-harmonicité et d'affronter la question de stationnarité, la projection du signal sur un ensemble de variables dans le temps des fonctions de base est suggéré dans [START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF], en utilisant des paramètres a k et b k de la Modèle quasi-harmonique (QHM) [START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF]. Cela donne le modèle eaQHM :

d(t) = � L � k=-L � a k + tb k �� Âk (t)

A.6.3 Synthèse

Le signal de parole peut être approximée par ses composantes variant dans le temps à partir de la dernière étape de l'adaptation :

d(t) = L � k=-L
Âk (t)e j φk (t) (A.68)

Un schéma de principe de l'algorithme est représenté dans la figure A.5.

La pleine bande aHM est similaire à la aHNM, si la partie sonore est omis, de sorte que le lecteur peut se référer à la section correspondante.

Synthesis Step

Sinusoidal Synthesis

1   i i ) ( ) ( ˆt t A k k  )) ( êxp( ) ( ˆt j t A k L L k k     FIGURE A
.5 -Schéma de principe du système eaQHM.

A.6.4 Évaluation

Dans cette section, les résultats seront affichés sur la resynthèse du signal de parole. Les deux mesures objectives et subjectives seront présentés. Une comparaison de l'état-of-the-art sera discutée ainsi. En raison de leur disponibilité, les modèles suivants seront sélectionnés pour la comparaison avec les modèles d'adaptation : Modèle sinusoïdale (SM), Modèle harmonique + bruit (HNM), et STRAIGHT. Dans ces expériences, une base de données de 32 énoncés de parole a été utilisé, dont 16 hommes et 16 femmes enceintes de 16 langues différentes : grec, franc ¸ais, anglais, espagnol, finnois, chinois, portugais, basque, japonais, italien, allemand, coréen, russe , arabe, indonésien et turc. Toutes les formes d'onde ont été échantillonnés à 16 kHz.

Dans une évaluation objective, la SRER est calculée pour l'ensemble de la forme d'onde. étant donné que seuls les modèles pleine bande tentent de reconstituer avec précision les signaux de parole, que ceux-ci seront pris en compte dans l'évaluation. Dans l'évaluation subjective, un test formel d'écoute a été menée afin de mesurer la qualité perc ¸ue. Dans cette évaluation, tous les modèles sont inclus : SM, HNM, STRAIGHT, aHM, aHNM, eaQHNM, eaQHM.

Dans le tableau A.4, la moyenne et l'écart type de les SRER pour tous les énoncés dans notre base de données sont présentées pour les deux estimateurs de hauteur. Deux estimateurs de hauteur différents (SWIPE and YIN) sont utilisés. Il est bien évident que la quasi-harmonicité peut capturer plus d'informations du signal de parole sous-jacente, avec le même nombre de paramètres de synthèse. Pour l'évaluation de

A.7 Modifications de la parole

Algorithmes pour le temps et pas mise à l'échelle ont été développés pour tous les modèles. Dans échelle de temps sinusoïdes en relation harmonique, l'interpolation de paramètres instantanés est nécessaire de modifier la vitesse d'articulation. Pour des amplitudes et des fréquences instantanées, ceci est effectué par l'intermédiaire linéaire et une interpolation spline, respectivement. Cependant, pour la phase, le processus n'est pas simple car il est dû à la rotation avance dans le temps les instants de temps. Par conséquent, il est proposé de supprimer cet effet en utilisant l'intégrale de kf 0 depuis le début du signal, et obtenir le phase relative -RP [START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF][START_REF] Kafentzis | Time-scale Modifications based on an Adaptive Harmonic Model[END_REF]. Ainsi, en supposant que la forme du signal est en train de changer sans à-coup, les valeurs de phase changent également en douceur d'une analyse instant de temps à l'autre. Ensuite, le RP peut être interpolée pour obtenir son homologue continu. En outre, une cannelure ou interpolation cubique est nécessaire tel que sa dérivée dans le temps, la fréquence, est toujours constante. De cette fac ¸on, le temps de la forme d'onde à l'échelle est la forme invariante.

Pour échelle de temps de la phase instantanée de sinusoïdes quasi-harmoniques, on peut suggérer de supprimer l'intégrale de f k (t) (au lieu de kf 0 (t)) de la k e trajectoire de phase instantanée, également dès le début du signal, en obtenant ainsi la phase relative qui est propriétaire de la majeure partie de l'aspect aléatoire, et l'ensemble de la forme d'impulsion glottale, et encore change de fac ¸on régulière à partir de l'instant d'un temps à l'autre. Cependant, cette approche de la problématique bien connue de de dispersion de phase, qui est, tandis que le signal proportionnel a la même teneur en fréquence, les phases entre les composants changent, résultant en une forme d'onde différente. Le problème de la dispersion de phase a déjà été abordée dans [START_REF] Quatieri | Shape-Invariant Time-Scale and Pitch Modifications of Speech[END_REF][START_REF] Federico | Waveform preserving time stretching and pitch shifting for sinusoidal models of sound[END_REF], entre autres. Une stratégie similaire sera suivie ici, en utilisant le concept de de retard de phase relative, proposée en [START_REF] Federico | Waveform preserving time stretching and pitch shifting for sinusoidal models of sound[END_REF].

Dans pitch-échelle, l'estimation d'un nouvel ensemble d'amplitude, la fréquence et les valeurs de phase est nécessaire en raison de pitch shift. Ces valeurs peuvent être obtenus par l'estimation de la dite amplitude et de phase des enveloppes dans le domaine spectral. La méthode Discrète All-Pole [START_REF] El-Jaroudi | Discrete All-Pole Modeling[END_REF] est utilisé dans ce travail pour les deux modèles. Pour la phase, une approche simple est proposé pour les systèmes basés sur aHM, ce qui implique le calcul et l'interpolation de la des phases relatives, tel que mentionné dans les sections de temps mise à l'échelle. Pour les systèmes basés sur eaQHM, la notion de des retards de phase par rapport est également utilisé afin de réduire la dispersion de phase.

Depuis les modifications de la part de bruit dans les systèmes hybrides ne s'appliquent que pour échelle de temps (en hauteur échelle, la partie sonore est laissé non modifié), nous allons présenter des modifications pour les systèmes à bande complète seulement.

A.8 Modifications basées sur aHM A.8.1 Temps-Echelle Modification

Pour une modification arbitraire d'échelle de temps, le temps t dans le signal d'origine est associé à un temps t � dans le signal modifié. Pour cela, une fonction de correspondance appelé la fonction de distorsion d'échelle de temps est défini : 1. Les amplitudes instantanées sont temps-échelle :

D(t) =
A � k (t � ) = A k (D -1 (t)) (A.71)
2. Afin de calculer φ � k (t � ), il est d'abord nécessaire de calculer les fréquences au temps. Les fréquences instantanées dans le signal modifié au moment de t � correspondent à la fréquence instantanée dans le signal original au moment D -1 (t � ) : kf � 0 (t � ) = kf 0 (D -1 (t � )) (A.72) où D -1 (t) est la fonction de distorsion d'échelle de temps inverse.

3. Enfin, pour obtenir une forme d'onde préservateur de forme, la phase relative (RP) les valeurs de l'analyse doivent être mis à l'échelle de temps. Par conséquent, nous calculons d'abord les temps à l'échelle PR continues, ∠ã k (t), à partir des valeurs correspondantes, ∠ã i k . Pour cela, le RP est d'abord calculée en extrayant l'intégrale de la fréquence de l'information de phase au moment de l'analyse instant t i a , comme dans l'équation (A.73) :

∠ã i k = ∠a i k -

Exemple d'application

Dans la figure A.8, la propriété d'invariance de forme de l'aHM est clairement démontré pour différents facteurs d'échelle de temps, 0.5, 1.5, et 2.5.

A.8.2 Pitch-Echelle Modification

Pour une modification arbitraire pitch-échelle, l'entrée f 0 (t) contour est mappé à un autre, f � 0 (t) = ρ(t)f 0 (t) dans le signal modifié, où ρ(t) est la fonction de facteur pitch-échelle. Lorsque ρ(t) > 1, puis les pas augmente, alors que l'inverse se produit quand ρ(t) < 1. Notez que pour un fixe ρ(t) = ρ, la modification de la hauteur est constante tout au long de la forme d'onde.

Dans le contexte de modèle harmonique adaptative, les paramètres de la partie déterministe doivent être transformées de la manière décrite ci-après. Rappelons que dans une fenêtre d'analyse centrée sur t i a , les composants instantanés {a i k , f

Exemple d'application

Dans la figure A.9, un exemple est montré pour différents facteurs pitch-échelle, 0.5, 1.5, et 2. A.9 Modifications basées sur eaQHM A.9.1 Temps-Echelle Modification Dans le contexte de modèle sinusoïdal adaptatif, les paramètres doivent être transformées de la manière décrite ciaprès. Supposons d'abord que, dans une fenêtre d'analyse centrée sur t i , les composants instantanés {A k (t i a ), f k (t i a ), φ k (t i a ), sont connus, et le composant trajectoires ont été calculées, à savoir A k (t), f k (t), φ k (t), à l'intérieur de ce cadre et vers le haut au centre de la trame suivante, t i+1 a . Ensuite, l'échelle de temps nécessite les étapes suivantes à effectuer.

1. Les amplitudes instantanées sont temps-échelle :

A � k (t � ) = A k (D -1 (t � )) (A.80)
2. Les fréquences instantanées dans le signal modifié au moment de t � correspondent à la fréquence instantanée dans le signal original au temps D -1 (t � ) :

f � k (t � ) = f k (D -1 (t � )) (A.81)
3. Les valeurs de phase instantanées de la phase fondamentale pour les trames de synthèse sont calculées : φ 0 (t j s ) = φ 0 (t j-1 s ) + β(φ 0 (t i a ) -φ 0 (t i-1 a )) (A.82) 4. Les valeurs de phase instantanées du k e fréquence pour les trames de synthèse sont calculées :

φ k (t j s ) = (τ j 0 + (τ i k -τ i 0 ))2πf j k (A.83) 5. Les courbes de phase instantanés φ k (t � ) sont calculées par intégration de la fréquence instantanée :

φ � k (t � ) = φ k (t j s ) + � t �

A.9.2 Pitch-Echelle Modification

Dans un contexte de modèle sinusoïdal adaptatif, les paramètres doivent être transformées de la manière décrite ciaprès. Supposons d'abord que, dans une fenêtre d'analyse centrée sur t i a , les composants instantanés {A k (t i a ), f k (t i a ), φ k (t i a ), sont connus, et les trajectoires de composants ont été calculés, c'est à dire A k (t), f k (t), φ k (t), à l'intérieur de ce cadre et vers le haut au centre de la trame suivante, t i+1 a . Alors, pas échelle nécessite les étapes suivantes à effectuer.

A.10 Applications

Le eaQHM est appliqué dans la modélisation musicale de son instrument, et par rapport au modèle standard et le sinusoïdale amortie sinusoïdale Modèle exponentielle (EDSM). En outre, la eaQHM est testé sur la reconnaissance et la classification de la parole émotionnelle, et comparé au modèle sinusoïdal à nouveau.

A.10.1 Modélisation des sons d'instrument de musique

Le but de l'expérience en ce qui concerne l'instrument des sons musicaux est de comparer la précision de la modélisation de la SM, EDS, et eaQHM pour un large éventail de sons d'instruments de musique, y compris de percussion et nonpercussive. Dans ce travail, la précision de la modélisation est mesurée par l'es srer locale et globale, calculée en utilisant l'équation (A.92) : SRER = 20 log 10 σ x(t) σ x(t)-x(t) (A.92)

Le SRER locale est mesurée sur une fenêtre juste avant le début d'évaluer l'étalement de l'attaque également connu sous le nom de pré-écho, un artefact très fréquent chez les modèles qui utilisent des sinusoïdes sinusoïdale quasi-stationnaires. Le SRER mondiale mesure la précision globale de la modélisation, de prendre toute son compte.

La précision de la modélisation dépend du nombre de partiels K et la taille de la fenêtre L pour les algorithmes sinusoïdales. Traditionnellement, les partiels ne sont pas supposés varier beaucoup à l'intérieur de la fenêtre d'analyse, et donc sont modélisés avec des sinusoïdes quasi-stationnaires dont les paramètres sont en moyenne sur la fenêtre, le changement entre les fenêtres en fonction de la taille de pas. D'autre part, en supposant que chaque sinusoïde une capture partielle, il existe un nombre minimum de sinusoïdes nécessaires pour représenter l'énergie d'oscillation dans un instrument de musique retentit. Ainsi, nous allons présenter une comparaison de l'es srer en fonction de K et L pour le SM, EDS, et eaQHM. Tout d'abord, nous décrivons l'instrument de musique modélisée sons et la sélection de valeurs de paramètres pour des algorithmes.

Au total, les sons d'instruments 90 musicales ont été utilisées dans ce travail1 .

Configuration expérimentale

L'estimation des paramètres de la SM suit [START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF] avec interpolation de phase par splines cubiques. L'estimation des paramètres pour EDS utilisée ici est décrite en détail ailleurs [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF], alors que l'estimation du nombre optimal de pôles (sinusoïdes) [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF] est utilisé pour la comparaison. Dans toutes les expériences, le seuil de convergence es srer est défini sur 0.01, la taille de la FFT est N = 4096 échantillons, et la fréquence d'échantillonnage pour tous les sons est f s = 16 kHz. La taille de l'étape de la SM et eaQHM était H = 1 ms (ce qui correspond à 16 échantillons). EDS estime les paramètres séparément pour les cadres non-chevauchement (fenêtre rectangulaire) de taille L. Pour chaque étape d'estimation de paramètres, EDS décompose le cadre de L/2 et utilise une taille de l'échantillon hop H = 1 lors de la construction de la matrice de Hankel [START_REF] Derrien | Parametric audio coding with exponentially damped sinusoids. Audio, Speech, and Language Processing[END_REF]. Avant de modélisation avec le SM, EDS, et le eaQHM, la fréquence fondamentale f 0 de tous les sons a été estimée à l'aide SWIPE [START_REF] Camacho | A sawtooth waveform inspired pitch estimator for speech and music[END_REF] parce que dans ce travail, la taille de la fenêtre L et le nombre maximum de sinusoïdale partiels K max en supposant harmonicité dépendent f 0 .

Résultats

Le nombre de (sinusoïdale) partiels K est un paramètre d'entrée importante qui influe directement sur la précision de la modélisation de la SM, EDS, et le eaQHM. Nous avons couru chaque algorithme avec différents numéros de partiels comme paramètre d'entrée (la taille de la fenêtre a été maintenue à L = 3T 0 ) et enregistré les valeurs es srer locales et globales résultant. Nous avons commencé à partir de K max et une diminution de K par 2 partiels. Nous nous attendions à la SM de converger rapidement vers une valeur maximale et à stabiliser à cause de l'algorithme de sélection de paramètre. La littérature sur EDS [START_REF] Badeau | Selecting the modeling order for the esprit high resolution method: an alternative approach[END_REF] suggère qu'il existe un nombre optimal de partiels pour l'audio, donc nous nous attendions à EDS de rendre une courbe qui devrait atteindre un maximum autour de ce point, puis diminuer. Enfin, l'adaptation permet la eaQHM pour représenter les petites variations temporelles telles que les phénomènes transitoires précisément que les modulations d'amplitude et de la fréquence des partiels (peu importe le nombre de partiels). Par conséquent, nous nous attendions à ce qui a donné le eaQHM valeurs plus élevées de es srer comme le nombre de partiels augmentée.

Table A.5 montre la différence de SRER moyenne entre eaQHM et EDS et eaQHM et SM pour les instruments de musique regroupés en familles. La rangée du bas montre la moyenne de tous les instruments marqués total. Les colonnes marquées locale et global présente la différence entre K, tandis que la colonne intitulée K max montre la différence de SRER global que pour le nombre maximum de partiels. S'il vous plaît noter que la taille de la fenêtre est constante à 3T 0 . La taille de la fenêtre L affecte également directement la précision de la modélisation de la SM, EDS, et la eaQHM. Nous avons couru chaque algorithme variable L de 3T 0 à 8T 0 avec un nombre constant de partiels K max et mesuré l'es srer local et global qui en résulte. Nous nous attendions à L un impact négatif sur les trois algorithmes différemment. Nous nous attendions à L pour avoir un plus grand impact sur le SM parce que les deux l'estimation des paramètres et la résolution temporelle dépendent de L. Nous nous attendions à L pour avoir un impact plus faible sur la précision de la modélisation pour EDS en raison de l'amplitude variant dans le temps des sinusoïdes localement stationnaires (en dépit de la valeur de fréquence constante à l'intérieur de la fenêtre.) Enfin, nous conjecturé que L aura un mineur effet sur la eaQHM affecte principalement parce L de la capacité de la eaQHM pour capturer des modulations d'amplitude et de fréquence à l'intérieur de la fenêtre.

Table A.6 montre la différence de es srer moyenne entre eaQHM et EDS et eaQHM et SM. Le locales et globales colonnes présentent la différence moyenne de es srer sur L, tandis que la colonne 3T 0 montre la différence de SRER global pour la taille de la fenêtre qui donne la meilleure précision de la modélisation pour tous les algorithmes. Le SRER local est utilisé pour évaluer la précision de la modélisation apparition et es srer mondiale évalue la performance générale. La colonne 3T 0 sera utilisé comme mesure de référence par famille d'instruments de musique et la ligne total comme référence par modèle. Enfin, le nombre d'adaptation est étudiée pour la eaQHM. On observe que les augmentations de SRER est rapidement après quelques adaptations, convergeant lentement à une valeur finale de plusieurs ordres de grandeur plus élevé qu'avant l'adaptation.

Ce travail a proposé d'utiliser un modèle sinusoïdal adaptatif surnommé eaQHM pour représenter un instrument de musique à percussion et nonpercussive sonne comme sinusoïdes modulées en amplitude et en fréquence. En général, la eaQHM rend encore représentation compacte de haute qualité avec des paramètres intuitifs. Le modèle représente onsets bien nettes avec transitoires d'attaque, spectres inharmoniques, et même les bruits mécaniques.

Dans l'ensemble, les résultats ont montré que la eaQHM surpasse EDS et SM en moyenne, dans les deux variantes dans tous les cas, sauf pour l'es srer locale sous EDS variation de taille de la fenêtre. Considérant que la taille de la fenêtre et le nombre de partiels dont les algorithmes effectuent leur mieux, le eaQHM a toujours dépassé EDS de plus de 10 dB et SM de plus de 25 dB en moyenne.

A.10.2 Analyse et Classification de la parole expressive

Discours émouvant (ou stressé) peut être définie comme le style de la parole produite par un haut-parleur chargé d'émotion. Ces styles de parole peuvent être caractérisées comme heureux, triste, en colère, neutre et peur discours, entre autres. Analyse du discours émotionnel pourrait fournir des informations sur l'état émotionnel du locuteur, qui peut être utile dans des applications telles que les soins de santé et d'urgence conditions, et est une étape de pré-traitement nécessaire dans des applications telles que la reconnaissance et la classification. En outre, les systèmes de reconnaissance du locuteur et de vérification pourraient bénéficier d'une telle analyse, ainsi que des applications de synthèse de la parole, comme la sélection de l'unité en fonction de synthèse texte-parole ou de la synthèse de la parole basé HMM.

Dans ce travail, le modèle etendu adaptatif quasi-harmonique (eaQHM) est utilisé pour démontrer sa capacité à analyser, Resynthesize, et de classer les discours émouvant. Le corpus de parole pour l'analyse et la resynthèse est un de haute qualité, à large bande base de données contenant de la parole de fonctionnement émotionnel. Tests d'écoute subjectifs ont été menées pour prouver la transparence du discours resynthétisé. Il est également montré que eaQHM peut efficacement modéliser tous les styles de discours émouvant dans cette base de données avec une grande précision, et ceci est démontré par le rapport SRER valeurs, par rapport à la SM standard. En outre, une tâche de classification de l'émotion est présenté selon le célèbre discours Sous simulé et le stress réel (SUSAS) [START_REF] Hansen | Getting started with SUSAS: A speech under simulated and actual stress database[END_REF] base de données, dans lequel il ya 11 pré-étiquetés parole émotionnelle corpus. Les résultats montrent que les caractéristiques sinusoïdales du eaQHM donnent des résultats plus élevés que ceux de la classification de la SM.

Le système de classification est la suivante : une tâche de classification basé sur un 128 bits de quantification vectorielle (VQ) a été conc ¸u en utilisant un corpus de sous-ensemble des SUSAS, étiqueté comme Fâché, Neutre, Souple, et Question. Un nombre total de 2520 des formes d'onde (630 par l'émotion) ont été utilisés. Un certain nombre de 756 formes d'onde ont été conservés pour tester (189 par l'émotion), tandis que le reste a été utilisé pour la formation. Tous les signaux à temps discret ont été normalisées à l'énergie de l'unité

x[n] = x[n] � � L-1 n=0 x 2 [n]
(A.93) où L est la longueur du signal dans les échantillons. Les deux modèles ont utilisé un taux de trame d'analyse de 2.5 ms.

Les 10 fortes composantes du spectre de la FFT et d'amplitude des 10 amplitudes les plus élevées sinusoïdaux fournis par les LS, avec leurs fréquences correspondantes, ont été extraits à partir de chaque trame d'analyse. La fenêtre d'analyse a été fixé à 30 ms pour le SM, et à 3 périodes de hauteur locales pour la eaQHM. Aucune distinction entre les parties vocales et non vocales discours a été prononcé dans ce travail.

Classification -Caractéristique unique

Dans un premier temps, deux tâches de classification ont été définis, chacun utilisant différentes caractéristiques (amplitudes et des fréquences). Vecteurs ayant M spectrales x i contenant les fonctions sélectionnées (amplitudes ou fréquences), la matrice de données X est créé comme

X = [x 1 x 2 • • • x M ] (A.

Classification -Caractéristiques combinés

Depuis classification unique fonction basée conduit à de faibles scores de classification, un système de classification combiné est proposé. Les annonces provenant de VQS amplitude et de fréquence sur la base sont normalisées par la plus haute AD correspondant. Ensuite, les annonces des émotions correspondantes sont ajoutés. Enfin, l'émotion avec la somme minimum de consignes est sélectionné comme l'émotion reconnue. De cette fac ¸on, lorsque les VQs ont décidé autrement, le VQ qui est plus "confiance" dans sa décision (AD minimum est beaucoup moins que d'autres annonces) peuvent influencer le résultat final.

Table A.9 présente les résultats de classification correspondantes pour eaQHM et SM en utilisant le schéma proposé. L'utilisation de ce système, en moyenne, la eaQHM classe correctement 65% des énoncés dans la base de

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Parameter estimation for aQHM. Upper panel: Amplitude (left) and Frequency (right) estimation for first component. Lower panel: Amplitude (left) and Frequency (right) estimation for second component. 2.6 Parameter estimation for eaQHM. Upper panel: Amplitude (left) and Frequency (right) estimation for first component. Lower panel: Amplitude (left) and Frequency (right) estimation for second component. 2.7 Upper Panel: Original signal. Middle panel: aQHM (left) and eaQHM (right) reconstructed signal. Lower panel: aQHM (left) and eaQHM (right) reconstruction error. . . . . . . . . . . . . . . . . . . . 2.8 Illustration of the Adaptive Iterative Refinement -AIR algorithm of aHM. . . . . . . . . . . . . . . . . 3.1 The Short Time Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 STRAIGHT-based Analysis, Modification, and Synthesis. . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Real signal TD-PSOLA pitch-scaling. Upper panel: Original waveform. Lower panel: Pitch-scale modified waveform. The pitch-scale modification factor is 2. . . . . . . . . . . . . . . . . . . . . . . . 3.4 Pitch-scale modification scheme with TD-PSOLA method. Upper panel: Original waveform. Middle panel: Three short time synthetic signals. Lower panel: Pitch-scale modified waveform. . . . . . . . . . 3.5 Time-scale modification with TD-PSOLA method. Upper panel: Original waveform. Middle panel: Three short time synthetic signals. Lower panel: Time-scale modified waveform. Time-scale modification factor is 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.6 Comparison between phase-vocoder, STRAIGHT, and TD-PSOLA time-scaling. First panel: Origial waveform. Second panel: Time-scaled signal obtained by the TD-PSOLA technique. Third panel: Timescaled signal obtained by the STRAIGHT method. Fourth panel: Time-scaled singal obtained by the the phase-vocoder technique. Time-scale modification factor is 2. . . . . . . . . . . . . . . . . . . . . . . . 3.7 Sinusoidal Analysis, Modification, and Synthesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.8 Harmonic + Noise Analysis, Modification, and Synthesis. . . . . . . . . . . . . . . . . . . . . . . . . . 3.9 The LF-model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 A flowchart for the analysis and synthesis part of a general hybrid system framework. Upper panel: Analysis part. Lower part: Synthesis part. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 extended adaptive Quasi-Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 4.3 extended adaptive Quasi-Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 adaptive Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker. . . . . . . . . 4.5 adaptive Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker. . . . . . . . 4.6 An example of modeling the noise part of a speech signal using (a) time-and-frequency modulated noise and (b) sample by sample lattice filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 A flowchart for a generalized full-band speech analysis system. Upper panel: analysis part. Lower part: synthesis part. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Glottal pulses and corresponding magnitude spectra. . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Spectral analysis of speech. Black colored parts denote voiced speech, green colored parts denote unvoiced speech. Upper panel: Speech signal. Middle panel: DFT-based spectrogram. Lower panel: FChT-based spectrogram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Spectral analysis of voiced speech. Upper panel: FFT of a voiced speech segment. MVF denotes Maximum Voiced Frequency. Lower panel: Fan-Chirp Transform. . . . . . . . . . . . . . . . . . . . . . . . 4.11 Spectral analysis of unvoiced speech. First column: Unvoiced speech waveform, its FFT-based magnitude spectrum, and its FChT-based magnitude spectrum. Second column: FFT-based spectrogram slice of the corresponding waveform. Third column: FChT-based spectrogram slice of the corresponding waveform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.12 Estimated waveforms for a stop sound. Upper panel: Original (left) and SM (right) reconstruction. Lower panel: aQHM (left) and eaQHM (right) reconstruction. The red ellipses mark the region where pre-echo occurs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.13 Estimated waveforms for a fricative sound. Upper panel: Original signal. Middle panel: SM (left) reconstruction and eaQHM (right) reconstruction. Lower panel: SM (left) and eaQHM (right) reconstruction error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.14 extended adaptive Quasi-Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker. . . 4.15 extended adaptive Quasi-Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker. . 4.16 Block diagram of the eaQHM system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.17 adaptive Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker. . . . . . . . . . . . 4.18 adaptive Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker. . . . . . . . . . . 4.19 Speech utterance (/krOk h E/) in Korean language by a female subject. First panel: Original signal, Second panel: aHM reconstruction, Third panel: eaQHM reconstruction, Fourth panel: SM reconstruction, Fifth panel: STRAIGHT reconstruction, Sixth panel: HNM reconstruction. . . . . . . . . . . . . . . . . . . . 4.20 Analysis data of a Greek male speaker for both adaptive models: (a) aHM tracks, (b) eaQHM tracks, (c) Local SRER for both models over time, (d) Speech waveform. . . . . . . . . . . . . . . . . . . . . . . . 4.21 Example of the listening test page. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.22 Mean Opinion Score (MOS) of the resynthesis quality between the original recording and the reconstructions with all models, with the 95% confidence intervals. . . . . . . . . . . . . . . . . . . . . . . . . . 4.23 Gender-based Mean Opinion Score (MOS) of the resynthesis quality between the original recording and the reconstructions with all models, with the 95% confidence intervals. . . . . . . . . . . . . . . . . . . 5.1 A flowchart for the analysis, synthesis, and modifications part of (a) a full-band and (b) a general hybrid system framework. Upper panel: Analysis part. Middle part: Modifications part. Lower part: Synthesis part. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Adaptive Harmonic Model time scaling: Original signal (first panel) and time-scaled signals (lower panels) for factors of 0.5, 1.5 and 2.5, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Adaptive Harmonic Model time scaling spectra: Original signal spectrum (upper panel) and time-scaled signals spectra (lower panels) for factors of 0.5, 1.5 and 2.5, respectively. . . . . . . . . . . . . . . . . 5.4 adaptive Harmonic Model pitch scaling: Original signal (upper panel) and pitch-scaled signals (lower panels) for factors of 0.5, 1.5 and 2.0, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 adaptive Harmonic Model pitch scaling spectra: Original signal spectrum (upper panel) and pitchscaled signals spectra (lower panels) for factors of 0.5, 1.5 and 2.0, respectively. . . . . . . . . . . . . 5.6 extended adaptive Quasi-Harmonic Model time scaling: Original signal (upper panel) and time-scaled signals (lower panel) for factors of 0.5, 1.5 and 2.5, respectively. . . . . . . . . . . . . . . . . . . . . . 5.7 extended adaptive Quasi-Harmonic Model time scaling spectra: Original signal spectrum (upper panel) and time-scaled signals spectra (lower panel) for factors of 0.5, 1.5 and 2.5, respectively. . . . . . . . . 5.8 extended adaptive Quasi-Harmonic Model pitch scaling: Original signal (upper panel) and pitch-scaled signals (lower panel) for factors of 0.5, 1.5 and 2.0, respectively. . . . . . . . . . . . . . . . . . . . . . 5.9 extended adaptive Quasi-Harmonic Model pitch scaling spectra: Original signal spectrum (upper panel) and pitch-scaled signals spectra (lower panel) for factors of 0.5, 1.5 and 2.0, respectively. . . . . . . . 6.1 Example of how adaptation increases the modeling accuracy. Plot of SRER as a function of number of adaptations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Comparison between global and local SRER as a function of the the number of partials (a,b) and the size of the window (c,d) for the three models (SM, EDS, and eaQHM). . . . . . . . . . . . . . . . . . . . . 6.3 Illustration of the signal decomposition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Result of the listening test. The figure shows the mean opinion score (MOS) and 95 % confidence interval for the four sinusoidal models tested. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 Impairment evaluation of the resynthesis quality, with the 95% confidence intervals. . . . . . . . . . . . 7.2 An example of analysis of emotional speech: First panel, neutral speech. Second panel, angry speech. Third panel, f 0 (t) tracks for each sample. Fourth panel, A 0 (t) tracks for each sample. . . . . . . . . . 7.3 An example of emotional speaking styles, in time and frequency: First panel, neutral. Second panel, angry. Third panel, soft. Fourth panel, question. The word "Point" is depicted in this example. . . . . . 7.4 The proposed classification scheme based on the combination of features. A k and f k denote the instantaneous amplitude and frequency components, and ADs denote the average distortion measures. . . . . A.1 à l'intérieur de la fenêtre d'analyse, la trajectoire de la fréquence d'une partie (ligne grise solide) est supposé être constant pour des modèles sinusoïdaux stationnaires (ligne en pointillé), tandis que eaQHM (ligne en pointillés) s'adapte à la forme de manière itérative de la fréquence instantanée. . . . . . . . . A.2 à l'intérieur de la fenêtre d'analyse, la trajectoire d'amplitude d'une partie (ligne grise solide) est supposé être constant pour des modèles sinusoïdaux stationnaires (ligne en pointillé), tandis que eaQHM (ligne en pointillés) s'adapte de manière itérative à la forme de l'amplitude instantanée. . . . . . . . . A.3 Modèle étendu adaptatif quasi-harmonique + bruit : Signal original (premier panneau) et le signal reconstruit (deuxième panneau) avec leurs spectrogrammes correspondants (troisième et quatrième panneaux) pour un locuteur masculin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Modèle étendu adaptatif quasi-harmonique : Signal original (premier panneau) et le signal reconstruit (deuxième panneau) avec leurs spectrogrammes correspondants (troisième et quatrième panneau) pour un locuteur masculin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.5 Schéma de principe du système eaQHM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.6 Mean Opinion Score (MOS) de la qualité de resynthèse entre l'enregistrement original et les reconstructions avec tous les modèles, avec les intervalles de confiance 95%. . . . . . . . . . . . . . . . . . . . . A.7 Comparative entre les sexes Mean Opinion Score (MOS) de la qualité de resynthèse entre l'enregistrement original et les reconstructions avec tous les modèles, avec les intervalles de confiance 95 %. . A.8 échelle de temps du Modèle harmonique adaptatif : Signal d'origine (en haut) et les signaux de temps échelle (panneaux inférieurs) pour les facteurs de 0, 5, 1, 5 et 2, 5, respectivement. . . . . . . . . . . . . A.9 hauteur-échelle du modèle harmonique adaptatif : le signal d'origine (en haut) et les signaux de hauteuréchelle (panneaux inférieurs) pour les facteurs de 0.5, 1.5 et 2.0, respectivement. . . . . . . . . . . . . A.10 Modèle étendu adaptatif quasi-harmonique échelle de temps : le signal d'origine (en haut) et les signaux de temps échelle (panneau inférieur) pour les facteurs de 0.5, 1.5 et 2.5, respectivement. . . . . . . . . A.11 Modèle étendu adaptatif quasi-harmonique échelle de hauteur : le signal d'origine (en haut) et les signaux de hauteur-échelle (panneau inférieur) pour les facteurs de 0.5, 1.5 et 2.0, respectivement. . . . . Chapter 1

  (a) The human vocal tract (from [GL18]). (b) The vocal folds area (from [GL18]).

Figure 1 . 1 :

 11 Figure 1.1: Anatomy of the human speech production system.

Figure 1 . 2 :

 12 Figure 1.2: An example of sinusoidal analysis. Upper panel: the speech signal. Lower panel: the decomposed frequencies present in the signal.

Figure 1 . 3 :

 13 Figure 1.3: Contribution of this thesis with respect to some well known speech processing research fields.

2 .

 2 In Part II: Speech Analysis, Synthesis, and Modifications (a) Chapter 3 briefly presents the related work in the field of speech analysis, synthesis, and modifications. The most important systems are categorized and described. (b) Chapter 4 describes analytically the suggested analysis, synthesis, and modification systems, and compares them with other, well-known, state-of-the-art modification systems. Objective and subjective results are presented. 3. In Part III: Applications (a) Chapter 5 presents applications of the newly proposed systems in musical instrument sound analysis. (b) Chapter 6 discusses the application of the aSMs in expressive speech analysis and classification.

  After applying frequency correction

Figure 2 . 1 :

 21 Figure 2.1: Frequency correction with QHM. Upper panel: Frequency estimation without frequency correction. Lower panel: Frequency estimation after frequency correction. Red dashed line denotes the estimated spectrum, and black solid line denotes the original spectrum of a speech frame.

Figure 2 . 2 :

 22 Figure 2.2: Illustration of the adaptation of the frequency trajectory of a sinusoidal partial inside the analysis window in aQHM. The figure depicts the first and second iterations of aQHM, showing local adaptation as iterative projection of the original waveform onto the model.

Figure 2 . 3 :

 23 Figure 2.3: Inside the analysis window, the frequency trajectory of a partial (solid grey line) is assumed to be constant for stationary sinusoidal models (dotted line), while eaQHM (dashed line) iteratively adapts to the shape of the instantaneous frequency.

Figure 2 . 4 :

 24 Figure 2.4: Inside the analysis window, the amplitude trajectory of a partial (solid grey line) is assumed to be constant for stationary sinusoidal models (dotted line), while eaQHM (dashed line) iteratively adapts to the shape of the instantaneous amplitude.

2 ,Figure 2 . 5 :

 225 Figure 2.5: Parameter estimation for aQHM. Upper panel: Amplitude (left) and Frequency (right) estimation for first component. Lower panel: Amplitude (left) and Frequency (right) estimation for second component.

Figure 2 . 6 ,

 26 Figure2.6, the same information is depicted for the eaQHM algorithm. As it can be seen in Figures 2.5 and 2.6, the eaQHM performs better than aQHM in the estimation of the time varying frequencies and, especially, of the time varying amplitudes.To test the robustness of the estimations provided by the eaQHM, additive white Gaussian noise of 20 and 10 dB SNR was added to the synthetic signal x(t) described above. For comparison purposes, results for the aQHM are also provided. The performance of the algorithms is measured through the Mean Absolute Error (MAE) for amplitudes and frequencies. The MAE of a parameter θ is defined as:

Figure 2 . 6 :

 26 Figure 2.6: Parameter estimation for eaQHM. Upper panel: Amplitude (left) and Frequency (right) estimation for first component. Lower panel: Amplitude (left) and Frequency (right) estimation for second component.

Figure 2 . 7 :

 27 Figure 2.7: Upper Panel: Original signal. Middle panel: aQHM (left) and eaQHM (right) reconstructed signal. Lower panel: aQHM (left) and eaQHM (right) reconstruction error.

  Using the frequency correction mechanism, the f0 and all its multiples are refined. A few more harmonics are considered, up to the mid-band. Using the frequency correction mechanism, the f0 and all its multiples are refined. All the harmonics are considered, up to the full-band.

  A final refinement of the f0 is obtained and consequently all harmonics are well localized.

Figure 2 . 8 :

 28 Figure 2.8: Illustration of the Adaptive Iterative Refinement -AIR algorithm of aHM.

Figure 3 . 1 :

 31 Figure 3.1: The Short Time Fourier Transform

Figure 3 . 3 :

 33 Figure 3.3: Real signal TD-PSOLA pitch-scaling. Upper panel: Original waveform. Lower panel: Pitch-scale modified waveform. The pitch-scale modification factor is 2.

Figure 3 . 4 :

 34 Figure 3.4: Pitch-scale modification scheme with TD-PSOLA method. Upper panel: Original waveform. Middle panel: Three short time synthetic signals. Lower panel: Pitch-scale modified waveform.

Figure 3

 3 Figure 3.5: Time-scale modification with TD-PSOLA method. Upper panel: Original waveform. Middle panel: Three short time synthetic signals. Lower panel: Time-scale modified waveform. Time-scale modification factor is 2.

Figure 3 . 6 :

 36 Figure 3.6: Comparison between phase-vocoder, STRAIGHT, and TD-PSOLA time-scaling. First panel: Origial waveform. Second panel: Time-scaled signal obtained by the TD-PSOLA technique. Third panel: Time-scaled signal obtained by the STRAIGHT method. Fourth panel: Time-scaled singal obtained by the the phase-vocoder technique. Time-scale modification factor is 2.

Figure 3 . 7 :

 37 Figure 3.7: Sinusoidal Analysis, Modification, and Synthesis.

Figure 3 . 8 :

 38 Figure 3.8: Harmonic + Noise Analysis, Modification, and Synthesis.

Figure 3 . 9 :

 39 Figure 3.9: The LF-model.

Figure 4 .

 4 Figure 4.1: A flowchart for the analysis and synthesis part of a general hybrid system framework. Upper panel: Analysis part. Lower part: Synthesis part.

Figure 4 . 2 :Figure 4

 424 Figure 4.2: extended adaptive Quasi-Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker.

Figure 4 . 4 :

 44 Figure 4.4: adaptive Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker.

Figure 4

 4 Figure 4.5: adaptive Harmonic + Noise Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker.

Figure 4 . 6 :

 46 Figure 4.6: An example of modeling the noise part of a speech signal using (a) time-and-frequency modulated noise and (b) sample by sample lattice filtering

Figure 4 .

 4 10 we show one slice of voiced speech of each spectrogram in Figure 4.9. In standard hybrid speech analysis systems, a MVF separates the spectrum in the deterministic (left part of upper panel in Figure 4.10) and a stochastic (right part of upper panel in Figure 4.10) part. The former is mostly represented by a sum of sinusoids, and the latter is modelled by modulated noise. A careful inspection of corresponding Fan-Chirp

Figure 4 . 7 :

 47 Figure 4.7: A flowchart for a generalized full-band speech analysis system. Upper panel: analysis part. Lower part: synthesis part.

  0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05 0.055 0of series of pulses FFT of single pulse

Figure 4 . 8 :

 48 Figure 4.8: Glottal pulses and corresponding magnitude spectra.

  1

Figure 4 . 9 :

 49 Figure 4.9: Spectral analysis of speech. Black colored parts denote voiced speech, green colored parts denote unvoiced speech. Upper panel: Speech signal. Middle panel: DFT-based spectrogram. Lower panel: FChT-based spectrogram.

Figure 4 . 10 :

 410 Figure 4.10: Spectral analysis of voiced speech. Upper panel: FFT of a voiced speech segment. MVF denotes Maximum Voiced Frequency. Lower panel: Fan-Chirp Transform.

Figure 4 .

 4 Figure 4.11: Spectral analysis of unvoiced speech. First column: Unvoiced speech waveform, its FFT-based magnitude spectrum, and its FChT-based magnitude spectrum. Second column: FFT-based spectrogram slice of the corresponding waveform. Third column: FChT-based spectrogram slice of the corresponding waveform.

Figure 4 .

 4 Figure 4.12: Estimated waveforms for a stop sound. Upper panel: Original (left) and SM (right) reconstruction. Lower panel: aQHM (left) and eaQHM (right) reconstruction. The red ellipses mark the region where pre-echo occurs.

Figure 4 .

 4 Figure 4.13: Estimated waveforms for a fricative sound. Upper panel: Original signal. Middle panel: SM (left) reconstruction and eaQHM (right) reconstruction. Lower panel: SM (left) and eaQHM (right) reconstruction error.

Figure 4 .

 4 Figure 4.14: extended adaptive Quasi-Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker.

Figure 4 .

 4 Figure 4.15: extended adaptive Quasi-Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker.

Figure 4 .

 4 Figure 4.16: Block diagram of the eaQHM system.

Figure 4 .

 4 Figure 4.17: adaptive Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a male speaker.

Figure 4 .

 4 Figure 4.18: adaptive Harmonic Model: Original signal (first panel) and reconstructed signal (second panel) along with their corresponding spectrograms (third and fourth panel) for a female speaker.

Figure 4 . 19 :

 419 Figure 4.19: Speech utterance (/krOk h E/) in Korean language by a female subject. First panel: Original signal, Second panel: aHM reconstruction, Third panel: eaQHM reconstruction, Fourth panel: SM reconstruction, Fifth panel: STRAIGHT reconstruction, Sixth panel: HNM reconstruction.

Figure 4 . 20 :

 420 Figure 4.20: Analysis data of a Greek male speaker for both adaptive models: (a) aHM tracks, (b) eaQHM tracks, (c) Local SRER for both models over time, (d) Speech waveform.

Figure 4 .

 4 Figure 4.21: Example of the listening test page.

Figure 4 .

 4 Figure 4.22: Mean Opinion Score (MOS) of the resynthesis quality between the original recording and the reconstructions with all models, with the 95% confidence intervals.

Figure 4 .

 4 Figure 4.23: Gender-based Mean Opinion Score (MOS) of the resynthesis quality between the original recording and the reconstructions with all models, with the 95% confidence intervals.

Figure 5

 5 Figure 5.1: A flowchart for the analysis, synthesis, and modifications part of (a) a full-band and (b) a general hybrid system framework. Upper panel: Analysis part. Middle part: Modifications part. Lower part: Synthesis part.

Figure 5 . 2 :

 52 Figure 5.2: Adaptive Harmonic Model time scaling: Original signal (first panel) and time-scaled signals (lower panels) for factors of 0.5, 1.5 and 2.5, respectively.

Figure 5 . 3 :

 53 Figure 5.3: Adaptive Harmonic Model time scaling spectra: Original signal spectrum (upper panel) and time-scaled signals spectra (lower panels) for factors of 0.5, 1.5 and 2.5, respectively.

Figure 5 . 4 :

 54 Figure 5.4: adaptive Harmonic Model pitch scaling: Original signal (upper panel) and pitch-scaled signals (lower panels) for factors of 0.5, 1.5 and 2.0, respectively.

FrequencyFigure 5

 5 Figure 5.5: adaptive Harmonic Model pitch scaling spectra: Original signal spectrum (upper panel) and pitch-scaled signals spectra (lower panels) for factors of 0.5, 1.5 and 2.0, respectively.

Figure 5 . 6 :

 56 Figure 5.6: extended adaptive Quasi-Harmonic Model time scaling: Original signal (upper panel) and time-scaled signals (lower panel) for factors of 0.5, 1.5 and 2.5, respectively.

FrequencyFigure 5 . 7 :

 57 Figure 5.7: extended adaptive Quasi-Harmonic Model time scaling spectra: Original signal spectrum (upper panel) and time-scaled signals spectra (lower panel) for factors of 0.5, 1.5 and 2.5, respectively.

Figure 5

 5 Figure 5.8: extended adaptive Quasi-Harmonic Model pitch scaling: Original signal (upper panel) and pitch-scaled signals (lower panel) for factors of 0.5, 1.5 and 2.0, respectively.

FrequencyFigure 5 . 9 :

 59 Figure 5.9: extended adaptive Quasi-Harmonic Model pitch scaling spectra: Original signal spectrum (upper panel) and pitch-scaled signals spectra (lower panel) for factors of 0.5, 1.5 and 2.0, respectively.

Figure 6 . 1 :

 61 Figure 6.1: Example of how adaptation increases the modeling accuracy. Plot of SRER as a function of number of adaptations.

Figure 6 . 2 :

 62 Figure 6.2: Comparison between global and local SRER as a function of the the number of partials (a,b) and the size of the window (c,d) for the three models (SM, EDS, and eaQHM).

Figure 6 . 3 :

 63 Figure 6.3: Illustration of the signal decomposition.

Figure 6 . 4 :

 64 Figure 6.4: Result of the listening test. The figure shows the mean opinion score (MOS) and 95 % confidence interval for the four sinusoidal models tested.

Figure 7 . 1 :

 71 Figure 7.1: Impairment evaluation of the resynthesis quality, with the 95% confidence intervals.

Figure 7 . 2 :

 72 Figure 7.2: An example of analysis of emotional speech: First panel, neutral speech. Second panel, angry speech. Third panel, f 0 (t) tracks for each sample. Fourth panel, A 0 (t) tracks for each sample.

Figure 7 . 3 :

 73 Figure 7.3: An example of emotional speaking styles, in time and frequency: First panel, neutral. Second panel, angry. Third panel, soft. Fourth panel, question. The word "Point" is depicted in this example.

Figure 7 Figure 7 . 4 :

 774 Figure 7.4: The proposed classification scheme based on the combination of features. A k and f k denote the instantaneous amplitude and frequency components, and ADs denote the average distortion measures.

  FIGURE A.1à l'intérieur de la fenêtre d'analyse, la trajectoire de la fréquence d'une partie (ligne grise solide) est supposé être constant pour des modèles sinusoïdaux stationnaires (ligne en pointillé), tandis que eaQHM (ligne en pointillés) s'adapte à la forme de manière itérative de la fréquence instantanée.

FIGURE A. 2 -

 2 FIGURE A.2à l'intérieur de la fenêtre d'analyse, la trajectoire d'amplitude d'une partie (ligne grise solide) est supposé être constant pour des modèles sinusoïdaux stationnaires (ligne en pointillé), tandis que eaQHM (ligne en pointillés) s'adapte de manière itérative à la forme de l'amplitude instantanée.

  FIGURE A.3 -Modèle étendu adaptatif quasi-harmonique + bruit : Signal original (premier panneau) et le signal reconstruit (deuxième panneau) avec leurs spectrogrammes correspondants (troisième et quatrième panneaux) pour un locuteur masculin.

FIGURE A. 4 -

 4 FIGURE A.4 -Modèle étendu adaptatif quasi-harmonique : Signal original (premier panneau) et le signal reconstruit (deuxième panneau) avec leurs spectrogrammes correspondants (troisième et quatrième panneau) pour un locuteur masculin.

FIGURE A. 6 -FIGURE A. 7 -

 67 FIGURE A.6 -Mean Opinion Score (MOS) de la qualité de resynthèse entre l'enregistrement original et les reconstructions avec tous les modèles, avec les intervalles de confiance 95%.

  τ ) > 0 est le taux de échelle de temps. Lorsque β(τ ) > 1, le taux d'articulation est ralentie, alors que l'inverse se produit quand β(τ ) < 1. Dans le contexte de modèle sinusoïdal adaptative, les paramètres doivent être transformées de la manière décrite ci-après. Rappelons que dans une fenêtre d'analyse centrée sur t i a , les composants instantanés {a i k , f i 0 }, sont connus. De ceux-ci, nous pouvons calculer leurs homologues continus, qui sont les amplitudes instantanées A k (t) = |a k (t)| et fréquences f 0 (t), obtenu par interpolation a i k et f i 0 , respectivement. Ensuite, la forme d'onde en temps-échelle, ŝT S (t � ) est donné par :ŝT S (t � ) = K � k=-K A � k (t � )e jφ � k (t � ) (A.70) où A � k (t � ) et φ � k (t �) sont calculés à l'aide de la manière suivante :

FIGURE A. 8 -

 8 FIGURE A.8échelle de temps du Modèle harmonique adaptatif : Signal d'origine (en haut) et les signaux de temps échelle (panneaux inférieurs) pour les facteurs de 0, 5, 1, 5 et 2, 5, respectivement.

  FIGURE A.9hauteur-échelle du modèle harmonique adaptatif : le signal d'origine (en haut) et les signaux de hauteuréchelle (panneaux inférieurs) pour les facteurs de 0.5, 1.5 et 2.0, respectivement.

0

  (2πf � k (u) + c(u))du (A.84)Ainsi, la version de son temps-échelle, s T S (t � ), pour un facteur d'échelle de temps constante est donnée par :ŝT S (t � ) = L � k=-L A � k (t � )e jφ � k (t � ) (A.85)Example of ApplicationDans la figure A.10, le temps-échelle à base de eaQHM est démontrée pour différents facteurs d'échelle de temps, 0.5, 1.5, et 2.5. La propriété d'invariance de forme est clairement démontrée.

FIGURE A. 10 -

 10 FIGURE A.10 -Modèle étendu adaptatif quasi-harmonique échelle de temps : le signal d'origine (en haut) et les signaux de temps échelle (panneau inférieur) pour les facteurs de 0.5, 1.5 et 2.5, respectivement.

3.+ � t 0 (FIGURE A. 11 -

 011 FIGURE A.11 -Modèle étendu adaptatif quasi-harmonique échelle de hauteur : le signal d'origine (en haut) et les signaux de hauteur-échelle (panneau inférieur) pour les facteurs de 0.5, 1.5 et 2.0, respectivement.

  94) Les dictionnaires sont alors conc ¸us basé sur la minimisation de la distorsion moyenne (Average Distortion -AD) entre les vecteurs de formation et les vecteurs de dictionnaire dans la matrice Y, où Y = [y 1 y 2

  eaQHM est évaluée et comparée à des systèmes à la pointe de la technologie, et leur est montré surpasser en termes de qualité de resynthèse, représentant avec succès l'attaque , transitoire, et une partie fixe d'un son d'instruments de musique. Enfin, une autre application est suggéré, à savoir l'analyse et la classification des discours émouvant. Le eaQHM est appliqué sur l'analyse des discours émouvant, offrant à ses paramètres instantanés comme des caractéristiques qui peuvent être utilisés dans la reconnaissance et la quantification vectorielle à base classification du contenu émotionnel de la parole. Bien que les modèles sinusoïdaux sont pas couramment utilisés dans ces tâches, les résultats sont prometteurs. �������� Η Ημιτονοειδής Μοντελοποίηση είναι μια από τις πιο ευρέως χρησιμοποιούμενες παραμετρικές μεθόδους για την επεξεργασία σήματος φωνής και ήχου. Η ακριβής εκτίμηση των ημιτονοειδών παραμέτρων (πλάτη, συχνότητες, και φάσεις) είναι ένα κρίσιμο σημείο για τη ακριβή αναπαράσταση των σημάτων που αναλύονται. Στην παρούσα εργασία, με βάση τις πρόσφατες εξελίξεις στην ημιτονοειδή ανάλυση, προτείνουμε υψηλής ανάλυσης, προσαρμόσιμα ημιτονοειδή μοντέλα για συστήματα ανάλυσης, σύνθεσης, και τροποποίησης ομιλίας. Στόχος μας είναι να προσφέρουμε συστήματα που αναπαριστούν σήματα φωνής με εξαιρετικά ακριβή και συμπαγή τρόπο. σχήματος, χωρίς τεχνικά ελαττώματα, και υψηλής ποιότητας. Τα αποτελέσματα δείχνουν ότι τα συστηματα βασισμένα στην αρμονικότητα προτιμούνται έναντι αυτών της σχεδόν-αρμονικότητας, λόγω της απλότητας της αναπραστάσης. Επιπλέον, το eaQHM εφαρμόζεται στο πρόβλημα της μοντελοποίησης σημάτων ήχου, και συγκεκριμένα ήχων μουσικών οργάνων. Το eaQHM αξιολογείται και σύγκρινεται με stateof-the-art συστήματα, και έχει υψηλές επιδόσεις όσον αφορά την ποιότητα επανασύνθεσης, αναπαριστωντας με επιτυχία τα στάδια της επίθεσης, της μετάβασης, και της στατικότητας ενός ήχου μουσικού οργάνου. Τέλος, μια άλλη προτεινόμενη εφαρμογή έγκειται στην ανάλυση και ταξινόμηση της εκφραστικής ομιλίας. Το eaQHM εφαρμόζεται στην ανάλυση της εκφραστικής ομιλίας, παρέχοντας τις στιγμιαίες παραμέτρους του ως χαρακτηριστικά που μπορούν να χρησιμοποιηθούν στην αναγνώριση και ταξινόμηση, βασισμένη σε διανυσματικούς κβαντιστές, εκφραστικής ομιλίας. Αν και τα ημιτονοειδή μοντέλα δεν χρησιμοποιούνται συνήθως σε τέτοιες εφαρμογές, τα αποτελέσματα είναι ελπιδοφόρα. 3.1.1 The Phase Vocoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 The Speech Transformation and Representation using Adaptive Interpolation of Weighted Spectrum (STRAIGHT) model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.3 The Overlap-Add (OLA) Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.4 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Parametric Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 The Sinusoidal Model (SM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 The Harmonic Plus Noise Model (HNM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 The LF+ARX model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4 Other Approaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Conclusions and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Valeurs SRER mondiaux (dB) pour tous les modèles sur une grande base de données d'arrêts. Occlusives voisées sont également inclus dans la présente à des fins de comparaison. étape désigne le taux de trame d'analyse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.3 Valeurs es srer (dB) pour tous les modèles sur une grande base de données de fricatives. étape désigne le taux de trame d'analyse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.4 Valeurs SRER (dB) pour tous les modèles sur une base de données de 32 énoncés (16 de locuteurs, 16 de locutrices) en utilisant SWIPE et YIN estimateurs de hauteur. Moyenne et l'écart type sont donnés. . . . A.5 Différence moyenne de SRER (dB) entre eaQHM et EDS ou SM à travers le nombre de partiels K. . . . A.6 Différence moyenne de SRER (dB) entre la eaQHM et EDS ou SM dans la taille de fenêtre L. . . . . . . A.7 Classification note (%) pour quatre émotions de la base de données SUSAS, l'utilisation des fonctions d'amplitude extraites de eaQHM et SM (entre parenthèses). . . . . . . . . . . . . . . . . . . . . . . . . A.8 Classification note (%) pour quatre émotions de la base de données SUSAS, l'utilisation des fonctions de fréquence extraites de eaQHM et SM (entre parenthèses). . . . . . . . . . . . . . . . . . . . . . . . A.9 Classification note (%) sur la base des amplitudes et des fréquences pour une classification 128 bits VQ de 4 émotions de la base de données normalisées SUSAS. . . . . . . . . . . . . . . . . . . . . . . . . . Publications over the years of the thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Anatomy of the human speech production system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 An example of sinusoidal analysis. Upper panel: the speech signal. Lower panel: the decomposed frequencies present in the signal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Contribution of this thesis with respect to some well known speech processing research fields. . . . . . . 2.1 Frequency correction with QHM. Upper panel: Frequency estimation without frequency correction. Lower panel: Frequency estimation after frequency correction. Red dashed line denotes the estimated spectrum, and black solid line denotes the original spectrum of a speech frame. . . . . . . . . . . . . . 2.2 Illustration of the adaptation of the frequency trajectory of a sinusoidal partial inside the analysis window in aQHM. The figure depicts the first and second iterations of aQHM, showing local adaptation as iterative projection of the original waveform onto the model. . . . . . . . . . . . . . . . . . . . . . . . 2.

	List of Figures
	1.1

Εμπνευσμένοι από πρόσφατα προταθέντα μοντέλα, όπως το προσαρμόσιμο Σχεδον -Αρμονικό Μοντέλο (aQHM) και το προσαρμόσιμο Αρμονικό Μοντέλο (aHM), διατυπώνουμε τη θεωρία της προσαρμόσιμης Ημιτονοειδούς Μοντελοποίησης και προτείνουμε ένα μοντέλο που ονομάζεται εκτεταμένο προσαρμόσιμο Σχεδον -Αρμονικό Μοντέλο (eaQHM), το οποίο είναι ένα μη παραμετρικό μοντέλο, ικανό να προσαρμόσει τα στιγμιαία πλάτη και φάσεις των συναρτήσεων βάσης του στα τοπικά χρονικά μεταβαλλόμενα χαρακτηριστικά του σήματος της φωνής, αμβλύνοντας έτσι τη γνωστή υπόθεση της τοπικής στασιμότητας. Αποδεικνύεται ότι το eaQHM παρουσιάζει υψηλότερες επιδόσεις από το aQHM στην ανάλυση και ανασύνθεση των έμφωνων τμημάτων φωνής. Με βάση το eaQHM, ένα υβριδικό σύστημα ανάλυσης / σύνθεσης ομιλίας παρουσιάζεται (eaQHNM), μαζί με μια υβριδική έκδοση του του aHM (aHNM). Επιπλέον, παρουσιάζουμε κίνητρα για μια αναπαράσταση του σήματος της φωνής σε όλο το φάσμο και σε όλη τη διάρκεια του, χρησιμοποιώντας το eaQHM, αναπαριστώντας έτσι όλα τα μέρη του σήματος της φωνής, με υψηλής ανάλυσης AM-FM ημίτονα. Η αξιολόγηση δείχνει ότι η προσαρμοσιμότητα και η σχεδόν-αρμονικότητα είναι αρκετή για να παράξει πολύ υψηλή ποιότητα στην ανασύνθεση των άφωνων τμημάτων της φωνής. Στη συνέχεια, παρουσιάζεται το σύστημα πλήρους φάσματος ανάλυσης και σύνθεσης βασισμένο στο eaQHM, το οποίο υπερτερεί συστημάτων που θεωρούνται state-of-the-art, υβριδικά ή πλήρους ανάλυσης, στην ανάλυση και ανασύνθεση φωνής. Η υπεροχή του στην ποιότητα ανασύνθεσης επιβεβαιώθηκε με αντικειμενικές και υποκειμενικές αξιολογήσεις. ΄Οσον αφορά τις εφαρμογές, το eaQHM και το aHM εφαρμόζονται σε μετασχηματισμούς φωνής (κλιμάκωση χρόνου και κλιμάκωση θεμελιώδους συχνότητας). Οι μετασχηματισμοί που προκύπτουν είναι υψηλής ποιότητας, ακολουθώντας πολύ απλούς κανόνες, σε σύγκριση με άλλα συστήματα state-of-the-art. Οι έννοιες της σχετικής φάσης και της καθυστέρησης σχετικής φάσης είναι ζωτικής σημασίας για την ανάπτυξη μετασχηματισμένου σήματος με χαρακτηριστικά αναλλοίωτου B.1

Table 2

 2 .1, and the sampling frequency is F s = 8 kHz, while the window length is

	Sinusoid	1st	2nd
	f i	700	1000
	φ i (t)	π 10 + cos(2π80t)	π 3 + cos(2π50t)
	a i (t)	2 + 0.8 cos(2π100)t 2 + 0.6 cos(2π100t)

Table 2

 2

.1: The parameters of the synthetic signal. 10 msec. It should be noted that the amplitudes of the signal components are high-frequency modulated and thus, the local amplitude linearity is violated. The time-varying amplitudes a i (t) and the time-varying frequencies

Table 2 . 2 :

 22 MAE scores and SRER for aQHM and eaQHM for 10 4 Monte Carlo simulations.

			MAE scores and SRER
	SNR	Model	a 1 (t)	a 2 (t)	F 1 (t)	F 2 (t)	SRER(dB)
	∞	aQHM 0.2380 0.1842 7.6105 9.1731 eaQHM 0.0889 0.0949 5.9217 7.0505	22.6 42.0
	20 dB	aQHM 0.2235 0.1735 7.2704 7.8563 eaQHM 0.1036 0.1079 6.1682 7.1241	18.2 20.3
	10 dB	aQHM 0.2317 0.1860 8.6071 9.0302 eaQHM 0.1490 0.1476 8.0513 8.1022	10.7 10.9

Table 2 .

 2 3 also presents the average number of adaptations (NoA) needed for the algorithms to converge. It is evident that, on average, eaQHM scores higher in terms of SRER, requiring, however,

		ARCTIC database evaluation
	Step	Method Mean (dB) Std (dB) NoA
	1 msec	aQHM eaQHM	34.5 35.8	4.6 5.7	2.9 3.8
	2 msec	aQHM eaQHM	31.0 33.2	4.0 5.0	3.5 3.9
	4 msec	aQHM eaQHM	30.8 32.8	3.4 4.6	3.6 6.1

Table 2

 2

.3: Mean and Standard Deviation of SRER (in dB) for approximately 50 minutes of voiced speech from the ARCTIC database.

  An analysis window h[n] is applied on

	speech	F0 extraction	Spectral Envelope extraction
				Analysis parameters
	Analysis part
	Modifications part
	Modification factors	F0 & Spectral Envelope modifications
	Variable filter		Source generation
		Resynthesis
	Synthesis part	speech
	Figure 3.2: STRAIGHT-based Analysis, Modification, and Synthesis.

  .

					Original signal
	Amplitude
	0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8
					TD-PSOLA
	Amplitude
	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6
					STRAIGHT
	Amplitude
	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6
					Phase Vocoder
	Amplitude
	0	0.2	0.4	0.6	0.8	1	1.2	1.4	1.6
					Time (s)

Table 4 .

 4

1: Global and Local Signal to Reconstruction Error Ratio values (dB) for all models on stop sound /t/.

Table 4 . 2 :

 42 high SRER values, is time consuming and is not realistic for applications. Hence, different frame rates are selected, namely 1ms, 2ms, and 4ms. Parameters other than the frame rate Global and Local Signal to Reconstruction Error Ratio values (dB) for all models on a small database of stops. Voiced stops are also included in this for comparison purposes.

	Small Scale Validation for Stop Sounds
	Global Signal to Reconstruction Error Ratio (dB)
	Model	/p/	/t/	/k/	/b/	/d/	/g/
	SM	13.5 14.6 13.4 17.2 15.3 17.6
	aQHM 20.8 23.2 23.2 28.9 27.9 28.2
	eaQHM 27.1 31.2 28.4 35.5 33.5 33.1
	Local Signal to Reconstruction Error Ratio (dB)
	Model	/p/	/t/	/k/	/b/	/d/	/g/
	SM	7.5	4.4	7.2 12.6 12.8 13.1
	aQHM 22.2 24.1 24.1 28.8 25.3 28.7
	eaQHM 29.0 33.7 29.4 35.7 36.7 35.3

Table 4 . 3 :

 43 .3 presents the results per phoneme, in terms of mean value of global SRER. Global Signal to Reconstruction Error Ratio values (dB) for all models on a large database of stops. Voiced stops are also included in this for comparison purposes.Step denotes the analysis frame rate.

		Large Scale Validation for Stop Sounds
		Global Signal to Reconstruction Error Ratio (dB)
	Step	Model	/p/	/t/	/k/	/b/	/d/	/g/
	1ms	SM eaQHM 25.4 25.7 27.2 32.9 32.2 32.9 12.7 12.8 12.4 16.6 14.9 15.3
	2ms	SM eaQHM 26.1 26.1 26.0 31.7 31.4 34.6 12.8 12.7 12.3 16.5 15.0 15.4
	4ms	SM eaQHM 23.7 24.2 24.4 29.4 29.5 30.9 12.9 12.6 12.2 16.7 15.0 15.3

Table 4 .

 4 5 presents the results per fricative, in terms of mean value of SRER.

			Large Scale Validation for Fricatives
			Signal to Reconstruction Error Ratio (dB)
	Step	Model	/v/	/D/	/s/	/S/	/f/	/T/	/z/	/Z/
	1 ms	SM eaQHM 26.4 25.6 24.1 26.4 25.8 24.3 29.5 28.9 14.7 13.2 13.9 11.3 12.7 15.1 17.5 17.3
	2 ms	SM eaQHM 23.5 23.1 22.6 24.7 23.5 22.6 28.6 27.8 13.1 11.3 12.1 10.4 10.2 14.7 15.9 15.2
	4 ms	SM eaQHM 22.4 22.2 21.9 23.1 22.6 21.7 27.5 27.1 12.2 10.6 11.2 9.6 9.7 8.9 13.3 13.7

Table 4 .

 4 5: Signal to Reconstruction Error Ratio values (dB) for all models on a large database of fricatives.Step denotes the analysis frame rate.

Table 4 . 6 :

 46 Signal to Reconstruction Error Ratio values (dB) for all models on a database of 32 utterances (16 of male speakers, 16 of female speakers) using SWIPE and YIN pitch estimators. Mean and Standard Deviation are given.

			SRER Performance
			Speakers
	Model		SWIPE	YIN
		Males	Females	Males	Females
	SM	18.6 (1.90) 18.6 (3.64) 14.3 (2.20) 16.2 (3.28)
	aHM	23.9 (2.66) 18.9 (3.27) 23.9 (2.61) 19.9 (3.05)
	eaQHM 34.5 (2.39) 30.9 (3.00) 34.4 (2.45) 30.7 (3.19)

  Algorithm 5 Pitch-scaling using eaQHM Require: A set of instantaneous parameters per analysis time instant: {A k (t i a ), f k (t i a ), φ k (t i a )}

	Compute pitch-scaled frequencies ρf k (t) using Eq. (5.41)
	Compute the spectral envelope DAP (t i a ) around time instant t i a .
	Sample the spectral envelope at the corresponding frequencies ρf k using Eq. (5.42)
	Interpolate instantaneous amplitudes over successive time instants t a i to obtain A � k (t) Estimate φ 0 (t j s ) using Eq. (5.43) Estimate φ k (t j s ) using τ j 0 and Eq. (5.44) Estimate φ � k (t) using Eq. (5.45)
	Pitch scaled speech
	Synthesize ŝP

S (t) using sinusoidal synthesis rean, Russian, Arabic, Indonesian, and Turkish. All waveforms were sampled at 16 kHz. Examples are available at http://www.csd.uoc.gr/ ˜kafentz/listest.html.

  Bass Trombone (nv, na, stac), Bass Trumpet (na, vib), Cimbasso (nv, na, stac), Contrabass Trombone (stac), Contrabass Tuba (na, stac), Cornet, French Horn (nv, na, stac), Piccolo Trumpet (nv, na, stac), Tenor Trombone (nv, vib, na, stac), C Trumpet (nv, na, stac), Tuba (vib, na, stac), Wagner Tuba na, stac)WoodwindsAlto Flute (vib, na), Bass Clarinet (na, sforz, stac), Bassoon (na, stac), Clarinet (na, stac), Contra Bassoon (sforz, stac), English Horn (na, stac), Flute (nv, vib, na, stac, tr), Oboe (na, stac), Piccolo Flute (nv, vib, na, stac, sforz)

	Strings	Cello (na, vib, pz), Double Bass (vib), Harp, Viola (na, nv, vib, stac, piz), Violin (na, nv, vib, stac, piz)
	Percussion	Glockenspiel (wo, me, pl), Marimba, Vibraphone (me, pl, bow), Xylophone (wo, me)

Popular

Accordion, Acoustic Guitar, Baritone Sax, Bass Harmonica, Chromatic Harmonica, Classic Guitar, Mandolin, Pan Flute, Tenor Sax, Ukulele Keyboard Celesta (na, nv, stac), Clavinet, Piano Table 6.1: Musical instrument sounds used in all experiments. See text in 6.2.1 for a description of the terms in brackets

Table 6 . 2 :

 62 K max Local Global K max Mean SRER difference (dB) between eaQHM and EDS or SM across the number of partials K.

	Brass	7.69	9.22	7.48	22.44	9.05	5.41
	Woodwinds	2.95	6.90	19.60	17.21	12.18	29.93
	Bowed Strings	6.32	3.45	21.52	14.46	6.06	31.21
	Plucked Strings	13.26	11.96	15.75	24.69	25.63	42.99
	Bowed Percus-sion	-3.89	2.00	21.80	2.65	10.56	37.63
	Struck Percus-sion	2.83	-1.52	11.88	7.68	2.77	21.06
	Popular	1.09	-0.44	11.93	3.55	-1.21	18.09
	Keyboard	6.33	4.41	3.15	16.99	11.87	24.79
	Total	4.55	4.50	14.14	14.15	9.07	26.39

Table 6 .

 6 Comparison between the local SRER 3: Mean SRER difference between the eaQHM and EDS or SM across the window size L.

		SRER(eaQHM-EDS)	SRER(eaQHM-SM)
		Local Global 3T 0	Local Global 3T 0
	Brass	-3.07	10.74	27.27 12.20	14.75	31.19
	Woodwinds	-2.81	6.92	18.21 11.63	13.43	30.44
	Bowed Strings	3.01	7.55	28.11 10.85	12.62	38.18
	Plucked Strings	-12.38 -15.89	7.78	21.16	17.57	49.88
	Bowed Percus-sion	-4.26	6.86	21.80 10.83	15.85	37.63
	Struck Percus-sion	8.13	5.85	10.82 15.24	11.73	19.25
	Popular	-0.91	5.57	15.25	9.00	9.02	21.91
	Keyboard	-4.25	-4.02	3.15	12.90	10.60	24.79
	Total	-2.30	2.95	16.55 12.46	12.78	31.66

variation for the SM in Figure (6.2b) and its global counterpart in Figure (a)

Table 6 . 4 :

 64 Comparison of model complexity for SM, EDS, and the eaQHM for the analysis and synthesis stages. The table presents the parameters (real numbers) to estimate (analysis complexity) and to represent (synthesis complexity) each sinusoid inside a frame.

Table 6

 6

			Synthesis
	SM	stationary stationary (OLA)
	HM	stationary	Splines
	aHM	adaptive	Splines
	eaQHM	adaptive	Splines

.5: Comparison of representations of frequency components for the analysis and synthesis stages of the sinusoidal algorithms used.

Table 6 . 6 :

 66 Musical instrument sounds used in the listening test.

		Brass	Woodwinds
	Double Bass Bass Trombone Bass Clarinet
	Cello	Bass Trumpet	Bassoon
	Viola	Cimbasso	Clarinet Bb
		Contrabass Tuba English Horn
		Tenor Trombone
		Tuba
		Wagner Tuba
	Table

Table 6 . 7 :

 67 Average Signal to Reconstruction Error Ratio (SRER) across musical instrument sounds.

	SM	HM	aHM eaQHM
	33.86 34.84 36.53	50.62

Table 7 . 1 :

 71 Signal to Reconstruction Error Ratio values (dB) for both models on a small acted speech database. Mean and Standard Deviation are given.

	.1.
		SRER Performance (Wideband Speech Database)
			Female Speaker
			Speaking Styles
	Model	Angry	Happy	Neutral	Sad
	SM	14.8 (1.36) 17.5 (3.0) 16.5 (1.36) 21.2 (1.64)
	eaQHM 28.8 (1.24) 33.1 (1.81) 34.9 (2.23) 34.8 (3.60)
			Male Speaker
	SM	17.0 (1.45) 14.3 (0.76) 16.0 (1.67) 16.5 (1.63)
	eaQHM 35.7 (2.04) 31.6 (3.49) 33.3 (2.56) 33.1 (2.74)

Table 7 . 2 :

 72 Signal to Reconstruction Error Ratio values (dB) for both models on the SUSAS database. Mean and Standard Deviation are given.

		SRER Performance (SUSAS)
			Speaking Styles
	Model	Angry	Loud	Clear	Fast
	SM	16.6 (3.06) 16.8 (3.01) 16.8 (3.06) 16.7 (3.03)
	eaQHM 32.3 (5.61) 32.8 (5.59) 32.6 (5.62) 32.9 (5.58)
		Question	Soft	Neutral	Slow
	SM	16.8 (3.00) 16.7 (3.05) 16.8 (3.01) 16.8 (3.05)
	eaQHM 32.8 (5.57) 32.9 (5.61) 32.9 (5.58) 32.9 (5.60)

Table 7

 7 .4. It can be seen that in both cases the angry speaking style stands

			VQ Classification in % -Amplitudes
				Predicted Class
			Angry Neutral	Soft	Question
		Angry	77(72)	14(14)	2(3)	7(11)
	Class	Neutral Soft	4(4) 3(5)	64(63) 31(30) 56(50) 18(18)	14(15) 10(15)
		Question	6(4)	21(22) 13(20)	60(55)

Table 7 .

 7 3: Classification score (%) for four emotions of the SUSAS database, using amplitude features extracted from eaQHM and SM (in parenthesis).out of the rest of speaking styles. This is expected since this speaking style is very different than the others in terms of amplitude and frequency distributions[START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF].

			VQ Classification in %-Frequencies
				Predicted Class
			Angry Neutral	Soft	Question
		Angry	71(70)	6(6)	7(5)	21(18)
	Class	Neutral Soft	6(6) 3(3)	55(38) 13(25) 65(59) 24(28)	15(27) 14(13)
		Question 17(18) 18(24) 14(25)	50

Table 7 .

 7 5: eaQHM and SM based Confusion Table in % based on amplitudes and frequencies for a 128-bit VQ classification between 4 emotions of the normalized SUSAS database.

			VQ combined classification in %
				Predicted Class
			Angry Neutral	Soft	Question
		Angry	83(77)	5(5)	1(5)	11(13)
	Class	Neutral Soft	15(4) 10(2)	58(48) 18(29) 56(54) 12(24)	15(24) 16(15)
		Question 20(17)	6(24)	11(21)	63(38)

  1) où ψ k (t) désigne l'ensemble des fonctions non-stationnaires base, C k (t) désigne le terme amplitude du modèle, 2K + 1 est le nombre d'exponentielles (d'où K + 1 sinusoïdes), et enfin w(t) est la fenêtre d'analyse avec le soutien de [-T, T ].Dans les modèles sinusoïdaux classiques, y compris le modèle sinusoïdal[START_REF] Mcaulay | Speech Analysis/Synthesis based on a Sinusoidal Representation[END_REF], le modèle harmonique (HM)[START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], et d'autres, l'ensemble des fonctions de base ψ k (t) dans la partie d'analyse est fixe en fréquence et en amplitude. Par exemple, les fonctions de base de la SM sont sous la forme de (t) est l'amplitude instantanée variant dans le temps de la k e de la fonction de base, φ k (t) est la phase instantanée de la k e de la fonction de base, calculée comme l'intégrale de la fréquence instantanée, f k (t).

	ψ SM k (t) = 1 • e j2πf k t , C SM k	(t) = a k	(A.2)
	où les amplitudes et des fréquences des fonctions de base sont constants à l'intérieur de la fenêtre d'analyse (1 et f k ,
	respectivement). Cependant, dans les ASMS, comme on le décrit dans les sections, amplitudes et de fréquences suivantes
	des fonctions de base sont non-paramétrique et dépend des caractéristiques réelles du signal analysé :
		ψ aSM k	(t) = α k (t) • e jφ k (t) ,	(A.3)
	où α k Le terme amplitude du modèle, C aSM k	(t), est variable dans le temps pour tous les aSMs. Plus précisément,

  8) où t i est toujours le centre de la fenêtre d'analyse et A k (t) est l'amplitude instantanée de la composant k e obtenu à partir d'une étape de l'adaptation (ou d'initialisation) précédente. L'estimation des paramètres inconnus du eaQHM est similaire à celle de QHM [PRS08] : est la matrice contenant les valeurs de la fenêtre dans la diagonale, s est le vecteur de signal d'entrée, la matrice E e est défini comme E e = [E e0 |E e1 ], et les sous-matrices E ei , i = 0, 1 n'ai éléments donné par

	�	â b�	= (E e	H W H WE e ) -1 E e	H W H Ws	(A.9)
	où W

  (t) est la trajectoire de la fréquence de la composant k e . Il est évident que les fonctions de base sont adaptées aux caractéristiques locales d'amplitude du signal. A noter que l'amplitude instantanée A k (t) est divisé par A k (t i ), de manière à avoir une valeur de l'unité au centre de la fenêtre d'analyse.En utilisant la définition de phase, la phase instantanée d'un seul composant, φ(t), est calculée comme l'intégrale de la fréquence instantanée, f (t). La fréquence instantanée est obtenue à partir d'une estimation de paramètre initial, tel que QHM. Pour interpoler les valeurs de phase entre deux instants successifs, t i , t i+1 , l'équation suivante est proposée : ) est l'estimation de la phase instantanée au temps instant t i . Cependant, cette solution ne prend pas en compte les conditions aux limites du cadre au moment de instant t i+1 . Par conséquent, il n'existe aucune garantie que la valeur de phase au temps instantanée t i+1 , En décrivant la modélisation de l'amplitude et de la fréquence de la eaQHM dans la fenêtre d'analyse, les figures A.1 et A.2 montrent comment les modèles sinusoïdaux classiques comme HM, SM, ou QHM se comportent dans leur fenêtre d'analyse . Leurs fonctions exponentielles de base sont à l'arrêt en fréquence, étant ainsi inefficace sur la représentation des courbes de fréquence fortement non stationnaires. Le même raisonnement s'applique pour les courbes d'amplitude, bien que la fréquence estimation est beaucoup plus importante que l'estimation d'amplitude.

	De cette fac ¸on, l'équation (A.14) est satisfaite si nous choisissons r(t i+1 ) comme
		r(t i+1 ) =	π(φ(t i+1 ) + 2πM -φ(t i+1 ) 2(t i+1 -t i )	(A.18)
	où M est calculée comme dans l'équation (A.15).
	frequency	analysis window	a t	Stationary estimation Real frequency Adaptive estimation Center of window
		a t			time
					k ,	(A.11)
	et la phase instantanée du k e composant peut être calculé comme
		φk (t) = φk (t i ) +	� ti+t	2πf k (u)du, t ∈ [-T, T ],	(A.12)
					ti
	où f k φ(t) = φ(t i ) +	� t+ti	2π f (u)du	(A.13)
					ti
	où φ(t i φ(t)	� �	t=ti+1 = φ(t i+1 ) + 2πM	(A.14)
	où M est un entier choisi de manière appropriée pour être aussi proche que possible de
		M = round	� φ(t i+1 ) -φ(t i ) 2π	�	(A.15)
	où round(•) est arrondi à l'entier le plus proche de la fonction. Afin d'assurer la phase poursuite sur les limites de trame,
	il est suggéré [PRS11] pour modifier l'équation (A.13). Comme :
		φ(t) = φ(t i ) +	� t+ti	(2π f (u) + c(u))du	(A.16)
					ti
	où c(u) est donnée par
		c(u) = r(t i+1 ) sin	� π(u -t i ) t i+1 -t i	�	(A.17)

  représente la déviation standard de x(t), x(t) est le signal réel et x(t) est le signal reconstruit, et où � est un seuil de convergence, généralement fixé à 0.02. Comme dernière étape de l'algorithme, le signal peut enfin être approximée comme la somme de ses éléments AM-FM :

	Algorithm 6 Décomposition AM-FM utilisant eaQHM
	Require: Fournir une estimation de la fréquence initiale f 0 k (t 1 )
	for cadre i = 1 to L do
	Calculer a i k , b i k utilisant LS Réviser f 0 k (t i ) utilisant Eq. (A.19) Calculer Â0 k (t i ) et φ0 k (t i ) utilisant Eq. (A.20) Définir f 0 k (t i+1 ) = f 0 k (t i )
	end for
	Interpolation des paramètres { Â0 k (t), f 0 k (t), φ0 k (t)}
	L'adaptation des amplitudes et des phases :
	for adaptation m = 1 to • • • do
	for cadre i = 1 to L do
	Calculer a i k , b i k utilisant φm-1 k Définir f m k (t i ) = f m k (t l-1 ) + ηk (t) de Eq. (A.16) et Eq. (A.9) Définir Âm k (t i ) = |a i k | et φm k (t i ) = ∠a i k
	end for
	end for
	Interpolation des paramètres { Âm k (t), f m k (t), φm k (t)}
	où le SRER m est le Ratio Erreur-Signal-sur-Reconstruction du signal re-synthétisée dans l'adaptation m e , définie
	comme	SRER = 20 log 10	σ x(t) σ x(t)-x(t)	(A.22)
	où σ x
				21)

  Table A.1 présente également le nombre moyen d'adaptations (NoA) nécessaire pour les algorithmes de convergence. Il est évident que, en Moyenne et deviation standard de SRER (en dB) pour environ 50 minutes de parole voisée de la base de données ARCTIC. moyenne, des scores plus élevés eaQHM en termes de SRER, nécessitant cependant un peu plus d'itérations que aQHM.

		évaluation de base ARCTIC
	étape	méthode valeur moyenne (dB) Std (dB) NoA
	1 msec	aQHM eaQHM	34.5 35.8	4.6 5.7	2.9 3.8
	2 msec	aQHM eaQHM	31.0 33.2	4.0 5.0	3.5 3.9
	4 msec	aQHM eaQHM	30.8 32.8	3.4 4.6	3.6 6.1
	TABLE A.1 -

  jωn (A.23) où h u [n] est la fenêtre d'analyse, et x[n] est le signal de parole. Avoir cette représentation temps-fréquence de la parole, l'étape de modification consiste à appliquer les modifications souhaitées au courant de l'analyse des spectres à court de temps, d'obtenir à court terme des spectres de synthèse, Y (t a (k), ω), et puis de les synchroniser sur une nouvelle série d'instants, appelé les des instants de synthèse, t s (k). Cependant, pour le temps et les modifications à l'échelle du terrain, il est commode de se référer à un modèle paramétrique de la production de la parole, bien que ses paramètres ne sont pas estimées explicitement[START_REF] Moulines | Non-parametric techniques for pitch-scale and time-scale modification of speech[END_REF].

  En 1986, McAulay et Quatieri suggéré leur célèbre modèle sinusoïdal (SM). En Dans ce travail, la forme d'onde vocale s(t) est supposé être la sortie du passage d'une forme d'onde d'excitation vocale e(t) grâce à un système linéaire h(t) représentant les caractéristiques de l'appareil vocal. Le système h(t) est supposé prendre en compte à la fois pour la forme de l'impulsion glottale et de la réponse impulsionnelle du conduit vocal. Le modèle d'excitation utilisée binaire non voisée / voisée est remplacé par une somme de sinusoïdes de la forme

		N
	e(t) =	�	a k (t) cos(Ω k (t))	(A.25)
		k=1

où N est le nombre de sinusoïdes, a k (t) est l'amplitude variant dans le temps associé à chaque onde sinusoïdale, et la phase d'excitation Ω k (t) est l'intégrale de la variation temporelle de fréquence ω k (t)

Ω k (t) = � t 0 ω k (σ)dσ + φ k (A.

26) où φ k est le décalage de phase fixe parce que les ondes sinusoïdales ne sont généralement pas en phase. La fonction de transfert du conduit vocal H(ω; t) peut s'écrire H(ω; t) = M (ω; t)e jψ(ω;t) (A.27) La dépendance à t est dû au fait que la réponse impulsionnelle du conduit vocal est variable dans le temps. L'amplitude et la phase du système le long de chaque trajectoire fréquentielle ω k t) sont donnés par M k (t) = M [ω k (t); t] (A.28) et ψ k (t) = ψ[ω k (t); t] (A.29) Ainsi, lorsque le signal d'excitation e(t) passe à travers la linéaire variant dans le conduit vocal linéaire variant dans le temps h(t), dont la sortie est la représentation sinusoïdale du signal de parole

  32)La validité de cette représentation est soumise à l'hypothèse de stationnarité des amplitudes et des fréquences d'excitation, par rapport à la réponse impulsionnelle du conduit vocal. Estimation des paramètres se fait via FFT pic cueillette, obtenant ainsi des amplitudes, des fréquences et phases. Ensuite, la séparation des paramètres du système et d'excitation est effectuée.A la synthèse, il existe une procédure de correspondance entre les valeurs des paramètres calculés à deux limites de trame consécutives. Ensuite, un schéma d'interpolation appropriée est appliquée sur les paires résultant de amplitude et de phase des échantillons d'excitation et les fonctions de l'appareil vocal sur chaque image, et enfin les ondes sinusoïdales sont générés en utilisant les composants interpolées.

  La procédure d'analyse est constitué de deux étapes : premièrement, l'estimation des paramètres du modèle ARX-LF, et deux, la décomposition du signal résiduel. Le signal résiduel a été proposé de décomposer de trois manières différentes[START_REF] Agiomyrgiannakis | ARX-LF-based source-filter methods for voice modification and transformation[END_REF] : une décomposition basée sur HNM, un modèle de bruit modulé, et un modèle harmonique. La synthèse est effectuée de manière synchrone brai passant par la source glottale reconstruite par l'intermédiaire du filtre AR variant dans le temps.Lorsque les modifications sont appliquées, puis deux étapes intermédiaires sont impliqués : i) la détermination de la séquence de l'analyse des indices de trame donnée d'un courant de f 0 et les coefficients de variation de temps, et ii) la modification et la synthèse des trames de parole sélectionnées. La première étape est très similaire au schéma d'analyse PSOLA. Mise à l'échelle de temps, une duplication typique ou l'élimination des cadres est effectué. Mise à l'échelle pour la hauteur, la forme d'onde de LF et le signal résiduel doivent être modifiées, puisque paramètres AR qui décrivent l'appareil vocal restent intacts. D'autres approches incluent le travail de Serra [Ser89], George et Smith [GS97], Degottex [Deg10], Jensen et al [NHD98a, JJH99, JHJ04], ainsi que le raffinement des méthodes déjà évoquées [DQ07, DQM09].

A.37) où a k [n] sont les coefficients variant dans le temps de la p modèle AR d'ordre qui décrit l'appareil vocal, et e[n] est le signal résiduel qui porte l'information qui reste non capturée par le modèle ARX-LF .

  .1 Le système d'analyse et de synthèse basé sur eaQHM désigne la composant déterministe et n(t) désigne la composant stochastique. La séparation des composants est réalisée en utilisant les schémas présentés dans[START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF], c'est à dire une première analyse est effectuée afin de discriminer la parole en régions vocales et non vocales, puis le terrain pour les régions voisées est déterminée, comme expliqué plus haut. Les paragraphes suivants présentent le décomposition de la partie déterministe et stochastique en détail.

	Le signal de parole est décomposé en deux parties, à savoir
	s(t) = d(t) + n(t)	(A.38)
	où (t)

  , b k sont les amplitudes complexes et les pentes complexes de la k e composant, respectivement, fk = k f0 sont les fréquences d'analyse, f0 est une estimation initiale de la fréquence fondamentale, et L est le nombre de quasiharmoniques, comme spécifié par le maximum exprimé fréquence. L'estimation des paramètres du modèle est obtenue par les moindres carrés, comme décrit dans[START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]. Comme il a été mentionné dans la section précédente, QHM est en mesure de corriger, dans le sens des moindres carrés, les défauts de fréquences qui sont dues à une estimation inexacte de la fréquence fondamentale. Soit η k représentent le décalage de fréquence de la k e composant :η k = f k -fk (A.40) où f k est la fréquence réelle et fk est une estimation de f k . Il a été montré dans [PRS08] qu'une projection de b k sur a k résultats une estimation du décalage de fréquence, qui est : }, �{b k } et �{a k }, �{b k } sont les parties réelles et imaginaires des amplitudes complexes et les pentes complexes, respectivement. En utilisant l'équation (A.41), les fréquences d'analyse peuvent être mis à jour et le signal peut être modélisé à nouveau en utilisant QHM, mais avec une nouvelle série de fréquences d'analyse, fk = fk + ηk , et conduisant ainsi à une représentation de signal plus précis. Cependant, seule l'estimation de la fréquence précise est adressée sont des estimations des amplitudes instantanées, les fréquences et les phases de la k e composant, respectivement, c(τ ) est le terme de cohérence de phase (l'équation (A.16)), et t i est le centre de la fenêtre d'analyse. L'adaptation est terminée en utilisant le mécanisme de correction de fréquence d'abord introduit dans [PRS08], et précise que l'estimation de l'écart entre le réelle k e fréquence et l'on estime qu'un, appelé η k = f k -fk , est donnée par Ainsi, lors de la première adaptation, pour la durée de l'analyse instantanée t i , les fréquences instantanées sont fk (t i ) = k f0 (t i ) + ηk (t i ) et les phases instantanées deviennent Ensuite, une solution des moindres carrés pour la a k , b k en utilisant ces fréquences raffinées (et phases) conduit à une meilleure estimation des amplitudes instantanées Âk (t) = |a k (t)| et la ηk termes. En ajoutant itérativement le ηk terme de l'adaptation en cours sur la k e fréquence titre de l'adaptation précédente, les trajectoires de fréquence représentent les fréquences réelles sous-jacentes mieux. Ce mécanisme d'adaptation s'arrête selon un critère de reconstruction liés au rapport à SRER. Enfin, il est essentiel de décrire l'estimation des paramètres variables dans le temps. Le k e trajectoire amplitude instantanée, Âk (t), est calculée par interpolation linéaire des estimations successives. Interpolation Spline pourrait être une alternative, mais il ne garantit pas la positivité des trajectoires. Pour cette raison, une interpolation linéaire est préférée.

	avec			αk (t) =	Âk (t + t i ) Â(t i )	(A.43)
	et	φk (t) = φk (t i ) + 2π	� t	� fk (τ ) + c(τ )	�	dτ, t ∈ [-T, T ]	(A.44)
					ti
	où Âk (t), fk (t), φk (t) ηk =	1 2π	�{a k }�{b k } -�{a k }�{b k } |a k | 2	(A.45)
		φk (t) = φk (t i ) + 2π	� t	( fk (τ ) + c(τ ))dτ	(A.46)
								ti
										fk t	�	w(t)	(A.39)
	où a k ηk =	1 2π	�{a k }�{b k } -�{a k }�{b k } |a k | 2	(A.41)
	où �{a k � L �	�	a k + tb k	��	αk (t)e j φk (t)	�	�	w(t)	(A.42)
		k=-L

par QHM. Le principe de modèle fixe est toujours valable dans un cadre d'analyse. Pour faire face à ce problème, la projection du signal sur un ensemble de fonctions de base variant dans le temps est suggéré dans

[START_REF] Pantazis | Adaptive AM-•FM signal decomposition with application to speech analysis[END_REF]

et

[START_REF] Kafentzis | An Extension of the Adaptive Quasi-Harmonic Model[END_REF]

:

d(t) =

  e j φk(t) .La composant stochastique, n(t), défini comme le résidu entre le signal analysé et la partie déterministe, ne tient pas compte uniquement pour les parties non vocales du discours, mais aussi pour le bruit de frottement dans des parties vocales. Il est modélisée comme : est égal à 1 ms. Depuis l'enveloppe de l'énergie a un comportement de pitch-synchrone[START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF], il peut être approchée dans un cadre à l'aide d'une somme de sinusoïdes quelques : est un petit entier, généralement 3 ou 4, et f 0 est la fréquence fondamentale du cadre. Estimation d'amplitude est effectuée à l'aide de pointe cueillette sur le spectre de courte durée du signal d'enveloppe d'énergie. Ce dernier est obtenu en utilisant un 30 ms Hamming fenêtre d'analyse et d'un taux de 5 ms de trame.SynthèsePendant la synthèse, la composant déterministe est obtenue à partir des paramètres de la dernière étape de l'adaptation de l'analyse, comme dans l'équation A.47. La composant stochastique n(t), est obtenu par une méthode simple additionrecouvrement (OLA), en utilisant les paramètres de la partie analyse. Enfin, la partie de synthèse est donnée parExemplesUn des exemples est représenté dans la figure A.3 pour un locuteur masculin.A.4.2 Le système d'analyse et de synthèse basé sur aHMDepuis l'aHM a été développé par Degottex et Stylianou en 2013[START_REF] Degottex | Analysis and synthesis of speech using an adaptive full-band harmonic model[END_REF], la aHNM a été développé au cours de cette thèse comme une première étape vers des modifications de la parole à partir de modèles adaptatif. Seule la partie analyse et la synthèse sera décrite ici.K est le nombre de composants, et A k (t), kφ 0 (t) sont les amplitudes instantanées et les phases instantanées de la k e composant, respectivement. S'il vous plaît noter que la phase instantanée de la k e composant est un multiple entier de la phase instantanée de la première harmonique, f 0 , et que l'analyse de la parole vocale est full-band.

		� M		�
	ê(t) =	�	A k e j2πf k t+φ k	w(t)	(A.50)
		k=-M
	où M ŝ(t) = d(t) + n(t)		(A.51)
				A k (t)e jkφ0(t)	(A.53)
		k=-K
	où	φ 0 (t) =	� t	2πf 0 (u)du		(A.47) (A.54)
			0
	Analyse du partie stochastique
	n(t) = e(t)(u G (t) * q(t))		(A.48)
	où u G (t) représente une composante de bruit blanc gaussien qui est convoluée par un temps variable autorégressive
	(AR) filtrage à réponse impulsionnelle q(t) et e(t) représente le domaine temporel enveloppe. Analyse LPC standard
	est utilisée pour l'estimation du filtre AR, avec un ordre de p = 18, pour une fréquence d'échantillonnage de f s = 16
	kHz. L'enveloppe du domaine temporel est un facteur très important, car il est essentiel pour la fusion efficace entre le
	déterministe et la partie stochastique [Sty96]. Dans [PS08], il a été montré qu'une enveloppe de temps en fonction de
	l'énergie est un bon choix. L'enveloppe d'énergie est donnée par :
		e(t) =

T � u=-T |n(t + u)| (A.49) où T Dans la partie de l'analyse, la partie déterministe et stochastique sont séparés et modélisé. En général, le premier peut être décrit comme s(t) = s d (t) + s s (t) (A.52) où s(t) désigne le signal de parole, et s d (t), s s (t) désignent la partie déterministe et stochastique, respectivement. Les modèles de pièces déterministes les quasi-périodicité de la parole voisée comme une somme de composantes harmoniques variant dans le temps, ainsi s d (t) = K �

TABLE A . 2 -

 A2 .2 présente les résultats par phonème, en termes de valeur moyenne de es SRER mondiale, pour plusieurs tailles de pas. Pour la modélisation des sons fricatifs, une stratégie similaire est Valeurs SRER mondiaux (dB) pour tous les modèles sur une grande base de données d'arrêts. Occlusives voisées sont également inclus dans la présente à des fins de comparaison. étape désigne le taux de trame d'analyse.suivie. Pour valider nos hypothèses, 485 ont exprimé et fricatives sourdes ont été extraites automatiquement à partir des énoncés de parole. Fricatives voisées comprennent /v/, /D/, /s/, and /S/, tandis que les non voisées sont /f/, /T/, /z/, and /Z/. Le nombre de fricatives extraites du haut-parleur mâle était presque le même avec ceux de sujets féminins. Les mêmes taux de trame que précédemment sont sélectionnés, soit 1 ms, 2 ms et 4 ms. Les paramètres autres que la fréquence d'image sont les mêmes que dans les sections précédentes. TableA.3 présente les résultats par fricative, en termes de valeur moyenne de SRER. En conclusion, il a été démontré que les modèles adaptatifs sont capables de modélisation de Valeurs es srer (dB) pour tous les modèles sur une grande base de données de fricatives. étape désigne le taux de trame d'analyse.

				à grande échelle de validation pour fricatives
					SRER mondial (dB)
	étape modèle	/v/	/D/	/s/	/S/	/f/	/T/	/z/	/Z/
	1 ms	SM eaQHM 26.4 25.6 24.1 26.4 25.8 24.3 29.5 28.9 14.7 13.2 13.9 11.3 12.7 15.1 17.5 17.3
	2 ms	SM eaQHM 23.5 23.1 22.6 24.7 23.5 22.6 28.6 27.8 13.1 11.3 12.1 10.4 10.2 14.7 15.9 15.2
	4 ms	SM eaQHM 22.4 22.2 21.9 23.1 22.6 21.7 27.5 27.1 12.2 10.6 11.2 9.6 9.7 8.9 13.3 13.7
	TABLE A.3 -
			à grande échelle de validation pour l'arrêt des sons
					SRER mondial (dB)
		étape modèle	/p/	/t/	/k/	/b/	/d/	/g/
		1ms	SM eaQHM 25.4 25.7 27.2 32.9 32.2 32.9 12.7 12.8 12.4 16.6 14.9 15.3
		2ms	SM eaQHM 26.1 26.1 26.0 31.7 31.4 34.6 12.8 12.7 12.3 16.5 15.0 15.4
		4ms	SM eaQHM 23.7 24.2 24.4 29.4 29.5 30.9 12.9 12.6 12.2 16.7 15.0 15.3

  fonctionne comme un moyen d'affiner l'estimation de f 0 pour trames voisées. Depuis iQHM détient également l'hypothèse de stationnarité, il a été jugé ne pas être crucial dans la fréquence de raffinement, en particulier dans des cadres non-dits, où le raffinement pourrait conduire à l'instabilité (par exemple les sauts brusques) de l'environ f 0 . Ainsi, l'estimation iQHM a été abandonné dans ce système, ce qui permet une complexité réduite sans perte de précision. Comme dernière étape, le signal global peut être synthétisé par l'interpolation des valeurs de |a k | et f k successives sur des instants de temps de l'analyse de t i , obtenant ainsi

					d(t) =	L �	Âk (t)e j φk (t)	(A.59)
						k=-L
	où
					Âk (t) = |a k (t)|			(A.60)
					φk (t i ) = ∠a k (t i )			(A.61)
	et		φk (t) = φk (t i ) +	� t	(2πk f0 (u) + c(u))du	(A.62)
							ti
	-0.	0	0.2	0.4	0.6		0.8	1	1.2		1.4	1.6	1.8
									fk t	�	w(t)	(A.58)

où a k est l'amplitude complexe de la harmonique k e , fk = k f0 sont les fréquences d'analyse, et L est le nombre d'harmoniques que couvrir l'ensemble du spectre à la fréquence de Nyquist. L'estimation des paramètres du modèle est obtenue par les moindres carrés, comme décrit dans

[START_REF] Stylianou | Harmonic plus Noise Models for Speech, combined with Statistical Methods, for Speech and Speaker Modification[END_REF]

. Contrairement à

[START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF]

, où l'estimation de f 0 initiale est affinée à l'aide d'un QHM itératif (iQHM), dans notre travail ne f 0 raffinement est nécessaire, réduisant ainsi la complexité globale de l'algorithme, et une estimation de l'amplitude simple pour chaque composant est effectuée. Dans

[START_REF] Pantazis | Analysis/Synthesis of Speech based on an Adaptive Quasi-Harmonic plus Noise Model[END_REF]

, iQHM

  Dans ce modèle, a k , b k sont l'amplitude complexe et la pente complexe de la composant k e , et Âk (t), fk (t), φk (t) sont des estimations de l'amplitude instantanée, la fréquence et la phase du composant k e , respectivement, de l'étape de l'analyse précédente. Les paramètres de a k , b k sont obtenus par Moindres Carrés comme indiqué à la section A.2.2. Il est évident que les fonctions de base, où le signal est projetée sont variables dans le temps. L'adaptation est terminée en utilisant le mécanisme de correction de fréquence d'abord introduit dans[START_REF] Pantazis | On the Properties of a Time-Varying Quasi-Harmonic Model of Speech[END_REF], et précise que l'estimation de l'écart entre le réelle fréquence k e et l'on estime qu'un, appelé η k = f k -fk , est donnée par Ainsi, lors de la première adaptation, pour la durée de l'analyse instantanée t i , les fréquences instantanées sont fk (t i ) = k f0 (t i ) + ηk (t i ) et les phases instantanées deviennent Ensuite, une solution des Moindres Carrés pour les a k , b k en utilisant ces fréquences raffinées (et phases) conduit à une meilleure estimation des amplitudes instantanées Âk (t) = |a k (t)| et la ηk termes. En ajoutant itérativement le ηk terme de l'adaptation en cours sur la fréquence k e titre de l'adaptation précédente, les trajectoires de fréquence s'écartent de harmonicité stricte et représentent les fréquences réelles sous-jacentes mieux. En outre, et au contraire des travaux antérieurs [PTRS10, PRS11], où la fréquence correction estimation ηk sur chaque adaptation doit être inférieure à f 0 /2, en cette approche, il est supposé que, après chaque adaptation des fréquences estimées deviennent de plus en plus localisée aux fréquences réelles, de sorte que la correction de fréquence pour un temps d'analyse donné t i instantanée est contrainte en tant que|η k (t i )| ≤ f0 (t i ) • • • , M } estle nombre actuel de l'adaptation et M est le nombre maximum de permis adaptations (dans nos expériences, M = 6). De cette fac ¸on, toute valeur relativement grande de correction de fréquence -ce qui conduit souvent à des artefacts audibles -qui pourrait être obtenue dans une étape ultérieure de l'adaptation sera supprimée. Cette contrainte est également motivée par le fait que des parties non voisées devraient être traités par le même modèle. De cette fac ¸on, les composants AM-FM sont conservés plus "serré" dans leur variabilité dans la représentation de bruit. Enfin, ce système d'adaptation se poursuit jusqu'à ce qu'un critère de convergence soit atteint, ce qui est lié à le SRER globale, c'est à dire lorsque le SRER cesse d'augmenter après chaque adaptation, alors l'algorithme est considéré comme ayant convergé.

					�
					e j φk (t)	�	w(t)	(A.63)
	avec		Âk (t) =	Âk (t + t i ) Âk (t i )	(A.64)
	et φk (t) comme dans l'équation (A.62). ηk =	1 2π	�{a k }�{b k } -�{a k }�{b k } |a k | 2	(A.65)
	φk (t) = φk (t i ) +	� t	(2π fk (u) + c(u))du	(A.66)
				ti
					m + 1	(A.67)
	où m in{1,

TABLE A . 4 -

 A4 la qualité perc ¸ue, un test formel d'écoute a été conc ¸u. Les Valeurs SRER (dB) pour tous les modèles sur une base de données de 32 énoncés (16 de locuteurs, 16 de locutrices) en utilisant SWIPE et YIN estimateurs de hauteur. Moyenne et l'écart type sont donnés. résultats à partir de 34 auditeurs sont représentées dans les figures A.6 et A.7.

			SRER Performance
			Locuteur
	modèle		SWIPE	YIN
		mâles	femelles	mâles	femelles
	SM	18.6 (1.90) 18.6 (3.64) 14.3 (2.20) 16.2 (3.28)
	aHM	23.9 (2.66) 18.9 (3.27) 23.9 (2.61) 19.9 (3.05)
	eaQHM 34.5 (2.39) 30.9 (3.00) 34.4 (2.45) 30.7 (3.19)

  kφ 0 (t i a )(A.73) Ensuite, les valeurs de RP sont interpolées, obtenant ainsi ∠ã � k (t � ), comme :∠ã � k (t � ) = ∠ã k (D -1 (t � )) (A.74)et enfin, la fréquence de mise à l'échelle de temps intégrée est rajouté aux valeurs interpolées RP :De cette fac ¸on, la forme d'onde conserve sa forme pour des parties vocales, c'est à dire le temps échelle est forme invariante.

	φ� k (t � ) = ∠ã � k (t � ) +	� t � 0	2πkf � 0 (u)du	(A.75)

  i 0 }, sont connus. De ceux-ci, nous pouvons calculer leurs homologues continus, qui sont les amplitudes instantanéesA k (t) = |a k (t)| et fréquences f 0 (t), obtenu par interpolation a i k et f i 0 , respectivement. Ensuite, la forme d'onde de hauteur-échelle, s P S (t), pour un facteur pitch-échelle constante est donnée par : ŝP S (t) = et φ � k (t) sont calculés à l'aide de la manière suivante : 1. Afin de calculer φ � k (t), il est d'abord nécessaire de calculer les fréquences de pas à l'échelle. Ainsi, les nouvelles fréquences sont données par : kf 0 (t) ← ρkf 0 (t) (A.77) 2. Les amplitudes instantanées au moment de l'analyse instants t i a , A � k (t i a ), sont calculés à partir d'échantillonnage de l'enveloppe spectrale à la correspondante fréquences ρkf 0 : ) est l'enveloppe à base de Discrete-All-Pole construite autour de temps instant t i a . Ensuite, le k e d'amplitude instantanée est interpolée linéairement sur des instants successifs. 3. Ensuite, la phase instantanée doit être recalculée. Pour cela, le RP est d'abord calculée en extrayant l'intégrale de la fréquence fondamentale initiale de l'information de phase à l'analyse temps instant t i a , comme dans l'équation (A.55). Ensuite, les valeurs de RP sont interpolées, obtenant ainsi ∠ã k (t), et enfin, la fréquence de pas-échelle intégrée est rajouté aux valeurs interpolées RP :

	K
	�	A � k (t)e jφ � k (t)	(A.76)
	k=-K
	où A � k (t) A � k (t i a ) = DAP (t i a , ρkf 0 )	(A.78)
	où DAP (t i a , f φ� k (t) = ∠ã k (t) +	2π f s	� t 0	ρkf 0 (u)du	(A.79)

  1. Afin de calculer φ � k (t), il est d'abord nécessaire de calculer les fréquences de hauteur à l'échelle. Ainsi, les nouvelles fréquences sont données par :f k (t) ← ρf k (t) (A.86)2. Les amplitudes instantanées au moment de l'analyse instants t i a , A � k (t i a ), sont calculés à partir d'échantillonnage de l'enveloppe spectrale à la correspondante fréquences ρf k :

	A � k (t i

a ) = DAP (t i a , ρf k ) (A.87)

où DAP (t i a , f ) est l'enveloppe à base de Discrete-All-Pole construite autour de temps instant t i a . Ensuite, le k e d'amplitude instantanée est interpolée linéairement sur des instants successifs.

  SRER(eaQHM-EDS) SRER(eaQHM-SM)Local Global K max Local Global K max Différence moyenne de SRER (dB) entre eaQHM et EDS ou SM à travers le nombre de partiels K. Différence moyenne de SRER (dB) entre la eaQHM et EDS ou SM dans la taille de fenêtre L.

	Brass	7.69	9.22	7.48	22.44	9.05	5.41
	Woodwinds	2.95	6.90	19.60	17.21	12.18	29.93
	Bowed Strings	6.32	3.45	21.52	14.46	6.06	31.21
	Plucked Strings	13.26	11.96	15.75	24.69	25.63	42.99
	Bowed Percus-sion	-3.89	2.00	21.80	2.65	10.56	37.63
	Struck Percus-sion	2.83	-1.52	11.88	7.68	2.77	21.06
	Popular	1.09	-0.44	11.93	3.55	-1.21	18.09
	Keyboard	6.33	4.41	3.15	16.99	11.87	24.79
	Total	4.55	4.50	14.14	14.15	9.07	26.39
	TABLE A.5 -SRER(eaQHM-EDS)	SRER(eaQHM-SM)
		Local Global 3T 0	Local Global 3T 0
	Brass	-3.07	10.74	27.27 12.20	14.75	31.19
	Woodwinds	-2.81	6.92	18.21 11.63	13.43	30.44
	Bowed Strings	3.01	7.55	28.11 10.85	12.62	38.18
	Plucked Strings	-12.38 -15.89	7.78	21.16	17.57	49.88
	Bowed Percus-sion	-4.26	6.86	21.80 10.83	15.85	37.63
	Struck Percus-sion	8.13	5.85	10.82 15.24	11.73	19.25
	Popular	-0.91	5.57	15.25	9.00	9.02	21.91
	Keyboard	-4.25	-4.02	3.15	12.90	10.60	24.79
	Total	-2.30	2.95	16.55 12.46	12.78	31.66
	TABLE A.6 -

  Pour chacun des quatre émotions évoquées plus haut, un dictionnaire a été conc ¸u en utilisant l'algorithme LBG[START_REF] Linde | An algorithm for vector quantizer design[END_REF]. L'émotion est reconnu par la distorsion moyenne minimale. La matrice de confusion pour la classification basée amplitude est donnée dans le tableau A.7, alors que pour une base de fréquence correspondante est donnée dans le tableau A.8. Il peut être vu que dans les deux cas, la fac ¸on de

			VQ Classification in % -Amplitudes
				Classe prédit
			fâché	neutre	souple question
		Fâché	77(72)	14(14)	2(3)	7(11)
	Classe	Neutre Souple	4(4) 3(5)	64(63) 31(30) 56(50) 18(18)	14(15) 10(15)
		Question	6(4)	21(22) 13(20)	60(55)

• • • y C ] (A.95) et C est la taille du dictionnaire. L'AD est défini comme AD = 1 C C � k=1 min y i ∈Y d 2 (x k , y i ) (A.96) où d(x, y) est la distance euclidienne (ED) entre les vecteurs x et y.

TABLE A . 7 -

 A7 Classification note (%) pour quatre émotions de la base de données SUSAS, l'utilisation des fonctions d'amplitude extraites de eaQHM et SM (entre parenthèses).parler fâché se distingue du reste de styles de parole. Cela devrait puisque cette fac ¸on de parler est très différente de celle des autres en termes d'amplitude et de fréquence des distributions[START_REF] Ramamohan | Sinusoidal model-based analysis and classification of stressed speech[END_REF].En général, les paramètres obtenus à partir de la tête eaQHM à de meilleurs scores de classement dans tous les cas. En outre, la fac ¸on de parler fâché a le pourcentage le plus élevé de classification correcte pour les deux modèles et Classification note (%) pour quatre émotions de la base de données SUSAS, l'utilisation des fonctions de fréquence extraites de eaQHM et SM (entre parenthèses).deux ensembles de fonctionnalités. Le style question de la parole est le plus difficile à classer correctement lorsque les fréquences sont utilisées comme fonctions, et nous pouvons voir que c'est surtout confondu avec le style neutre de parler. D'autre part, la fac ¸on de parler souple a le score le plus bas de la classification lorsque les amplitudes sont utilisés comme caractéristiques.

			VQ Classification in %-Frequencies
				Classe prédit
			Fâché	Neutre Souple Question
		Fâché	71(70)	6(6)	7(5)	21(18)
	Classe	Neutre Souple	6(6) 3(3)	55(38) 13(25) 65(59) 24(28)	15(27) 14(13)
		Question 17(18) 18(24) 14(25)	50(33)
	TABLE A.8 -

  TABLE A.9 -Classification note (%) sur la base des amplitudes et des fréquences pour une classification 128 bits VQ de 4 émotions de la base de données normalisées SUSAS. tandis que le SM atteint 54%. Apparemment, tous les styles de parole ont été favorisés par ce système combiné. Surtout le question et le fâché style de parole atteint augmentation significative de leurs taux de classification dans les deux modèles. Alors que le fâché style de parler déjà d'un pourcentage relativement élevé, le question style oratoire a augmenté son score intéressant de classification correcte. Cependant, la fac ¸on de parler souple et neutre n'a pas changé de manière significative leurs pourcentages. Ceci suggère que la somme pondérée des consignes avant le classement peut être plus approprié.

						données,
			VQ classification combiné dans %
				Classe prédit
			Fâché	Neutre Souple Question
		Fâché	83(77)	5(5)	1(5)	11(13)
	Classe	Neutre Souple	15(4) 10(2)	58(48) 18(29) 56(54) 12(24)	15(24) 16(15)
		Question 20(17)	6(24)	11(21)	63(38)

http://www.csd.uoc.gr/~kafentz/listest/pmwiki.php?n=Main.EAQHM-LT

'Popular' and 'Keyboard' musical instruments are from the RWC Music Database: Musical Instrument Sound http://staff.aist.go. jp/m.goto/RWC-MDB/. All other musical instruments are from Vienna Symphonic Library database of musical instrument samples http: //www.vsl.co.at/en/65/71/84/1349.vsl

http://gillesdegottex.eu/ExCaetano2013simil

http://www2.csd.uoc.gr/˜kafentz/listest/pmwiki.php?n=Main.Exprtest

'Popular' and 'Keyboard' musical instruments are from the RWC Music Database : Musical Instrument Soundhttp://staff.aist.go.jp/ m.goto/RWC-MDB/. Tous les autres instruments de musique sont la base de données Vienna Symphonic Library d'échantillons d'instruments de musiquehttp://www.vsl.co.at/en/65/71/84/1349.vsl

Acknowledgements

Remerciements

Τμήμα Επιστήμης Υπολογιστών του Πανεπιστημίου Κρήτης, και την καθηγήτρια Régine Le Bouquin-Jeannes, από το Πανεπιστήμιο της Rennes 1, για το χρόνο που αφιέρωσαν στην ανάγνωση αυτής της διατριβής και για την αποδοχή τους να συμμετάσχουν στην εξεταστική επιτροπή. Θα ήθελα επίσης να ευχαριστήσω τον καθηγητή Gernot Kubin, από το Τεχνολογικό Πανεπιστήμιο του Graz της Αυστρίας, και τον καθηγητή Bastiaan. W. Kleijn, από το Πανεπιστήμιο του Wellington της Νέας Ζηλανδίας, για το ότι δέχθηκαν να γράψουν αναφορά για τη διατριβή μου. Επίσης, θα ήθελα να ευχαριστησω το Δρ. Γιάννη Πανταζή, που μου προσέφερε πολύτιμη βοήθεια στην αρχή αυτής της διατριβής.

Επίσης, ευχαριστώ τους συναδέλφους μου, πρώην και νυν, με τους οποίους μοιράστηκα της μέρες μου στο Πανεπιστήμιο Κρήτης, και ειδικότερα στο Εργαστήριο Πολυμέσων του Τμήματος Επιστήμης Υπολογιστών. Η παρουσία τους το έκανε ένα υπέροχο εργασιακό περιβάλλον και ένα δεύτερο σπίτι για μένα, , για την έγνοια και τη φιλία τους, έστω κι αν κάποιοι απ΄ αυτούς είναι μακριά απ΄ την Κρήτη.

Τελευταίους αλλά όχι έσχατους, θα ήθελα να ευχαριστήσω θερμά την οικογένειά μου: τους γονείς μου, Παναγιώτη Καφεντζή και Διαμάντω Τσιρολιά, για τα πάντα. Τους χρωστάω αυτό που είμαι σήμερα. Την αδελφή μου, Μαρία-Χρυσάνθη, για την υπομονή, την αγάπη, την έγνοια, την ανεκτικότητα, και την υποστήριξή της, σε όλα αυτά τα χρόνια που είμαστε μαζί στην Κρήτη. Τον αδελφό μου, Στέλιο, για την αγάπη και την έγνοια του. Τέλος, τη σύζυγό μου ΄ Αννα, για την τεράστια υπομονή και αγάπη της, και για το ότι θα φέρει στη ζωή το πιο πολύτιμο δώρο μας, το γιο μας, Παναγιώτη.