Mathieu Lagrange
email: mathieu.lagrange@ircam.fr

Roland Badeau

Bertrand David

Nancy Bertin

Jose Echeveste

Olivier Derrien

Sylvain Marchand

Laurent Daudet

The DESAM toolbox: spectral analysis of musical audio

come   L'archive ouverte pluridisciplinaire

INTRODUCTION

Audio signal processing has made tremendous progress in the last decade, on both quantitative and qualitative levels. It now extends way beyond "traditional" audio engineering community ("hi-fi" hardware and software design, audio effects, audio coding, ...), sharing strong interaction with neighboring fields such as music perception and psychoacoustics, Music Information Retrieval (MIR), auditory displays, tools for composition and home studios, human-machine interfaces, ... to name but a few.

Arguably, one of the most challenging tasks, shared by most of these communities, is to decompose a complex, multiinstrumental, music signal into meaningful entities, or "objects", that not only convey some information from the signal processing point of view, but are also carry intrinsic meaning from a musicological / perceptual prospective (as opposed to e.g. time-frequency "atoms" in the analysis/synthesis framework, or signal "features" in the frame-based analysis). This type of processing has sometimes been called "mid-level" representations in the literature, and is also related to CASA framework [START_REF] Bregman | Auditory Scene Analysis: The Perceptual Organization of Sound[END_REF]. Here, "objects" or "sound elements" can be notes, or structural elements of a note, such as harmonic tracks of partials. From these, one could possibly infer what notes are being played ( i.e. the score), but also what are the instruments, what type of playing technique is being played or what type of recording techniques have been used.

Obviously, no single technique can perform such a task perfectly on any type of real-life, polyphonic music. But what does "perfect" mean in this context? Obviously, this task will remain an ill-posed problem unless a specific signal model is defined. The goal of the DESAM Toolbox (that draws its name from the collaborative project: Décomposition en Eléments Sonores et Applications Musicales, funded by the French ANR), is to provide a range of state-of-the-art signal processing tools that decompose music files according to different signal models, giving rise to different "mid-level" representations, in other words a set of coefficients of a parametric model. As the main target of this toolbox is a widespread use for rapid prototyping of algorithms, especially in the academic community, as well as an educational purpose, it is composed of a set of MATLAB functions, that can be used for academic research under the only restrictions of a GPL license. A preliminary version is available online 1 and will be maintained and upgraded according to user's feedback.

After motivating the need for such a toolbox in Section 2, the collaborative project that led to the set of tools presented in this toolbox is described in Section 3. The overall organization of the toolbox is then introduced in Section 4. The core tools, respectively based on sinusoidal models and more general spectral models are described in Sections 5 and 6, and the application to automatic music transcription is finally presented in Section 7.

RELATION TO OTHER MATLAB TOOLBOXES

Yet, why another audio Matlab toolbox? At time of writing, most widely-used Matlab toolboxes have two targets:

• either related to perception such as the Auditory Toolbox [START_REF] Slaney | Auditory toolbox version2[END_REF], in which case the main goal is to make some perceptually-relevant pre-processing for further analysis tasks;

• or related to Music Information Retrieval tasks (MIR Toolbox [START_REF] Lartillot | A Matlab toolbox for musical feature extraction from audio[END_REF], MA Toolbox [START_REF] Pampalk | A Matlab Toolbox to Compute Similarity from Audio[END_REF]), in which case the signal parameters are relatively simple, most of the time extracted on a frame-by-frame basis.

We believe there is still a widespread need for tools that implement recent state-of-the-art analysis methods, able not only to analyze audio signals according to various audio signal models -and can hence be used as input parameters for MIR systems -, but also to resynthesize the music from the extracted parameters -intended for audio coding in the engineering point of view, or sparse coding in the neural information processing community. These newer techniques, such as high-resolution methods, or NMF, have already proven their usefulness for a wide range of real-life problems. It is now time to deliver them to the whole community in order for them to become "classic" tools, in the same way as the Short Time Fourier Transform or the Additive Synthesis have been for decades. The users will notice that the core algorithm of each of these powerful tools is always strikingly simple and compact, we believe that they provide foundations for a multitude of future improvements.

DESAM PROJECT

The DESAM project was a fundamental research project involving four French laboratories:

• CNRS LTCI (Laboratoire Traitement et Communication de l'Information), Paris, France;

• LAM (Lutheries -Acoustique -Musique) team, Institut Jean Le Rond d'Alembert, UPMC Univ. Paris 6;

• LaBRI (Laboratoire Bordelais de Recherche en Informatique), Bordeaux 1 University;

• STIC (Laboratoire Sciences et Technologies de l'Information et de la Communication), Var and Toulon University.

Headed by the LTCI, the project started in November, 2006 and it ended in February, 2010. It was divided in two parts. The first one was devoted to the theoretical and experimental study of parametric and non-parametric techniques for decomposing audio signals into sound elements. The second part focused on some musical applications of these decompositions.

Decompositions into sound elements

Their pitch and their timbre, specific to the instrument, characterize musical notes. When these notes are well modeled by a mixture of sinusoids, the estimation of frequencies, amplitudes, and their time-variations, are useful to analyze the pitch and timbre of the sound. In this project, we have developed innovative highresolution (HR) methods for time-frequency analysis, in order to estimate the fine time variations of these two parameters [START_REF] Badeau | Adaptive subspace methods for high resolution analysis of music signals[END_REF][START_REF] David | HRHATRAC algorithm for spectral line tracking of musical signals[END_REF][START_REF] David | Fast sequential LS estimation for sinusoidal modeling and decomposition of audio signals[END_REF]. The modeling of non-stationary sinusoids was further addressed in [START_REF] Marchand | Generalization of the derivative analysis method to non-stationary sinusoidal modeling[END_REF].

Besides, since a musical piece is composed of multiple notes played at different times, it is naturally described as a combination of sound elements (which can be either isolated notes, combinations of notes, or parts of notes). Such a representation is called sparse, since a limited number of sound elements permits to describe the whole musical content. Therefore, the first approach that we investigated in order to decompose a sound aims at producing the sparsest representation [START_REF] Daudet | Audio sparse decompositions in parallel -Let the greed be shared![END_REF]. The second approach was based on the non-negative matrix factorization (NMF), which we have refined for our needs [START_REF] Bertin | A tempering approach for Itakura-Saito non-negative matrix factorization. With application to music transcription[END_REF][START_REF] Hennequin | NMF with timefrequency activations to model non-stationary audio events[END_REF]. It exploits the redundancies in a musical piece (a single tone being generally repeated several times) in order to identify the sound elements via their spectral characteristics and their various occurrences through time.

Musical Applications

Analyzing a polyphonic recording in order to extract or to modify its musical content (e.g. the instruments, the rhythm or the notes) is a difficult exercise, even for an experienced musician. The DE-SAM project aimed at making a machine capable of performing such tasks. Let us mention some of them:

• The ability of identifying musical instruments from recordings is a key task in music indexation retrieval. An important characteristic of a sound that defines the perception of timbre is its spectral envelope.

• The ability to estimate the pitch of a sound (on a scale from low to high) is critical for identifying musical notes, but remains difficult in a polyphonic recording, because of the overlap of sounds.

• If producing a sound given a musical score happens to be easy both for the musician and computer, the inverse problem, called automatic transcription, which aims at recovering a musical score from a recording, proves to be much more complex and requires expert skills.

• Storing and transmitting an increasing volume of musical recordings requires coding this data in a format as compact as possible, making a compromise between the quantity of coded information, and the quality of the reproduced sound.

The decompositions into sound elements provide a representation of the signal as a sum of more elementary entities. From these entities, high level descriptors are extracted and are useful for instrument recognition, rhythm estimation, and multiple pitch estimation. These tasks are all necessary when the design of an automatic transcriber is targeted.

We have thus proposed new methods for estimating and comparing the spectral envelopes of musical sounds [12,[START_REF] Lagrange | Robust similarity metrics between audio signals based on asymmetrical spectral envelope matching[END_REF]. We have also proposed original pitch estimation methods, capable of estimating up to ten simultaneous notes [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF][START_REF] Badeau | Expectationmaximization algorithm for multi-pitch estimation and separation of overlapping harmonic spectra[END_REF], which have been used in an automatic transcription algorithm designed for piano music [START_REF] Emiya | Automatic transcription of piano music based on HMM tracking of jointly-estimated pitches[END_REF]. An alternate transcription scheme based on NMF has been developed for a larger class of instruments [START_REF] Bertin | Enforcing Harmonicity and Smoothness in Bayesian Non-negative Matrix Factorization Applied to Polyphonic Music Transcription[END_REF]. Besides, the precision of the decomposition permitted a physical analysis of sound production in musical instruments [START_REF] Le Carrou | Sympathetic string modes in the concert harp[END_REF], and the development of more effective methods for coding and modifying sounds. Two approaches have been retained for coding. The first one, based on HR methods, permitted to reach very low bit rates [START_REF] Derrien | Damped sinusoids and subspace based approach for lossy audio coding[END_REF]. The second one, based on sparse decompositions, was a scalable audio coder which can reach transparency [START_REF] Ravelli | Union of MDCT bases for audio coding[END_REF]. Signal modifications were performed either by resampling the sinusoidal modeling parameters [START_REF] Raspaud | Enhanced resampling for sinusoidal modeling parameters[END_REF], or by modifying the sound elements of a sparse decomposition [START_REF] Derrien | Time-Scaling Of Audio Signals With Multi-Scale Gabor Analysis[END_REF].

OVERVIEW OF THE TOOLBOX

The main contributions of the DESAM project have been integrated in the DESAM Toolbox, a software library written in Matlab language and distributed under the terms of the GNU General Public License (GPL). As shown on Figure 1, the toolbox is organized in three main parts. The first two parts are dedicated to the core of the toolbox, namely the representation of audio using sinusoidal or spectral models. The last part is dedicated to an application task: the transcription of polyphonic music.

The part dedicated to sinusoidal models is further separated into 2 subparts. The first one groups methods that deal with the estimation of the parameters of the sinusoidal model over a short time observation interval (i.e. frame level). The second one groups methods which address the issue of estimating and/or tracking those parameters over a long period of time (i.e. song level). Another complementary way of describing the spectral content of audio signals, explored in the second part, is to introduce constraints on spectral shapes or envelopes. Once estimated, they can be considered for deriving timbral features or estimating the fundamental frequencies of polyphonic sounds.

SINUSOIDAL MODELS

Short-term Models

Given a short observation window of a given audio signal, one would like to estimate the frequency and the amplitude of a given number of sinusoidal components. The DESAM Toolbox provides several and complementary ways of achieving this task. A first approach is able to estimate those parameters as well as their first derivative by considering the Fourier spectrum as its underlying representation. In order to get rid of frequency resolution constraints, a second approach considers High Resolution (HR) methods.

Fourier-based Methods

The DESAM Toolbox includes efficient non-stationary sinusoidal estimation using enhanced versions of either the reassignment method or the derivative method (see [START_REF] Marchand | Generalization of the derivative analysis method to non-stationary sinusoidal modeling[END_REF]).

Both methods have been proven to be equivalent in theory and practice and to achieve nearly-optimal results in terms of estimation precision (provided that the frequency resolution is sufficient to isolate the spectral peaks corresponding to the sinusoids), see [START_REF] Marchand | On the Equivalence of Phase-Based Methods for the Estimation of Instantaneous Frequency[END_REF][START_REF] Lagrange | Estimating the Instantaneous Frequency of Sinusoidal Components Using Phase-Based Methods[END_REF][START_REF] Marchand | Generalization of the derivative analysis method to non-stationary sinusoidal modeling[END_REF][START_REF] Hamilton | Theoretical and Practical Comparisons of the Reassignment Method and the Derivative Method for the Estimation of the Frequency Slope[END_REF].

Although the resolution is still limited to the width of a bin of the discrete Fourier transform (see Figure 2 for an example of this frequency overlap phenomenon), the precision is close to the optimal. Moreover, the non-stationary model is more general than the one used in HR analysis, since it considers more general frequency modulations:

s(t) = P X p=1 ap(t) exp(jφp(t)),
where P is the number of partials, and

ap(t) = ap exp(µpt), φp(t) = φp + ωpt + 1 2 ψpt 2 .
For the reassignment method, the syntax is:

[a, mu, phi, omega, psi, delta_t] = reassignment (x, Fs, m)
The input parameters are:

• x, the signal frame to be analyzed;

• Fs, the sampling frequency (in Hz);

• m, the bin index where to pick the spectral peak (optional); and the output values are a, mu, phi, omega and psi, corresponding respectively to the estimated amplitude, amplitude modulation, phase, frequency and frequency modulation of the spectral peak. delta_t is the reassigned time.

For the derivative method, the syntax is quite similar:

[a, mu, phi, omega, psi] = derivative (x, d1, d2, Fs, m) except that this function requires the first and second derivatives d1 and d2 of the signal x, which can be computed using the function drv = discrete_derivative (src, Fs)

where src is the source signal and drv is its derivative. The test function of the discrete differentiation generates the Figure 1 of [START_REF] Marchand | Generalization of the derivative analysis method to non-stationary sinusoidal modeling[END_REF]. The test_global.m procedure generates the full tests (Figures 23456of [START_REF] Marchand | Generalization of the derivative analysis method to non-stationary sinusoidal modeling[END_REF]) with heavy computations requiring some time, though. For now, the implementation of the reassignment method is more efficient. Thus, we recommend this method for the estimation of the sinusoidal parameters. See the test_example.m2 script for a small example using this method.

Simple High Resolution (HR) analysis/synthesis

An analysis/synthesis scheme for musical signals has been proposed in [START_REF] Badeau | High resolution methods for estimating and tracking modulated sinusoids[END_REF]. It is based on the Exponentially Damped Sinusoids (EDS) model:

s(t) = P X p=1 αp z t p ,
where P is the number of partials, the complex amplitudes αp are of the form αp = ap exp(jφp) (where ap > 0 is the real amplitude and φp ∈ R is the phase of the partial p), and the complex This model is estimated by means of a HR subspace method called ESPRIT, which overcomes the spectral resolution limit of the Fourier transform, and achieves an accurate estimation of the sinusoidal components of the signal. Initially, the signal is preaccentuated. The noise autoregressive envelope is estimated, and the signal is whitened with a FIR Filter [START_REF] Badeau | High resolution methods for estimating and tracking modulated sinusoids[END_REF].

The function HR_analysis.m 3 decomposes the audio signal into a sum of exponentially damped sinusoids, on a single time-segment, and the noise component is discarded. The function HR_synthesis.m re-synthesizes the audio signal corresponding to these EDS components. An optional wide-band whitening filter is proposed in the analysis function. The syntax is:

[poles, amplitudes] = HR_analysis(signal, order, whitening) signal = HR_synthesis(poles, amplitudes, signal_length)

The parameters are:

• signal, signal to be analyzed or synthesized;

• order, model order;

• whitening, flag for optional whitening;

• poles, vector containing the signal poles;

• amplitudes, vector containing the complex amplitudes;

• signal_length, length of the signal to be synthesized (in samples).

Subband analysis and dynamic segmentation for HR audio coding

For an efficient audio coding application, the HR analysis/synthesis can be performed in frequency subbands and on a dynamic frame-by-frame basis. In order to reduce the complexity, we use a perfect-reconstruction filter bank, which keeps only 3 Directory "sinusoidalModels/shortTerm/highResolution_lma". the positive-frequencies in the signal. A dynamic frame-by-frame segmentation is performed according to an onset detection algorithm [START_REF] Duxbury | A hybrid approach to musical note onset detection[END_REF]. The corresponding script is demo.m. The audio codec described in [START_REF] Derrien | Damped sinusoids and subspace based approach for lossy audio coding[END_REF] is based on this analysis/synthesis scheme.

Adaptive subband analysis and fast HR subspace tracking

An adaptive version of this method, using a classical filter bank (with subband-by-subband whitening) and a fixed frame-length analysis/synthesis, is also described in [START_REF] Badeau | High resolution methods for estimating and tracking modulated sinusoids[END_REF]. This scheme leads to a new representation, called the HR-ogram, where the signal components are represented as points in the time-frequency plane, see Figure 3. The stochastic part is then defined as the residual of this decomposition. The deterministic and stochastic parts can thus be processed separately, leading to high quality audio effects.

The program included in the DESAM Toolbox4 decomposes the audio signal into a sum of exponentially damped sinusoids and autoregressive noise. Then the ESTER method [START_REF] Badeau | A new perturbation analysis for signal enumeration in rotational invariance techniques[END_REF] is used to estimate the number of sinusoids, and a fast adaptive algorithm (called Sequential Iteration) performs the subspace tracking [START_REF] Badeau | EDS parametric modeling and tracking of audio signals[END_REF]. Sinusoids/noise separation is achieved by projection onto the signal subspace and the noise subspace [START_REF] David | An EDS modelling tool for tracking and modifying musical signals[END_REF], and by reconstruction with the synthesis filter bank. The whole processing is described in Part III of reference [START_REF] Badeau | High resolution methods for estimating and tracking modulated sinusoids[END_REF], and was presented at Acoustics'08 [START_REF] Badeau | Adaptive subspace methods for high resolution analysis of music signals[END_REF]. The main function is analyse.m, whose syntax is the following:

[z,alpha,x] = analyse(s,Fs)

The input and output parameters are:

• s, signal to be analyzed;

• Fs, sampling frequency (preferably 44100 Hz);

• z, matrix containing the signal poles;

• alpha, matrix containing the complex amplitudes;

• x, sinusoidal part of the signal. This code was successfully applied to the decomposition of sounds from various musical instruments.

Long-term Models

Conventional Partial Tracking

Following pioneering work of MacAulay & al [START_REF] Mcaulay | Speech analysis/synthesis based on a sinusoidal representation[END_REF] and Serra & al [START_REF] Serra | Spectral Modeling Synthesis: A Sound Analysis/Synthesis System Based on a Deterministic plus Stochastic Decomposition[END_REF], most partial tracking algorithms link together spectral peaks estimated using short-term methods such as the ones described in the previous section in order to form partials, i.e. sinusoidal oscillators whose parameters evolve slowly with time.

The approaches cited above, respectively termed Maq and Serra consider heuristics such as frequency proximity in order to enforce the slow variation of the parameters. In [START_REF] Lagrange | Enhancing the tracking of partials for the sinusoidal modeling of polyphonic sounds[END_REF], more sophisticated approaches were developed in order to ensure those constraints, respectively by considering the predictability of the evolution and by enforcing that the spectral content of those evolution is of low frequency. Those approaches are respectively termed LP (for Linear Prediction) and HF (for High Frequency analysis), see Figure 4 for some results.

The syntax for the Maq tracking method is:

[P, Z,tag] = peaks2partialsMaq(A, F, tag, Z, deltaF) where

• A, F are respectively the amplitudes and the frequencies of the spectral peaks;

• tag is the partial index value assigned to the next created partial;

• Z is the state of the active tracks;

• deltaF is the maximal frequency deviation allowed between 2 successive peaks in any partial.

• P is the partial's label assigned to the spectral peaks. The 3 other methods follow a similar syntax. One can notice that this syntax allows the trackers to be used in a streaming fashion as shown in the demo.m 5 script where the spectral data to be processed is split in two successive sets of peaks.

The four methods have been successfully tested in a small set of audio data. However, it should not be considered as authoritative implementation of the above described algorithms at this stage (as more extensive testing is still required).

Tracking of frequency components: the HRHATRAC algorithm [6]

HRHATRAC stands for High Resolution HArmonics TRACking and denotes an algorithm aiming at modeling musical sounds as multiple, slowly varying, spectral trajectories surrounded by an additive noise. HRHATRAC combines the efficiency of one of the most recent subspace tracking algorithm [START_REF] Badeau | Fast Approximated Power Iteration Subspace Tracking[END_REF] with a gradient update for adapting the signal poles estimates. Hence it is able to update the frequency of each component from a time-instant to the next. It leads to a representation of the sinusoidal content of the signal in terms of spectral trajectories (slowly varying frequency components). 5 Directory "sinusoidalModels/longTerm/telecomParisTech_echeveste" The corresponding DESAM Toolbox function hrhatrack returns for the kth frequency component its instantaneous frequency fp(t) = (2π) -1 φ ′ p (t) and instantaneous amplitude modulation ap(t). The sinusoidal part of the signal can thus be obtained as P p ap(t) exp(jφp(t)). The main function is hrhatrack.m6 , whose syntax is the following:

[freqs,amps] = hrhatrack(s,Ns,P,beta,muL,muV)

The input and output parameters are:

• s, signal to be analyzed; • Ns, size of the signal subspace. Ideally, the number of complex frequency components, if it is known. If not known, overestimate Ns (typically by a factor 1.5 to 2);

• P, size of the autocovariance matrix (P × P ), default P = 3*Ns;

• beta, forgetting factor for the updated covariance matrix, default: 0.99;

• muL,muV, gradient steps for the eigenvalues and the eigenvectors updates respectively, default 0.9;

• freqs,amps, fp(t) and ap(t). A demo script is also included: demo_hrhatrack.m.

Adaptive estimation scheme for instantaneous amplitudes

In [START_REF] David | Fast sequential LS estimation for sinusoidal modeling and decomposition of audio signals[END_REF] an adaptive estimation scheme is proposed for sample-wise update of the instantaneous amplitudes of known frequency components. The resulting decomposition of the signal in terms of sinusoidal content + noise is derived simultaneously. A fast sequential Least Squares algorithm is used which, for a given basis of r distinct frequency components, recursively derives the Least Squares estimates of the associated amplitudes and phases. While a direct calculation would achieve a O(nr 2 ) (n being the total number of samples) complexity, the main cost of our implementation is only of 4r multiplications per sample.

The main function is fastls.m7 , whose syntax is the following:

[b,xc,er] = fastls(s,z,n)

The input and output parameters are:

• s, signal to be analyzed; • z, z = [z1, z2, z3,...], complex poles;

• n, length of the analysis snapshot;

• b, matrix of the instantaneous amplitudes sequence;

• xc, sinusoidal part of the signal;

• er, residual (s = xc + er). A demo script is also included: demo1_fastls.m.

Enhanced resampling

Once estimated using the above described methods, the parameters of the partials can be resampled using the method proposed in [START_REF] Raspaud | Enhanced resampling for sinusoidal modeling parameters[END_REF] which is suitable for general signals (non necessarily zero-centered or uniformly-sampled) and used for time scaling purposes as in [START_REF] Raspaud | A Generalized Polynomial and Sinusoidal Model for Partial Tracking and Time Stretching[END_REF]. The syntax is:

dst = enhanced_resample (src, src_param , dst_param)
where src and dst are the source and destination (resampled) signals, respectively. The other input parameters src_param and dst_param are either the sampling rate given as a scalar (uniform sampling case) or the sampling times given in a vector (non-uniform sampling case). This function resamples the source signal according to the specified rates or times (see the test_global.m8 function for a small example).

SPECTRAL MODELS AND APPLICATIONS

In this section, tools related to the estimation of the spectral envelope of a sound spectrum are presented, with applications to the study of the timber and the multipitch analysis.

ARMA envelope estimation

We proposed in [12] new algorithms for estimating autoregressive (AR), moving average (MA), and ARMA models in the spectral domain. These algorithms were derived from a maximum likelihood approach, where spectral weights are introduced in order to selectively enhance the accuracy on a predefined set of frequencies, while ignoring the other ones. This is of particular interest for modeling the spectral envelope of harmonic signals, whose spectrum only contains a discrete set of relevant coefficients. In the simple case of AR modeling, we proved that the proposed algorithm converges to the optimal solution, and that the convergence rate is enhanced by remapping the poles at each iteration. In the context of speech processing, our simulation results showed that the proposed method provides a more accurate ARMA modeling of nasal vowels than the Durbin method.

The main demonstration function is test.m9 , which performs the numerical simulations presented in [12], and plots the corresponding figure.

Timbral descriptors

Most of the approaches dealing with timbre description consider a concise encoding of the spectral envelope like the Mel-Frequency Cepstral Coefficients (MFCCs). Alternatively, we studied in [START_REF] Lagrange | Explicit modeling of temporal dynamics within musical signals for acoustical unit similarity[END_REF] descriptors which are encoding the pseudo-periodic evolution of some relevant part of the envelope through time and demonstrated that those new features can conveniently be combined with the previous ones in order to improve the description capabilities of the features set. Also, it is commonly considered that all the data is described using the same set of features. However, one can easily notice major discrepancies in some realistic settings between the database and the request side, for example regarding audio quality. Considering the matching scheme proposed in [START_REF] Lagrange | Robust similarity metrics between audio signals based on asymmetrical spectral envelope matching[END_REF], a smooth description of the spectral content is required on the database side in order to maximize generalization properties. To that end, we studied various envelopes based on AR modeling and the True Envelope approach. On the request side, the audio quality can be much lower, only some prominent spectral peaks are considered.

The script demo.m10 computes a large set of those spectrotemporal features over a predefined signal and displays its selfsimilarity matrix. Features based on MFCC computation rely on the Matlab code published by Dan Ellis [START_REF] Ellis | PLP and RASTA (and MFCC, and inversion) in Matlab[END_REF].

EM-based Multiple pitch estimation

The problem of multi-pitch estimation consists in estimating the fundamental frequencies of multiple harmonic sources, with possibly overlapping partials, from their mixture. In [START_REF] Badeau | Expectationmaximization algorithm for multi-pitch estimation and separation of overlapping harmonic spectra[END_REF], we introduced a novel approach for multi-pitch estimation, based on the statistical framework of the expectation-maximization algorithm, which aims at maximizing the likelihood of the observed spectrum. The proposed method was particularly promising, due to its robustness to overlapping partials, and its capacity to simplify the multi-pitch estimation task into successive single-pitch and spectral envelope estimations. It requires a proper initialization, involving a first stage of basic multi-pitch estimation for instance, and could advantageously make use of heuristics, in order to avoid staying trapped in local maxima. The effectiveness of this approach was confirmed by our simulations in the context of musical chord identification, performed on audio-like synthetic signals.

The main function is test.m11 , which performs the numerical simulations presented in [START_REF] Badeau | Expectationmaximization algorithm for multi-pitch estimation and separation of overlapping harmonic spectra[END_REF], and plots the corresponding figures.

AUTOMATIC MUSIC TRANSCRIPTION

In reference [START_REF] Bertin | Enforcing Harmonicity and Smoothness in Bayesian Non-negative Matrix Factorization Applied to Polyphonic Music Transcription[END_REF], we presented a new method for automatic music transcription. The proposed approach was based on a Bayesian model which introduces harmonicity and temporal smoothness constraints into the non-negative matrix factorization of timefrequency representations, providing a meaningful mid-level representation of the data. The model involves superimposed Gaussian components having a harmonic structure, while temporal continuity was enforced through an inverse-Gamma Markov chain prior. We then presented two algorithms which perform the maximum a posteriori (MAP) estimation of the model parameters. The first one [START_REF] Bertin | Enforcing Harmonicity and Smoothness in Bayesian Non-negative Matrix Factorization Applied to Polyphonic Music Transcription[END_REF] is a space-alternating generalized expectationmaximization (SAGE) algorithm, whereas the second one [START_REF] Bertin | Fast bayesian NMF algorithms enforcing harmonicity and temporal continuity in polyphonic music transcription[END_REF] is a novel multiplicative algorithm designed for minimizing an objective function composed of the Itakura-Saito divergence plus a prior penalty term. The proposed algorithms outperformed other benchmarked NMF approaches in a task of polyphonic music transcription, evaluated on a realistic piano music database.

Thus the program included in the DESAM Toolbox 12 computes the fast Bayesian constrained Itakura-Saito NMF of a magnitude ERB (equivalent rectangular bandwidth) transform, with basis spectra representing partial clusters and fundamental frequencies on the MIDI scale tuned at 440 Hz.

The main function is bertin_multipitch.m, whose syntax is the following: bertin_multipitch(wavfile, framewise_f0file,notewise_f0file)

The parameters are:

• wavfile, input wave file to be transcribed; • framewise_f0file, output file with framewise fundamental frequency (f0) transcription every 10 ms;

• notewise_f0file, output file with transcription of the onset, offset and f0 of each note.

CONCLUSION

The initial version of the DESAM Toolbox was described. It introduces a rather large set of processing tools for estimating parameters of widely used spectral models. This toolbox is therefore aimed at the researchers community interested in the modeling of musical audio but could be of potential interest for anyone interested in audio and acoustics. The Matlab code is distributed under the GNU Public License (GPL) and anyone willing to add new methods and/or improve the existing ones is encouraged to contact the maintainer of the toolbox (Mathieu Lagrange).

Further development will include, a more coherent Application Programming Interface (API) for methods tackling similar problems, Octave compatibility, and the development of experimental testbeds for evaluating and comparing the proposed methods.

Figure 1 :

 1 Figure 1: Overview of the DESAM Toolbox.

Figure 2 :

 2 Figure 2: Fourier Spectrogram of glockenspiel tones.

Figure 3 :

 3 Figure 3: Estimating the frequencies of glockenspiel tones using the fast HR subspace tracking method. Notice how the frequency overlap is reduced compared to a conventional Fourierbased spectrogram.

Figure 4 :

 4 Figure 4: Tracking the harmonics of a clarinet using the LP method.

http://www.tsi.telecom-paristech.fr/aao/logiciels-et-bases-dedonnees/desamToolbox.tgz

Directory "sinusoidalModels/shortTerm/phaseBased"

Directory "sinusoidalModels/shortTerm/highResolution_telecomParisTech".

Directory "sinusoidalModels/longTerm/telecomParisTech_david"

Directory "sinusoidalModels/longTerm/amplitudeEstimation"

Directory "sinusoidalModels/longTerm/parametersResampling".

Directory "spectralModels/envelope/telecomParisTech_badeau".

Directory "spectralModels/envelope/telecomParisTech_lagrange"

Directory "spectralModels/harmonicMixture/telecomParisTech_badeau".

The research leading to this paper was supported by the French GIP ANR under contract ANR-06-JCJC-0027-01, Décompositions en Éléments Sonores et Applications Musicales -DESAM. Mathieu Lagrange and Nancy Bertin have contributed to the work presented in this paper when they were working at CNRS LTCI.


Sylvain Marchand
email: sylvain.marchand@labri.fr

Dominique Fourer
email: dominique.fourer@labri.fr

BREAKING THE BOUNDS: INTRODUCING INFORMED SPECTRAL ANALYSIS

Sound applications based on sinusoidal modeling highly depend on the efficiency and the precision of the estimators of its analysis stage. In a previous work, theoretical bounds for the best achievable precision were shown and these bounds are reached by efficient estimators like the reassignment or the derivative methods. We show that it is possible to break these theoretical bounds with just a few additional bits of information of the original content, introducing the concept of "informed analysis". This paper shows that existing estimators combined with some additional information can reach any expected level of precision, even in very low signal-to-noise ratio conditions, thus enabling high-quality sound effects, without the typical but unwanted musical noise.

INTRODUCTION

For decades, researchers have spent lots of efforts improving the precision of sound analysis, since it is often crucial for the perceived quality of the sound transformations. And yet this quality is not sufficient for demanding applications.

One approach is to try to improve the analysis methods even further, without guarantee of success though. Indeed, theoretical bounds may exist, indicating the minimal error (i.e. maximal quality) reachable without extra information (blind approach).

Another approach is to inject some information. This can be prior knowledge about the sound sources and / or the way the human auditory system will perceive them (computational auditory scene analysis approach). But this information can also come from a manual annotation (semi-automatic analysis). Moreover, when access to the compositional process is given, another option is to use some bits of the ground truth as additional information in order to help the analysis process. This is the concept of "informed analysis" (in opposition to the blind approach), used recently to improve sound source separation [START_REF] Parvaix | A Watermarking-Based Method for Single-Channel Audio Source Separation[END_REF]. Of course, using the whole information would be cheating. Here, we intend to determine and use only the minimal amount of information.

Where this information should be added then? It could be anywhere in the analysis / transformation / synthesis chain. We would like to quantify the minimal amount of information necessary to achieve a specific task (e.g. some audio effect) with a desired level of quality. The measurement of the quality could be done at the end of the processing chain, using either objective measurements based on signal-to-noise ratios or subjective listening tests.

Here we want to measure and improve the quality of the sinusoidal modeling approach, of great interest for sound transformations as shown by McAulay and Quatieri [START_REF] Mcaulay | Speech Analysis/Synthesis Based on a Sinusoidal Representation[END_REF] for speech signals and Serra and Smith [START_REF] Serra | Spectral Modeling Synthesis: A Sound Analysis/Synthesis System Based on a Deterministic plus Stochastic Decomposition[END_REF] for musical sounds. We start by the analysis stage, and we want to evaluate its quality objectively.

Sinusoidal analysis usually consists of two steps: peak picking (short-term analysis) and partial tracking (long-term analysis). Whereas evaluating partial tracking is still an open issue [START_REF] Lagrange | Assessing the Quality of the Extraction and Tracking of Sinusoidal Components: Towards an Evaluation Methodology[END_REF], the evaluation of the peak picking has a clear theoretical background, since the Cramér-Rao lower bounds (CRBs) give the minimal error variance for unbiased estimators of the peak parameters.

To go below these bounds, the only solution is to add some information. An interesting approach could be to use prior knowledge about the sound structure, giving constraints among the parameters of the different sinusoids. Here, we prefer to stay at the most general level, with no priors on the sound sources. We will take advantage of a few bits of the ground truth to improve the quality of the estimation of the parameters of each sinusoid.

The present work should be regarded as a proof of concept of informed sound analysis. We focus on the estimation of the frequency of the sinusoids, a well-studied problem. Even with a rather naive approach we show that we can go below the theoretical bounds, which means that we are able to extract the sinusoids with a much better precision than without information, allowing musical applications with a sufficient quality.

Starting with the most studied problem (frequency estimation) of the first analysis step (peak picking) of the sinusoidal modeling chain, we expect that the precision gain should propagate through the chain and improve the overall quality. For example, accurate spectral peaks would ease the partial tracking step, and so on.

Of course, as a long-term research, we should optimize qualitatively and quantitatively the additional information at each stage of the complete analysis / transformation / synthesis chain.

This paper is organized as follows. Section 2 explains the classic frequency estimation using the reassignment method and recalls the theoretical lower bounds for the estimation error. Section 3 then presents our new approach, namely "informed estimation", consisting in taking advantage of a few bits of additional information to derive a more precise estimate. This improvement is illustrated by our experimental results in Section 4.

CLASSIC FREQUENCY ESTIMATION

Sinusoidal modeling involves a complete analysis / transformation / synthesis chain. The estimation of the frequency of each sinusoid is often the very first step. Thus, for high-quality sound transformations, the precision of this estimation is crucial in practice. Yet, theoretical considerations show that this precision is limited.

Sinusoidal Model

Additive synthesis can be considered as a spectrum modeling technique. It is originally rooted in Fourier's theorem, which states that any periodic function can be modeled as a sum of sinusoids at various amplitudes and harmonically related frequencies. In this paper we consider the sinusoidal model under its most general expression, which is a sum of complex exponentials (the partials) with time-varying amplitudes ap and non-harmonically related frequencies ωp (defined as the first derivative of the phases φp). The resulting signal s is thus given by:

s(t) = P X p=1 ap(t) exp(jφp(t)). (1)
The amplitudes and frequencies may evolve within an analysis frame (non-stationary case) under first-order amplitude and frequency modulations. Furthermore, as the present study focuses on frequency precision (i.e. accurately measuring this parameter) rather than frequency resolution (i.e. resolving closely-spaced sinusoids), the signal model is reduced to only one partial (P = 1). The subscript notation for the partials is then useless. We define then Π0 as being the value of the parameter Π at time 0, corresponding to the center of the analysis frame. The signal s is then given by:

s(t) = exp 0 B B B @ (λ0 + µ0t) | {z } λ(t)=log(a(t)) +j " φ0 + ω0t + ψ0 2 t 2 « | {z } φ(t) 1 C C C A (2)
where µ0 (the amplitude modulation) is the derivative of λ (the log-amplitude), and ω0 (the frequency), ψ0 (the frequency modulation) are respectively, the first and second derivatives of φ (the phase). Thus, the log-amplitude and the phase are modeled by polynomials of degrees 1 and 2, respectively (see [START_REF] Girin | Comparing the Order of a Polynomial Phase Model for the Synthesis of Quasi-Harmonic Audio Signals[END_REF] for the corresponding synthesis method). These polynomial models can be viewed either as truncated Taylor expansions of more complicated amplitude and frequency modulations (e.g. tremolo / vibrato), or either as an extension of the stationary case (where µ0 = ψ0 = 0).

Reassignment Method

This paper focuses on the estimation of the frequency ω0. As mentioned in [START_REF] Betser | Estimation of Frequency for AM/FM Models Using the Phase Vocoder Framework[END_REF][START_REF] Marchand | Generalization of the Derivative Analysis Method to Non-Stationary Sinusoidal Modeling[END_REF], there are many estimators. Many efficient ones are based on the short-time Fourier transform (STFT):

Sw(t, ω) = Z +∞ -∞ s(τ )w(τ -t) exp (-jω(τ -t)) dτ (3)
where Sw is the short-time spectrum of the signal s. This involves an analysis window w, band-limited in such a way that for any frequency corresponding to one specific partial (corresponding to some local maximum in the magnitude spectrum), the influence of the other partials can be neglected (in the general case when P > 1). We use the zero-centered (symmetric) Hann window of duration T , defined on the [-T /2; +T /2] interval by:

w(t) = 1 2 " 1 + cos " 2π t T «« . ( 4
)
The reassignment method, first proposed by Kodera, Gendrin, and de Villedary [START_REF] Kodera | A New Method for the Numerical Analysis of Non-Stationary Signals[END_REF][START_REF] Kodera | Analysis of Time-Varying Signals with Small BT Values[END_REF], was generalized by Auger and Flandrin [START_REF] Auger | Improving the Readibility of Time-Frequency and Time-Scale Representations by the Reassignment Method[END_REF] for time and frequency. By considering Equation (3), one can easily derive:

∂ ∂t log (Sw(t, ω)) = jω - S w ′ (t, ω) Sw(t, ω) (5)
where w ′ denotes the derivative of w. Since the frequency is the derivative of the phase φ = ℑ(log(Sw)), we then obtain the reassigned frequency ω:

ω(t, ω) = ∂ ∂t ℑ (log (Sw(t, ω))) = ω -ℑ " S w ′ (t, ω) Sw(t, ω) « . (6)
In practice, for a partial p corresponding to a local maximum m of the (discrete) magnitude spectrum at the (discrete) frequency ωm, the estimate of the frequency is given by ω0 = ω(0, ωm).

The reassignment method seems currently the best STFT-based method in terms of efficiency and estimation precision, at least regarding frequency (see [START_REF] Betser | Estimation of Frequency for AM/FM Models Using the Phase Vocoder Framework[END_REF]). The generalized derivative method [START_REF] Marchand | Generalization of the Derivative Analysis Method to Non-Stationary Sinusoidal Modeling[END_REF] could be used too, as well as high-resolution methods [START_REF] Badeau | Performance of ESPRIT for Estimating Mixtures of Complex Exponentials Modulated by Polynomials[END_REF]. In all cases, the precision is close to optimal. The high-resolution methods improve the frequency resolution, but not the estimation precision, which is always limited by the Cramér-Rao lower bound.

Theoretical Bound

In practice, we consider discrete-time signals s, with sampling rate Fs, consisting of 1 complex exponential generated according to Equation ( 2) with an initial amplitude a0, and mixed with a Gaussian white noise of variance σ 2 -the signal-to-noise ratio (SNR) is then a0/σ. To make the parameters independent of the sampling frequency, in the remainder of this paper we normalize µ and ω (by Fs).

The analysis frames we consider are of odd length N = 2H + 1 samples (the duration of the analysis window being T = N/Fs), with the estimation time 0 set at their center. In Equation ( 3), the continuous integral turns into a discrete summation over N values, with an index from -H to +H (the fast Fourier transform is used).

When evaluating the performance of an estimator in the presence of noise and in terms of the variance of the estimation error, an interesting element to compare with is the Cramér-Rao bound (CRB). The CRB is defined as the limit to the best possible performance achievable by an unbiased estimator given a data set. For the model of Equation [START_REF] Mcaulay | Speech Analysis/Synthesis Based on a Sinusoidal Representation[END_REF], for the five model parameters, these bounds have been derived by Zhou et al. [START_REF] Zhou | On Polynomial Phase Signal with Time-Varying Amplitudes[END_REF]. We will restrict our study to the frequency parameter, and consider the asymptotic version (for a large N and a high number of observations) of the corresponding bound.

Djurić and Kay [START_REF] Petar | Parameter Estimation of Chirp Signals[END_REF] have shown that the CRB depends on the time sample n0 at which the parameters are estimated, and that the optimal choice in terms of lower bounds is to set n0 at the center of the frame, i.e. n0 = H, since the CRB depends on:

ǫ k (µ, N ) = N -1 X n=0 " n -n0 N " k exp " 2µ n -n0 N " . (7)
As explained by Zhou et al. [START_REF] Zhou | On Polynomial Phase Signal with Time-Varying Amplitudes[END_REF], the expression of the bound is different whether there is a frequency modulation or not (because this changes the degree of the polynomial associated to the phase). In the absence of frequency modulation (ψ = 0), the lower bound for the frequency ω is given by:

CRB(N, σ, a0, µ) ≈ σ 2 ǫ0 2a 2 0 N 2 (ǫ0ǫ2 -ǫ1 2 ) , (8)
whereas in the presence of frequency modulation, it turns into:

CRB ≈ σ 2 (ǫ0ǫ4 -ǫ2 2 ) 2a 2 0 N 2 (ǫ0ǫ2ǫ4 -ǫ1 2 ǫ4 -ǫ0ǫ3 2 + 2ǫ1ǫ2ǫ3 -ǫ2 3 ) . (9)
Thus, the precision of the estimation of the frequency of each sinusoid is limited by this CRB, at least without using additional information. . .

INFORMED FREQUENCY ESTIMATION

Now, ω is a frequency parameter and ω is its estimation -done using the classic estimation method (see Section 2). We are aiming at reducing the estimation error |ω -ω| by introducing a few bits of the representation of the exact value Ω at the best place in the representation of the estimated value Ω. This will lead to a frequency estimation method which is informed by the additional bits, and thus more precise.

Coding Convention

Let ω be a normalized frequency, thus within the [0; .5) interval. Let us consider then its k-length fixed-point binary representation, denoted by Ω = (Ω1, Ω2, • • • , Ω k ), Ω(i) denoting Ωi, obtained with the standard binary coding application C : R → {0, 1} k , thus:

Ω(i) = ⌊ω • 2 (i+1) ⌋ mod 2 (10)
where ⌊x⌋ denotes the integral part of x. Each coded value Ω can thus be decoded with the standard binary decoding function:

D(Ω) = k X i=1 Ω(i) • 2 -(i+1) (11)
and, in these conditions, we have |ω -D(C(ω))| < 2 -(k+1) (i.e. the limit of the precision due to the quantification using the k bits).

Information Extraction

For a given ω, the most informative area starts at the most significant bit of the error |ω -ω|. Indeed, replacing this first erroneous bit could divide the error by 2. Then, replacing in turn each subsequent bit could produce a comparable enhancement. However, this approach is not realistic since it would make the error correcting algorithm dependent on the frequency ω (which is unknown). However, for a given noise variance σ 2 , we can study the distribution of the most significant bit of the error |ω -ω| (for all possible ω). We define Iσ the index of the most significant bit (and with a significant number of occurrences). As shown in Figures 1 and2, this index is close to the one of the mode of the distribution, and Iσ is a growing function of the noise parameter σ. Once Iσ is known, the procedure for extracting I containing the d bits of information is almost straightforward:

C ← C(ω) I(1 : d) ← C(Iσ -1 : Iσ + d -2
) except that we choose to extract the d-bit sub-code from 1 bit before Iσ, to be able to handle special cases (see below).

Error Correction

In practice, σ is unknown and has to be estimated. We could use the method proposed in [START_REF] Meurisse | A New Analysis Method for Sinusoids+Noise Spectral Models[END_REF], since this method tends to overestimate the noise level which will result in an underestimation of Iσ. This will lead to a loss of efficiency of the information, but the results will be at least as good as with the classic estimation.

Overestimating Iσ would be dangerous. Indeed, the first Iσ-1 bits must be reliable, otherwise the informing bits might lead to a wrong representation possibly producing an important error (thus decreasing the estimation precision instead of increasing it).

In the present work, σ is supposed to be known. The design of a complete method including the estimation of σ is part of our future research.

We define the d-informed coded estimation of ω from the representation of ω as follows:

Ω = 0 B B @ Ω1, • • • , ΩIσ-1 | {z } Ω(1:Iσ -1) , I(2), • • • , I(d) | {z } I(2:d) , ΩIσ+d-1 , • • • , Ωk | {z } Ω(Iσ +d-1:k) 1 C C A (12
) and the informed estimation of the frequency is ω = D( Ω).

Whereas the informing bits are reliable, we might deal with the case of ∃i ∈ (0; Iσ), Ω(i) = Ω(i). In this case, modifying the bit values placed after Iσ may increase the error |ω -ω|, above the original estimation error |ω -ω|. Indeed, two close values can have very different representations. This is the case for for several signal-to-noise ratios (SNR = -8dB, 0dB, and 10dB) in the stationary case and with a precision of k = 24 bits. X = (0, 0, 1, 1, 1) and Y = (0, 1, 0, 0, 0). The binary representations are almost completely different, although we obtain Y by incrementing X only by 1. Fortunately, by definition of Iσ, we have:

|ω -ω| < 2 -Iσ (13)
and thus the Iσ -1 first bits of the representation of ω and ω can only differ by 1. This happens if and only if Ω(Iσ -1) = Ω(Iσ -1), that is I(1) = Ω(Iσ -1).

Finally, the algorithm taking advantage of the information I to compute a more precise estimate of the frequency ω is:

Ω ← C(ω) Ω(Iσ : Iσ + d -2) ← I(2 : d) ω ← D(Ω) if I(1) = Ω(Iσ -1) then Ω ante ← Ω(1 : Iσ -1) Ω post ← Ω(Iσ : k) ω + ← D(inc(Ω ante ), Ω post ) ω -← D(dec(Ω ante ), Ω post ) if |ω -ω + | < |ω -ω -| then ω ← ω + else ω ← ω - end if end if
where inc and dec stands for incrementing and decrementing the binary representation, respectively.

For more robustness, we have also to consider the eventuality when Iσ is overestimated, resulting in the violation of the Equation ( 13) by both ω + and ω -. This problematic case is easily detected when |ω -ω| ≥ 2 -Iσ . Then, it is safer to revert to the ω estimate.

Thanks to this algorithm, the first Iσ + d -2 bits of Ω are now reliable, i.e. Ω(i) = Ω(i) for i < Iσ + d -1. Thus ω gets closer to ω as d grows, and the estimation precision increases.

Theoretical Bound

We can also define a theoretical lower bound in the informed case, supposing the suitability of each additional bit of information. Indeed, in the best case, each bit of information is able to divide the standard deviation of the error by 2. Thus, for a number of d informing bits, an informed lower bound (ILB) for the variance of the error is given by:

ILB(d, N, σ, a, µ) = CRB(N, σ, a, µ)/2 2•d . (14)
In fact, the ILB is limited by the k-bit precision of the binary representation, unless the CRB itself falls below this precision. This gives an ILB consisting of 3 line segments, as shown in Figure 5: 1. the first one given by Equation ( 14), where the d bits of information are not enough to reach the k-bit precision limit;

2. the second (horizontal) segment where less than d bits are needed to reach the (constant) k-bits precision limit;

3. the third segment where the CRB allows to exceed the k-bit precision without any information (the estimation precision being then better than the representation precision).

PRACTICAL EXPERIMENTS AND RESULTS

With the informed approach, we can enhance the estimation precision even below the CRB (down to the ILB). We demonstrate this with numerical simulations. In practice, we are now able to reach any level of precision, at the expense of additional informing bits. The frequency trajectories of the partials are then much more accurate, and the unwanted musical noise due to the estimation errors is now inaudible.

Simulation Results

Let us first consider a discrete-time signal s with sampling rate Fs = 44100Hz, consisting of 1 complex exponential of amplitude a0 = 1 according to Equation (2) plus a Gaussian white noise of variance σ 2 . The signal-to-noise ratio (SNR) expressed in dB is 10log 10 (a 2 0 /σ 2 ) and goes from -20dB to +50dB by steps of 5dB. We consider frames of size N = 511, and we use the Hann analysis window w, defined for continuous time by Equation (4).

For each SNR, we consider 99 frequencies (ω0) linearly distributed in the (0, Fs/2) interval, and 9 phases (φ0) linearly distributed in (-π, +π). The amplitude modulation (µ0) is either 0 (stationary case) or one of 5 values linearly distributed in the [-100, +100] interval (non-stationary case). The frequency modulation (ψ0) is either 0 (stationary case) or one of 5 values linearly distributed in [-10000, +10000] (non-stationary case).

The values for Iσ (see Figure 2) were statistically estimated from 500 random values uniformly distributed in the ranges above.

We then compare the classic reassignment method (Section 2) with its informed variant (Section 3), and plot the Cramér-Rao and informed lower bounds. We consider two situations: either a fixed number d = 5 of informing bits (Figure 5) or all the necessary bits to reach the target precision corresponding to k = 16 or 24 bits (Figure 6).

When looking at the results of these experiments, as expected the informed reassignment exhibits a variance of the estimation error lower than CRB in every case.

Below -10dB (high-error range), the noise conditions are so bad that Iσ = 1 and every bit of information is useful.

Above -10dB, our informed method is less efficient because from time to time some informing bits are useless. So it is more difficult to reach the ILB (see Figure 5), although this is possible if the whole information can be used (see Figure 6).

It is also interesting to notice the robustness of the informing algorithm, which also works in the non-stationary case.

Vibrato Sound Experiment

The second experiment consists in estimating the (non-stationary) frequency trajectory of a partial with a vibrato as described in Figure 3. We clearly see that the trajectory obtained with the classic reassignment method is erroneous. These errors may result in an annoying musical noise. However, when informing the estimation up to 16 bits, the error is reduced and this noise becomes inaudible. Figure 3: Frequency estimation of a non-stationary sinusoid of frequency ω(t) = 220+2 sin(2π•2t) (in Hz), using the classic and informed versions of the reassignment method. The classic method obtains an irregular trajectory due to estimation error whereas the informed method trajectory is more reliable but quantified.

Natural Sound Experiment

The third experiment considers a 220-Hz piano tone of approximately 2 seconds. We first obtained the reference sinusoidal parameters from a real piano tone by the classic estimation in the absence of noise (so that the estimation is almost perfect). Second, the reference tone is obtained from these parameters by classic additive synthesis (with linear amplitude and phase interpolations).

Then a Gaussian noise with fixed variance σ 2 is added to the reference tone. Since we know σ and the amplitude of each partial, the signal-to-noise conditions are known (see Figure 4), so that we can extract all the needed information I to reach the desired precision (k = 16 bits) with our new method. We use then the classic and informed versions of the reassignment method to estimate the sinusoidal model parameters, and resynthesize the sounds using the classic additive synthesis.

As shown by informal listening tests1 , whereas an annoying musical noise can be heard with the sound obtained by the classic method, the one obtained with the informed method is similar to the reference sound.

This piano tone consists of 5 partials during 187 frames (approx. 2.16s), and thus represents 187×N ×16 = 1528912 bits at CD quality. The total information for the partials frequency at a precision of k = 16 bits is then 5×187×16 = 14960. However, in this experiment, only 10625 bits are needed with our algorithm to reach the desired precision. Our method can still be improved, to use even less information. But note that the additional information represents less than 0.7% (for only 5 partials, though) of the signal information. Such a low information ratio allows the use of watermarking techniques to embed the additional information within the signal itself, as in [START_REF] Parvaix | A Watermarking-Based Method for Single-Channel Audio Source Separation[END_REF].

CONCLUSIONS AND FUTURE WORK

In this paper, we have introduced a new method for increasing the precision of the sinusoidal analysis, a way below the Cramér-Rao lower bound (CRB), thanks to some a priori knowledge about the sinusoidal parameters to be estimated. This more precise estimation produces sounds of much higher quality, enabling audio effects without typical but unwanted "musical noise".

The present work should be regarded rather as a proof of concept of "informed analysis". Indeed, the informing method is still rather naive, and does not always reach the new informed lower bound (ILB) as it should. Investigating coding and information theories to enhance our method is part of our future research.

However, we have already shown that it is possible to decrease the estimation error of the frequency parameter with at least 2 bits of information, even in the non-stationary case, when the noise level is known though.

Our aim is now to design a complete informed analysis method, including the estimation of the noise level, to fully demonstrate its practical interest. The new method would also be able to inform other sinusoidal parameters, e.g. phase and amplitude.

Our long-term perspective is a complete informed analysis / synthesis chain able to create some additional information from the musical content (e.g. the several tracks of a musical piece) and use it with the sound signal (e.g. the mixed stereo signal of a CD-audio) to allow sound transformations of high quality.

The "informed" concept opens up new research horizons, and should lead for example to new source separation or music information retrieval methods.

Figure 2 :

 2 Figure 2: Index Iσ of the most significant bit of the estimation error |ω -ω| of the reassignment method (window size N = 511) as a function of the signal-to-noise ratio, in the stationary (top) and non-stationary (bottom) cases.

Figure 1 :

 1 Figure1: Histograms of the most significant bit index of the estimation error |ω -ω| of the reassignment method (window size N = 511) for several signal-to-noise ratios (SNR = -8dB, 0dB, and 10dB) in the stationary case and with a precision of k = 24 bits.

Figure 4 :

 4 Figure 4: Overall signal-to-noise ratio (top) and instantaneous bit rate (bottom) required to inform a sound of piano (with 5 partials).

Proc. of the 13 th Int. Conference on Digital Audio Effects (DAFx-10), Graz, Austria , September 6-10, 2010

Sound examples available on-line at URL: http://dept-info.labri.fr/~sm/DAFx10/

ACKNOWLEDGMENTS

This research was partly supported by the French ANR (Agence Nationale de la Recherche) DReaM project (ANR-09-CORD-006).