Jonathan Pinel

Laurent Girin
email: laurent.girin@gipsa-lab.grenoble-inp.fr

"Sparsification" of Audio Signals using the MDCT/IntMDCT and a Psychoacoustic Model -Application to Informed Audio Source Separation

Sparse representations have proved a very useful tool in a variety of domain, e.g. speech/music source separation. As strictly sparse representations (in the sense of ℓ 0 ) are often impossible to achieve, other ways of studying signals sparsity have been proposed. In this paper, we revisit the irrelevance filtering analysis-synthesis approach proposed in (Balazs et al., IEEE Trans. ASLP, 18(1), 2010), where the TF coefficients that are below some masking threshold are set to zero. Instead of using the Gabor transform and a specific psychoacoustic model, we use tools directly inspired from perceptual audio coding, for instance MPEG-AAC. We show that significantly better "sparsification performances" are obtained on music signals, at lower computational cost. We then apply the sparsification process to the informed source separation (ISS) problem and show that it enables to significantly decrease the computational cost at the ISS decoder.

INTRODUCTION Sparse representations of signals are representations

where most of the signal coefficients are zero (or close to zero, in the weak sense). They are interesting in signal processing algorithms because the "useful" information to be processed is concentrated in a small proportion of the representation space. Additionally, in some cases, the coefficients corresponding to different signals can be disjoint. Sparsity is thus used in a large variety of audio applications, such as enhancement [START_REF] Moussallam | Sound enhancement using sparse approximation with speclets[END_REF], source separation [START_REF] Mohimani | A fast approach for overcomplete sparse decomposition based on smoothed ℓ 0 norm. Signal Processing[END_REF][START_REF]Sparse decomposition of stereo signals with matching pursuit and application to blind separation of more than two sources from a stereo mixture[END_REF][START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF], or music transcription [START_REF] Abdallah | Unsupervised analysis of polyphonic music by sparse coding[END_REF]. Sparsity is also the core principle of compressed sensing.

Many works have been dedicated to studying sparse representations [START_REF] Daudet | Sparse and structured decompositions of signals with the molecular matching pursuit. Audio, Speech, and Language Processing[END_REF][START_REF] Nesbit | Oracle estimation of adaptive cosine packet transforms for underdetermined audio source separation[END_REF]8]. In the audio case, signals may sometimes be sparse in the time domain (e.g., with silent portions and/or Laplacian or Gaussian distribution) but it is well-known that they are far more sparse in the Time-Frequency (TF) domain (see Fig. 1 for an example). When unicity of decomposition is not at stake, it has been shown that even more sparse representations can be found using overcomplete bases (e.g., "8 * MDCT" [START_REF] Plumbley | Sparse representations in audio and music: From coding to source separation[END_REF]).

The intuitive way of measuring the sparsity of a given x 0 = # j, x j = 0 .

However, in the strict sense of ℓ 0 , "real-life" audio signals are generally not sparse at all: even if some coefficients are close to zero they are generally non zero.

To overcome this problem, different measures have been proposed to study signals sparsity, using tools such as ℓ p norms/quasinorms, tanh function, kurtosis or Gini index [START_REF] Hurley | Comparing measures of sparsity[END_REF]. However as shown in [START_REF] Karvanen | Measuring sparseness of noisy signals[END_REF], one must be careful when using those measures as they can lead to quite wrong assessments in certain circumstances.

In [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], the authors use a different approach: instead of looking for a representation that is strictly sparse (in the sense of ℓ 0 ), they use a representation where the audio signals are sparse in the weak sense (the TF domain using the Gabor transform 2 ) and they set the lower coefficients to zero so that the representation becomes strictly sparse. Then, the "sparsified" time-domain signal in obtained using the inverse TF transform. To ensure preservation of audio quality, their algorithm uses a PsychoAcoustic Model (PAM) based on simultaneous frequency masking 3 to adjust the selection threshold of what they call the "irrelevance filter".

Although different in implementation and destination, this algorithm is related to Perceptual Audio Coding (PAC) [START_REF] Painter | Perceptual coding of digital audio[END_REF], such as MPEG AAC [START_REF] Brandenburg | Overview of MPEG audio: Current and future standards for low bit-rate audio coding[END_REF]. PAC uses a PAM to estimate the maximum noise power that can be introduced by quantization at each frequency, and then adjusts the allocation of binary resource accordingly: the less relevant (i.e. more masked) a coefficient is, the coarser its quantization can be (and some low but non-zero coefficients are actually coded to zero).

Despite the more or less clear link between the irrelevance filter algorithm and PAC, or maybe because of this more or less clear link, the authors of [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF] have used specific TF transform and PAM, with specific settings. In the present paper, we revisit their algorithm using tools that are much closer to the PAC approach and more efficient: instead of the Gabor transform, we use either 1 Actually ℓ 0 is not a norm but the limit of (ℓ p ) p when p → 0 with ℓ p the usual norms/quasinorms. However this is not important here. 2 Basically, the Gabor transform is a discrete Short-Term Fourier Transform (STFT) with specific conditions for analysis-synthesis signal reconstruction. 3 When two pure tones or narrow-band noises close in frequency and with significantly different power are produced simultaneously, the human hear may perceive only the loudest [START_REF] Zwicker | Psychoacoustics Facts and Models[END_REF]).

a Modified Discrete Cosine Transform (MDCT) or its integer version (IntMDCT), similar to the one used in AAC (MPEG-4), and we use a PAM also directly inspired from AAC. Using those PAC tools, we show that a better "sparsification" of audio signals can be obtained when compared to the results of [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], with a simpler and more efficient signal decomposition/reconstruction framework.

Finally, we show that the proposed sparsification process can be exploited efficiently in the audio informed source separation (ISS) system proposed in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF]. In this system, the source signals are assumed to be available and the (linear instantaneous) mixing process is assumed to be controlled at the so-called ISS encoder. An a priori information about those sources and the mixing process is extracted and embedded within the mixture signal using a high-capacity data hiding technique [START_REF] Pinel | A high-capacity watermarking technique for audio signals based on mdct-domain quantization[END_REF]. At the ISS decoder, where only the mix signal is available, the side-information is extracted and used to help the separation process to deliver high-quality separated sources. We show that the sparsification process can be used as a pre-processing step for such ISS system to significantly reduce the computational cost of the separation. This paper is organized as follows: Section 2 presents the irrelevance filter algorithm in a general manner. Section 3 provides the details of our implementation using the MDCT and AAC-inspired PAM. Section 4 and 5 present experiments and results, and conclusions are drawn in Section 6.

THE BASIC "IRRELEVANCE FILTER" ALGO-RITHM

The block diagram of the irrelevance filter/sparsification algorithm is presented in Fig. 2. A time-frequency representation X of the signal x is calculated (block ①) (using the Gabor transform in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF] and the (Int)MDCT in the present study). A masking threshold M x is derived using a psychoacoustic model (block ②). The signal power spectral density (PSD) S x is calculated (as the square of TF coefficients; block ③) and is compared with the masking threshold M x (block ④). This yields a binary mask m that is used to set to zero the coefficients of X that are below the masking threshold (block ⑤):

∀(t, f ), m(t, f ) = 0 if S x (t, f ) < M(t, f ), 1 else. ( 2
)
AES 42 ND

① ② ③ ④ ⑤ ⑥ α x X M S x m X s x s
Fig. 2: Block diagram of the system.

∀(t, f ), X s (t, f ) = X(t, f ) • m(t, f ). (3)
This binary masking process leads to the sparsified representation X s in the TF domain. The signal is finally transformed back into the time-domain sparsified signal x s using the inverse TF transform (block ⑥).

Following the same line as in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], a parameter α (in dB) controls an arbitrary translation of the masking threshold M x . As mentioned by the authors of [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], "this shift gives a conservative way to deal with uncertainty effects resulting from removing time-frequency components and with inaccuracies in the masking model." The goal is to experimentally find the α value that provides the sparsest representation without impairing the audio quality. Different values of α will be tested and discussed in Section 4.

IRRELEVANCE FILTER REVISITED

MDCT and IntMDCT

As mentioned before, the Gabor transform was used as the TF transform in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF]. In the present study, we propose to use the Modified Discrete Cosine Transform (MDCT) or its integer version (IntMDCT). We first explain the foundations of this proposition, and then we provide a very brief technical presentation of the MDCT/IntMDCT (technical details can be found in many papers, e.g., the foundation paper [START_REF] Princen | Analysis/synthesis filter bank design based on time domain aliasing cancellation[END_REF].)

The MDCT is a real-valued time-frequency transform commonly used in audio signal processing, since it presents several interesting properties: i) the MDCT is critically sampled (it has the same overall number of coefficients in the time domain and the time-frequency domain), while being a lapped transform (with 50% overlap), and ii) under simple conditions, the MDCT ensures perfect reconstruction of the signal (when the transformed coefficients are not modified); more generally, even if the coefficients are modified, it has very good robustness against block effects during signal reconstruction by inverse MDCT (IMDCT). This property is intrinsically exploited in AAC compression to minimize the effects of MDCT coefficients quantization.

The combination of i) and ii) makes the MDCT a very interesting transform in the present study, since it has the potential for a very efficient decomposition/reconstruction of the signal, while ensuring the audio quality of the reconstructed sparsified signal. Also, and perhaps more importantly, the MDCT is characterized by the time-domain aliasing cancellation (TDAC) property: if the MDCT coefficients of a given signal are modified and the time-domain signal is reconstructed, the time-domain signal is modified as compared to the original signal, but when reapplying the transform to the modified signal the same modified coefficients are exactly recovered, despite the overlap process of the reconstruction. In other words the cross-frame influence of the overlap cancels out when returning to the transformed domain. Consequently, going anew to the TF domain from a sparsified reconstructed signal leads to a strictly sparse representation, i.e. the coefficients that have been zeroed by the irrelevance filter remain strictly equal to zero. Technically, the MDCT coefficients of a given frame t of N samples (N being even) of the host signal x is given for each f ∈ 0, N 2 -1 by:

X(t, f ) = 2 √ N N-1 ∑ n=0 x(t, n)w a (n) cos 2π N n ′ f ′ , (4)
where w a is the analysis window,

x(t, n) = x n + t N 2 , n ′ = n + N 4 + 1 2 , and f ′ = f + 1 2 .
The inverse transformation (IMDCT) of the same frame is given for each n ∈ [0, N -1] by:

x(t, n) = 2 √ N w s (n) N 2 -1 ∑ f =0 X(t, f ) cos 2π N n ′ f ′ , (5)
with w s the synthesis window. To ensure perfect reconstruction of the signal, w a and w s must satisfy the Princen-Bradley conditions [START_REF] Princen | Analysis/synthesis filter bank design based on time domain aliasing cancellation[END_REF]. When w a = w s = w (which is often the case), those conditions can be writ-ten as:

∀n ∈ 0, N 2 -1      w 2 (n) + w 2 n + N 2 = 1 w (n) = w (N -1 -n) (6)
In this paper we use the Kaiser-Bessel Derived window, commonly used with the MDCT (for example in MPEG-AAC). The window length N is set to 2048, an usual choice for 44.1kHz audio signals.

As for the IntMDCT, as said earlier this transform is an integer approximation of the classical MDCT. The technique to achieve this approximation is called the lifting scheme and is described in several papers, e.g., [START_REF] Geiger | Intmdct -a link between perceptual and lossless audio coding[END_REF]. The basic principle is to decompose the MDCT matrix (and the windowing process) into a product of matrices composed of 2 × 2 matrices of the form:

L a = 1 0 a 1 , (7)
with a ∈ R. This matrix L a (which inverse is L -a ) represents the linear application:

L a : R 2 -→ R 2 (x, y) -→ (x, y + ax) (8)
The principle of the lifting scheme is to replace all these applications by their integer approximation:

intL a : Z 2 -→ Z 2 (x, y) -→ (x, y + [ax]) (9)
where [.] denotes the integer rounding operation. The inverse of intL a is intL -a but as the application is an automorphism of Z 2 , it is exactly invertible, even with finite-accuracy computations.

PAM

The PAM used in our implementation is directly inspired from the PAM of the MPEG-AAC standard, and similar to the one that we used in [START_REF] Pinel | A high-capacity watermarking technique for audio signals based on mdct-domain quantization[END_REF]. The calculations are made in the time-frequency domain, however the transform used for the computations of the PAM is not the MDCT but the FFT. The main computations consist in a convolution of the FFT power spectrum of the host signal with a spreading function that models elementary frequency masking phenomenons, to obtain a first masking curve. This curve is then adjusted according to the tonality of the signal. 4 After that, some pre-echo control is applied, and finally the threshold is translated by the "conservative factor" α (in dB), resulting in the final masking threshold M x . As mentioned earlier, different values of α will be tested in Section 4.

EXPERIMENTS ON SPARSIFICATION

Settings

As in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], the algorithm has been evaluated in a listening experiment which goal is to find the value of the threshold translation parameter α. In those experiments, the MDCT is used; note that the two transforms provide quite close coefficient values, so that the sparsification results obtained with those two transforms are similar. The tested values of α are selected to -3, -4.5 and -6 dB after preliminary listening experiments. 10 musical excerpts (of different musical styles) of 5 seconds duration were used and 8 normal hearing subjects completed the experiments. The test was a classical ABX test: for each configuration (one excerpt and one alpha, unknown to the subject), the listener was presented a reference (the original signal) and two other signals in a random order. One of the signal was the reference and the other the sparsified signal. The subject had to chose which one was the original.

Results

For α = -6 dB, the percentage of correct answer (i.e. identification of reference signal) is 53.75%; for α = -4.5 dB, it is 55%; and for α = -3 dB, it is 78.75%. Therefore, these results show that on average, the sparsification is inaudible for α = -6 dB and -4.5 dB, or at the very least, it is inaudible for most of the excerpt/subject combinations. However, a difference can generally clearly be made for α = -3 dB. 5 Table 1 shows the average proportion of suppressed coefficients and suppressed energy, for an extended set of α values. It can be seen from this table that α = -6 dB (resp. α = -4.5 dB) corresponds to a suppression of about 74% of the TF coefficients (resp. 78%), representing only less than 3% (resp. 4%) of the signal energy. 4 The main reason why the PAM of the AAC works with the FFT and not the MDCT is because the phase information given by the FFT can be used to estimate the tonality of the signal in a better way than with the MDCT. 5 The PAM is not the same as the one used in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF] so the range of α values and the inaudibility limit may slightly differ from the one reported in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF]. When analyzing those results, several remarks must be made:

• 74% of suppressed coefficients may seem a huge proportion, but it must be mentioned that a quite large proportion of those suppressed coefficients were generally already very low before sparsification (as they only represent a few percents of the total energy of the original signal). This is the case for most high-frequency coefficients for example, as illustrated by Fig. 3. This clearly illustrates the natural sparsity of audio signals, in the weak sense.

• Most of the test subjects were untrained listeners, hence even if the test seems to show that for α = -6 dB, the sparsification is inaudible, this result is an average result valid for naive ears. The sparsification at α = -6 dB may sometimes be audible for "good" listeners after a few training period and several listenings. For example one of the author that is familiar with the test signals and sparsification effects can hear differences for α up to -7.5 dB for some excerpts. In particular, some specific effects on specific instruments can be detected after careful listening, e.g. typical musical noise on cymbals.

• However, even the "best" listeners (including the authors) could not find any difference for α = -9 dB and lower. In this case an average proportion of about 65% of the coefficients are suppressed (representing 1.2% of the energy), which remains quite impressive.

• In [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], a proportion of 32% of suppressed Gabor coefficients, representing less than 2% of signal energy, is reported (however, quite surprisingly, 16-kHz music signals were considered in [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF], leading to a lower amount of high-frequency low-energy coefficients). Therefore, our PAC-inspired modification of the irrelevance filter algorithm using MDCT and PAM seems to lead to a significant improvement in signal sparsification, whereas the analysis/synthesis process is much more efficient.

APPLICATION TO THE INFORMED SOURCE SEPARATION PROBLEM

Informed Source Separation of linear instantaneous mixtures

In previous papers [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF], we presented an Informed Source Separation (ISS) system that is able to separate I > 2 sources from stereo linear instantaneous mixtures. This system is designed with a specific coder-decoder configuration. At the coder, source signals are assumed to be available and the mixing process is assumed to be controlled. In each time-frequency (TF) bin, the mixture signal is assumed to be composed of at most two predominant sources, and the coder looks for the two sources that provide the best separation results by local 2 × 2 matrix inversion applied to MDCT coefficients (the other separated sources are set to zero). The side-information transmitted to the decoder is the index of those two predominant sources for each TF bin (and the mixing matrix parameters). In [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF] the side-information is embedded in the mix signal using a high-capacity data-hiding technique similar to the one presented in In previous papers [START_REF] Pinel | A high-capacity watermarking technique for audio signals based on mdct-domain quantization[END_REF]. At the decoder, where the original source signals are unknown, the extraction of this side-information enables to invert the (embedded) mixture in each TF bin to recover the source signals. This system is summarized in the diagram of Fig. 4. With such informed approach, it has been shown in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF] that 5 instruments and singing voice signals can be efficiently separated from 2-channel stereo musical mixtures, with a quality that significantly overcomes the quality obtained by a semi-blind reference method and that enables separate manipulation of the source signals during stereo music restitution (i.e. remixing).

Sparsification as a pre-process for ISS

In the ISS system of [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF], the separation is made by local matrix inversion applied to the MDCT coefficients of the mixture signal, since the linear instantaneous mixture in the time-domain results in an identical linear instantaneous mixture on the source MDCT coefficients. Therefore, if the sparsification process of Section 3 is first applied to the signals involved in the separation process, this can lead to a reduced number of non-zero MDCT coefficients and a significant simplification of this separation process. Here, we propose to apply the sparsification process independently to each source signal at the coder level before the mixing process. This sparsification is assumed to have no consequence on the quality of each source signal taken separately, and it is very likely to have no (or not much) consequence on the quality of the resulting mix. For each TF bin, we can thus mention the following cases:

Case 1: The MDCT coefficient of each source has been zeroed by the sparsification process; this results in a zero-coefficient for each channel of the mixture; in such case, the separation process by matrix inversion is unnecessary, all separated sources can be directly set to zero.

Case 2: The MDCT coefficient of at least one source has not been zeroed; in that case, the matrix inversion is needed, but we can distinguish the following subcases:

Case 2.1: If the MDCT coefficient of only one or two sources are non-zero, then the separation leads to the exact reconstruction of the MDCT coefficient of every (sparsified) sources (as opposed to the original system in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF], where the weakest but non-zero sources can corrupt the inversion process).

MDCT x L [n] x R [n] X R [f,t] s 1 [n] s 2 [n] s I [n] Oracle estimator S I [f,t] S 1 [f,t] I ft

Watermarking

Mixing (LISS)

Mixing matrix A Water- mark extraction MDCT MDCT Decoding 2 × 2 inver- sion A IMDCT IMDCT IMDCT ] , [ ˆ1 t f S ] , [ ˆt f S I ] [ ˆ1 n s ] [ ˆ2 n s ] [ ˆn s I MDCT MDCT MDCT MDCT Coding IMDCT IMDCT ] , [ t f X W L ] , [ t f X W R ] [ ~n x W L 16-bits PCM 16-bits PCM ] [ ~n x W R ] , [ ~t f X W L ] , [ ~t f X W R ] [ ~n x W L ] [ ~n x W R X L [f,t] I ft
Case 2.2: If the MDCT coefficient of more than two sources are non-zero, then the inversion of the two predominant sources can be disturbed by the other non-zeroed sources, as in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF], but also as in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF], those non-zero sources are assumed to be small compared to the two predominant sources and their influence on the separation process is assumed to be reasonable.

Note that after sparsification of the sources, the mixing process is carried out in the TF domain. Because the mixture coefficients are generally not integer (usually between 0 and 1), the resulting mixed TF coefficients are first rounded and then the mix signal is transformed back in the time-domain. This ensures the preservation of the values zeroed by the sparsification process (case 1 above), as mentioned in Section 3.1. The rounding error on non-zero coefficients is assumed to be negligible compared to their dynamics.

In the next subsection, we lead some experiments that measure the occurrence of each of those cases on realistic musical mixtures, and thus provide a first estimation of the computational gain that can be obtained at the ISS decoder

Experiments on sparsified ISS

The experiments were performed using 4 real music tracks of different styles (pop-rock, new-wave, funk, electro-jazz; duration 3, 4, 5 and 6 min) of 5 sources each (among guitar, bass, drums, lead vocals, saxophone, synthesizer, percussions). Table 2 shows the effect of the pre-mix source sparsification on the overlapping of the sources in the mix. It can be seen that on the average:

• Approx. 32% of the mix coefficients are zero (Case 1), leading directly to 32% of computational cost saving for the 2 × 2 matrix inversion of the separation process. Since approx. 65% of the coefficients are zeroed by the sparsification for each source taken separately, it can be deduced that approx. half of these sparsified coefficients are zeroed simultaneously for the 5 sources of the mix signal. Table 3: Sources overlapping in the mix after sparsification of each source.

or 2 sources (Case 2.1), meaning that the separation in those TF bin will be perfect (exact reconstruction of the sparsified sources).

To further complement those results, Table 3 shows that even in the case of more than 2 non-zero sources, the energy of the 2 most energetic sources represents on average more than 95% of the energy of all the sources (confirming the similar results presented in [START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF]). This confirms the two-predominant sources assumption on which the ISS method is based. In fact, as also shown in Table 2, the perfect reconstruction cases (Case 1 and Case 2.1) represent only about 10% of the total energy of the mix signal. Therefore, the sparsification as a preprocess for informed source separation yields only few improvements in separation quality. Its main interest in the present case lies in the very significant computational gain (32% on the average).

CONCLUSIONS AND PERSPECTIVES

The present study shows that it is possible to largely sparsify musical signals in the TF domain (in the strict sense of ℓ 0 , and in a conservative manner, i.e. with several back-and-forth transformations from time domain to TF domain), with more than 65% of zeroed coefficients without impairing audio quality. This is made possible by the use of PAC tools (MDCT and PAM), here borrowed from MPEG, which seem to be more efficient than the Gabor transform and specific PAM used in the previous inspiring study [START_REF] Balazs | Frequency sparsity by removing perceptually irrelevant components using a simple model of simultaneous masking. Audio, Speech, and Language Processing[END_REF].

The sparsification as a pre-process has been applied within the informed source separation system presented in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Parvaix | Informed source separation of linear instantaneous under-determined audio mixtures by source index embedding. Audio, Speech, and Language Processing[END_REF]. The sparsification of instrument/voice signals before making the mix leads to approx. 1/3rd computational cost saving at the ISS decoder where the separation is processed (for 5 sources). In addition, this process contributes to reduce the overlapping of the source signals in the TF domain, and thus enables sparsity-based source separation algorithms to yield better results than when applied on normal mix.

Fig. 1 :

 1 Fig. 1: Samples distribution in the time (1a) and timefrequency (1b) domains for a rock music signal (each distribution is represented on its total support).

Fig. 3 :

 3 Fig. 3: PSD (in dB) of a classical music excerpt before and after sparsification. The higher the amplitude the brighter the representation.

Fig. 4 :

 4 Fig. 4: Block diagram of the ISS system.

Table 1 :

 1 Proportion of Suppressed coefficients and suppressed energy for several values of α.

ACKNOWLEDGMENTS

This work is supported by the French National Research Agency (ANR) as part of the DReaM project (ANR 09 CORD 006).


Ben Faten
email: faten.ben-ali@gipsa-lab.grenoble-inp.fr

Ali

Laurent Girin
email: laurent.girin@gipsa-lab.grenoble-inp.fr

Sonia Djaziri Larbi
email: sonia.larbi@enit.rnu.tn

A Long-Term Harmonic plus Noise Model for Speech Signals

Keywords: speech analysis/synthesis, harmonic + noise model, long-term processing

The harmonic plus noise model (HNM) is widely used for spectral modeling of mixed harmonic/noise speech sounds. In this paper, we present an analysis/synthesis system based on a long-term two-band HNM. "Long-term" means that the time-trajectories of the HNM parameters are modeled using "smooth" (discrete cosine) functions depending on a small set of parameters. The goal is to capture and exploit the longterm correlation of spectral components on time segments of up to several hundreds of ms. The proposed long-term HNM enables joint compact representation of signals (thus a potential for low bit-rate coding) and easy signal transformation (e.g. time stretching) directly from the long-term parameters. Experiments show that it can be compared favourably with the shortterm version in terms of parameter rates and signal quality.

Introduction

In speech/music coders and analysis/synthesis systems, spectral parameters are usually extracted and processed on a shortterm (ST) basis, i.e. every 20ms or so. This is mainly due to the non stationarity of audio signals and/or real-time processing constraints. For speech signals, the evolution of the vocal tract shape and glottal source activity is often quite smooth and regular, and it can be captured in terms of slow AM/FM modulations. High correlation between successive ST spectral parameters is actually exploited for two or three consecutive ST frames in, e.g., differential coding, matrix quantization, or recursive coding [START_REF] Gray | Vector Quantization and Signal Compression[END_REF]. But for non-real-time applications (e.g., half-duplex communication, storage, or transformation), the analysis-synthesis process can be applied on a long-term (LT) basis, i.e. much larger signal windows. This is the foundation of the Temporal Decomposition (TD) technique [START_REF]Efficient coding of LPC parameters by temporal decomposition[END_REF], which consists of decomposing the trajectory of spectral parameters into "target vectors" which are sparsely distributed in time and linked by interpolative functions. TD has been recently revisited in [START_REF] Dusan | Speech compression by polynomial approximation[END_REF], where the trajectories of ten consecutive spectral vectors are modeled by fourth-order polynomials.

All those mentioned studies concern Linear Prediction Coding (LPC) parameters, widely used for speech coding. The LT approach can be extended to other spectral models, like sinusoidal models [START_REF] Mcaulay | Speech analysis synthesis based on a sinusoidal representation[END_REF] [START_REF] George | Speech Analysis synthesis and modification using an analysis by synthesis overlap add sinusoidal model[END_REF]. Thus, it was proposed in [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF] to model the LT trajectory of harmonic phases and amplitudes with a Discrete Cosine Model (DCM). In [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF] the LT frames are continuously harmonic sections with very variable size and "shape", and a fitting algorithm was proposed to automatically adjust the LT model according to the speech section content. In [START_REF] Firouzmand | Long-term flexible 2D cepstral modeling of speech spectral amplitude[END_REF], this adaptive scheme was extended to the LT modeling of harmonic amplitudes envelope parameters.

The present paper presents a new extension of [6][7] to the framework of the Harmonic + Noise Model (HNM), which is particularly appropriate for modeling mixed voiced/unvoiced speech signals. We focus on a two-band HNM, inspired by [START_REF] Stylianou | Applying the harmonic plus noise model in concatenative speech synthesis[END_REF], and we present the application of the LT modeling approach to the parameters of this two-band HNM. Those parameters are the spectral envelope (of both harmonic and noise amplitudes), the fundamental frequency F0, and the voicing cut-off frequency FV . Note that in [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF] [START_REF] Firouzmand | Long-term flexible 2D cepstral modeling of speech spectral amplitude[END_REF], only purely voiced sections where considered. In the present study, we generalize the modeling of the spectral envelope to any harmonic/noise combination, we introduce the LT modeling of FV , and we simplify the modeling of F0 (w.r.t. [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF]). We also describe and assess a complete original analysis-synthesis system based on the proposed LT-HNM. This paper is organized as follows. The HNM model is presented in Section 2. The LT modeling of HNM parameters trajectories is described in Section 3, resulting in a complete LT-HNM analysis-synthesis system. Experiments and results are given in Section 4.

The Two-Band Harmonic + Noise Model

In its general non-stationary form, the HNM can be formally expressed as:

s(n) = H h=1 a h (n)cos[φ h (n)] + ν(n), (1)
where h is the harmonic rank, H is the number of harmonics, a h (n) is the harmonic instantaneous amplitude, φ h (n) is the instantaneous phase, and ν(n) is the noise part of the signal. φ h (n) is (the sampled version of) the summation of instantaneous frequency over time (each frequency being the h-multiple of F0). For speech signals, those parameters are assumed to (slowly) vary in time, with possible "birth" and "death" of sinusoids, as in the more general sinusoidal model [START_REF] Mcaulay | Speech analysis synthesis based on a sinusoidal representation[END_REF].

In the present work, we use a simplified two-band version of the HNM in the spirit of [START_REF] Stylianou | Applying the harmonic plus noise model in concatenative speech synthesis[END_REF]. This version splits the frequency band into a harmonic band in the low frequency (LF) region, and a noise band in the high frequency (HF) region. The noise band is assumed to model HF random components with spectral coloration but no clear temporal structure. Those two bands are separated by a "boundary frequency" called the voicing cut-off (VCO) frequency FV . This two-band HNM model is very flexible in the sense that it can be used to represent purely harmonic frames (FV equal to the Nyquist frequency FNyq), purely unvoiced frames (FV = 0), or mixed voiced-unvoiced frames.

In the present study, the parameters of the two-band HNM are first extracted on a short-term (ST) basis, as in usual ST-HNM modeling, using analysis frames (indexed by k) of length w = 30ms and hop size r = 20ms. The fundamental frequency F0 is first estimated using Praat's autocorrelation method [START_REF] Boersma | Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound[END_REF] (which implicitly provides voiced/unvoiced segmentation). The VCO frequency FV is estimated using the method of [START_REF] Hermus | Estimation of the voicing cut-off frequency contour of natural speech based on harmonic and aperiodic energies[END_REF] based on the maximization of the sum of a cumulative periodic energy for the lower band and a cumulative aperiodic energy for the upper band. The estimated FV value is rounded to the nearest harmonic frequency, which becomes the last harmonic of the frame, indexed by H k . The estimation of the H k harmonic amplitude parameters a h (and initial phases φ h ) of the harmonic band is then made by least-square fitting between the (stationary) harmonic model (using the measured F0) and the signal within the k-th frame [START_REF] George | Speech Analysis synthesis and modification using an analysis by synthesis overlap add sinusoidal model[END_REF]. Finally, the frequencies fn and amplitudes an of the N k spectral components of the noise band are estimated by a peak-picking algorithm [START_REF] Mcaulay | Speech analysis synthesis based on a sinusoidal representation[END_REF], applied on the upper band of the FFT magnitude spectrum.

LT modeling of HNM parameters

3.1. The LT model and associated estimation process LT modeling of HNM parameters consists of 1) defining LT frames: in the present study a LT frame is either a continuously voiced (actually mixed voiced/unvoiced) or continuously unvoiced section of speech (as a sequence of K successive ST frames; LT frame boundaries are provided by the F0 analysis), and 2) Representing the trajectories of the HNM parameters on each LT frame by a sparse P -order time model. The goal is to reduce the data dimension from K to P + 1, with P being significantly lower than K, while preserving the essential shape of data trajectory. In the present study, we use a linear combination of cosine functions (called Discrete Cosine Model -DCM), since this model has provided good fitting and computational properties in previous studies [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF] [START_REF] Firouzmand | Long-term flexible 2D cepstral modeling of speech spectral amplitude[END_REF]:

Sm(n) = P p=0 cm,p cos(pπ n N ). (2)
Sm is either F0, FV , or a parameter of the spectral envelope of a h and an (see next section). The vector of M spectral parameters extracted at time instant n k = kr is denoted

S k = [S 1,k S 2,k . . . S M,k
] T ( T denotes the transpose operator; M is possibly equal to 1 for F0 or FV ). Thus, we actually have to model M trajectories of K values S m,k which are gathered in the M × K matrix S. Let us denote by M the (P + 1) × K "model matrix" of general term m p,k = cos(pπ n k N ), and let us denote by C the M × (P + 1) vector/matrix of model coefficients cm,p. When the order P is known, C is estimated by minimizing the weighted mean square modeling error (WMSE) at the n k instants, leading to:

C = S • W • M T • (M • W • M T ) -1 , (3)
W is a diagonal weight matrix that can be introduced to control the contribution of the data in the model computation. Also, a diagonal "regularizing" term can be added to the inverted matrix in (3) to fix possible ill-conditioning problems [START_REF] Cappé | Regularized estimation of cepstrum enveloppe from discrete frequency points[END_REF]. We do not detail this technical aspect here.

Model orders optimization

In the previous subsection, P is assumed to be known. In fact, for each LT speech section and each HNM parameter, the goal of efficient LT modeling is to automatically set the model order to a value that ensures a good trade-off between data compression (ideally P << K) and good modeling accuracy. For this aim, we propose the following algorithms.

LT modeling of F0

For F0 modeling, we define a target ratio D F 0 t for the modeling error (e.g. 1%) and apply the dichotomic search of Algorithm 1. Note that the last iteration is validated only if it leads to lower the error, and since all time frames are here assumed to have the same importance, all weights of W are set to 1. Of course, more refined fitting criteria and strategy can be used, e.g. perceptual criteria with adaptive time-weights [START_REF] Girin | Perceptual long term variable rate sinusoidal modeling of speech[END_REF].

Algorithm

LT modeling of FV

LT modeling of FV is similar to the LT modeling of F0, resulting in optimal CF V vector and PF V order. However, the modeled vectors Fv = C • M are rounded to the closest harmonic frequency, and the target error D F V t is expressed in terms of maximal deviation in (integer) number of harmonics, i.e. PF V is found as the minimum order so that the maximum modeling error remains within ±Q harmonic (Q can be set to 1 or 2).

2D-DCM modeling of the spectral amplitudes

The (harmonic and noise) amplitudes are LT modeled using a 2D modeling approach similar to the one presented in [START_REF] Firouzmand | Long-term flexible 2D cepstral modeling of speech spectral amplitude[END_REF] for purely harmonic spectra. This technique is here extended to mixed harmonic/noise sections of speech. The general principle is a two-step modeling: For each ST frame k of a given LT frame, a first DCM model of order M is applied in the frequency dimension, covering both harmonic and noise amplitudes. This model is similar to the discrete cepstrum proposed in [START_REF] Cappé | Regularized estimation of cepstrum enveloppe from discrete frequency points[END_REF] [START_REF] Galas | An improved cepstral method for deconvolution of source-filter systems with discrete cepstra: Application to musical sound signals[END_REF]. Then a second DCM of order P is applied on the resulting coefficients along the time dimension. M is variable from one LT section to the other but it is the same for all ST frames of the LT section. This enables (i) to switch from a H k + N k variablesize set of ST amplitudes to a fixed-size set of parameters that is suitable for LT modeling with (2), and (ii) to reduce the size of the parameter set to be time-modeled, since M is generally significantly lower than H k + N k . This is a major point for potential coding applications. For the same reason, we also want P to be significantly lower than K, as for F0 and FV modeling.

Therefore we propose the two-step Algorithm 2 to find an optimal joint setting for both M and P , ensuring both compact representation and modeling quality. In this algorithm, M k is the concatenation of the H k × M matrix of general term m h,m = cos(mπhF0(k)/Fnyq) and the N k × M matrix of general term mn,m = cos(mπfn(n, k)/Fnyq). Et1 and Et2 are user-defined target errors with Et1 < Et2. The search intervals are set to reasonable values, adapted to speech signals and LT frame length K. The algorithm can be refined with a dichotomic search similar to the one in Algorithm 1 for faster convergence. Also, because of the two-step structure, the al-gorithm may miss a better (M, P ) combination in the area of (Mopt, Popt). It can thus be completed with additional search within, e.g., (Mopt + i, Popt -j) or (Mopt -i, Popt + j) with i, j ∈ [START_REF] Gray | Vector Quantization and Signal Compression[END_REF][START_REF]Efficient coding of LPC parameters by temporal decomposition[END_REF]. Finally, W is here used in the first part of Algorithm 2 to give more importance to the harmonic amplitudes (weights set to 10) than to the noise peaks (weights set to 1). This was shown to ensure higher global quality for synthesized signals. In future works, a more rigorous criterion will have to be defined and tested regarding this important point.

A k = [a h (1, k), • • • , a h (H k , k), an(1, k), • • • , an(N k , k)] T
and calculate the corresponding model matrix M k . Calculate the coefficients vector D k of the frequency-DCM using a transposed version of (3) applied to A k and M k :

D k = (M T k • W • M k ) -1 • M T k • W • A k .

Synthesis of LT modeled signals

As for the signal synthesis, the harmonic part sh (n) is obtained in a very straightforward manner: the phase of each h-harmonic is obtained by summation of the h-multiple of the modeled F0 trajectory provided by (2) (plus the initial phase of the first ST frame of the LT section to preserve natural sounding). The modeled harmonic amplitudes are linearly interpolated between ST frames and sh (n) is obtained by components summation as in [START_REF] Gray | Vector Quantization and Signal Compression[END_REF]. The noise part ν(n) is generated with the overlap-add random phase sinusoid technique of [START_REF] Macon | Sinusoidal modeling and modification of unvoiced Speech[END_REF], using amplitudes sampled from the modeled spectral envelope in the noise band every ∆f = 70 Hz. Note that the modeled F0 and FV trajectories are independent, thus the upper harmonic trajectories may be "interrupted" by noise regions. This is managed using a "birth and death" process (involving local interpolation of a h to 0) [START_REF] Mcaulay | Speech analysis synthesis based on a sinusoidal representation[END_REF].

Experimental results

Data

We report the results of the LT modeling of 24 speech sentences sampled at 16kHz (13 male and female speakers and 4 languages), of total duration 50s. LT segmentation resulted in 291 LT sections with a mean duration of 0.17s and a maximum duration of 1.24s [136 voiced sections (33s) and 155 unvoiced sections (17s)]. The ST analysis hop size is r =20ms.

Compression gain of the LT modeling

We provide here the coefficient rates for the ST-HNM parameters set and the LT-HNM model, respectively denoted RST and RLT . We also compare the LT-HNM with a ST version with DCM modeling of the spectral envelope only, denoted 1D-HNM. For fair comparison, the envelope model of this 1D-HNM is calculated with a target error equal to E LT t2 . The coefficient rate of the 1D-HNM is denoted R1D and we have, for a given LT section,

RST = [ K k=1 (2 + 2(H k + N k ))]/(K.r), R1D = [K(Mopt + 3) + K k=1 H k ]/(K.r) and RLT = [(Mopt +1)(Popt +1)+PF 0 +PF v +H0 +3]/(K.r),
where H0 is the number of initial phases (issued from the first ST frame).

For the used database, the mean rate is RST = 6298 coeff/s. For example, when applying the LT-HNM with target errors (E LT t1 , E LT t2 ) = (0.6, 0.7)dB, we obtain RLT = 530 coeff/s, while R1D = 983 coeff/s (with 1D-target error E 1D t1 = 0.7dB). Hence, the LT-modeling achieves a rate gain of 91.5% compared to the ST-HNM1 and 46% compared to the 1D-HNM. To evaluate the compression gain due to 2D-DCM amplitude modeling only, we provide in Table1 the corresponding coefficients rates R A 1D and R A LT for different target error settings. It can be seen that the coefficient rates decrease when the target errors increase (as expected) and that, for a given E LT t2 , the best LT combination is systematically the one with E LT t1 immediately higher (i.e. the lower diagonal). This suggests that a more efficient overall LT modeling is obtained when the modeling of the spectral envelope is not much constrained. When comparing the optimal R A LT with R A 1D , we observe important rate gains that increase with target errors (up to 38%).

To illustrate the efficiency of the proposed LT-HNM, we plot in Fig. 1 gain is illustrated on Fig. 2 which depicts the LT model orders Mopt and Popt as a function of the mean value of (H k + N k ) over the LT section and K, respectively. The average coefficients gain is also around 2 for time-modeling, while it is more important (around 4) for frequency modeling, although with a significantly scattering. Fig. 3 provides an example of original and modeled trajectories of F0 and of the first harmonic amplitude. Both plots show that the modeled trajectories follow the original ones quite fairly given the low number of coefficients.

Quality of synthesis signals

The perceptual quality of the speech test signals modeled with the LT-HNM [for (E LT t1 , E LT t2 , D F 0 t , D Fv t ) = (0.6, 0.7, 1%, 8%)] and the reference 1D-HNM [for E 1D t1 = 0.7dB] was assessed with PESQ2 . The obtained mean scores are 2.4 and 2.3 respectively, while the ST-HNM score is 2.9. These results indicate that the LT-HNM model provides nearly the same signal quality than the 1D-HNM, while significantly reducing the coefficients rate.

Conclusion-Perspectives

A "flexible" LT-HNM model was presented, using DCM models for F0, FV and spectral amplitudes. Compared to the ST and 1D versions, a significant gain in coefficients rate was obtained. The proposed algorithms for setting the DCM orders enable a trade-off between coefficients rate gain and modeling quality. Future works will concern i) the use of perceptual criteria for LT model fitting, ii) a better LT modeling of the phase trajectories, iii) the use of the proposed model for speech transformation (e.g. time-stretching) and iv) the design of a lowbitrate LT speech coder based on the proposed LT-HNM, with "LT-quantization" of HNM parameters in the line of the one proposed in [START_REF] Girin | Adaptive long term coding of LSF parameters trajectories for large-delay/very-to utra-low bit rate speech coding[END_REF] for LPC parameters.

  Mmin to Mmax do for k = 1 to K do Concatenate harmonic and noise amplitudes into

  Decode the modeled amplitude vectors: Ãk = M k • D k . end for Calculate the WMSE E1 between all original and modeled amplitudes over the whole LT section. if E1 ≤ Et1 then Mopt ← M return (end of Part 1) end if end for Second part for P = Pmin to Pmax do Calculate the (Mopt + 1) × (P + 1) coefficients matrix C of the time-DCM by applying (3) to the spectral envelope matrix D = [D1, D2, • • • , DK ]. Decode the modeled spectral envelope matrix D = C • M. Let Dk denotes the k-th column of D. for k = 1 to K do Decode the new modeled amplitudes Ãk = M k • Dk . end for Calculate the new WMSE E2 between all original and modeled amplitude if E2 ≤ Et2 then Popt ← P and Copt ← C return end if end for

Figure 1 :

 1 Figure 1: LT model order of F0 (.) and FV (o) trajectories as a function of LT section length K for all test sections. The dashed lines are the 1 st and 2 nd bisectors. D F 0 t = 1%, D F V t = 8%.

Figure 2 :

 2 Figure 2: 2D amplitude modeling: Mopt as a function of ST amplitude vectors length (left) and Popt as a function of LT frame length K (right). (E LT t1 , E LT t2 ) = (0.6, 0.7)dB.

Figure 3 :

 3 Figure 3: Trajectories of LT-modeled F0 (left) and of the 1 st 2D-modeled harmonic amplitude (right) (E LT t1 , E LT t2 ) = (0.6, 0.7)dB.

  the DCM orders PF 0 and PF V as a function of the voiced LT sections length (number of ST frames K). An average gain of about 2 is obtained. The 2D-amplitude modeling

	E LT t1 /E LT t2	0.6	0.7	0.8	0.9	1.0	1.1
	0.4	858 738 648 585 544 496
	0.5	648 531 466 418 383 351
	0.6	-	433 367 324 296 269
	0.7	-	-	315 274 246 225
	0.8	-	-	-	246 217 197
	0.9	-	-	-	-	202 181
	1.0	-	-	-	-	-	173
	R A 1D	668 512 420 358 315 284
	Gain (%)	2	15	25	31	35	38
	Table 1: Coefficient rate R A LT of 2D-DCM amplitude modeling for different target errors (E LT t1 , E LT 1D t2 ), and coefficient rate R A of the reference 1D-HNM with E 1D t1 =E LT t2 .

Here the gain is not only due to the LT-modeling of F 0 , Fv and the spectral amplitudes, but also because the noise frequencies fn and the harmonic phases φ h do not need to be sent to the decoder.

Perceptual Evaluation of Speech Quality, ITU-T Recom. P.862.


Shuhua Zhang
email: shuhua.zhang@gipsa-lab.grenoble-inp.fr

Laurent Girin
email: laurent.girin@gipsa-lab.grenoble-inp.fr

An Informed Source Separation System for Speech Signals

Keywords: underdetermined source separation, speech mixture, speech signals sparsity, signal compression

In two previous papers, we proposed an audio Informed Source Separation (ISS) system which can achieve the separation of I > 2 musical sources from linear instantaneous stationary stereo (2-channel) mixtures, based on audio signal's natural sparsity, pre-mix source signals analysis, and side-information embedding (within the mix signal). In the present paper and for the first time, we apply this system to mixtures of (up to seven) simultaneous speech signals. Compared to the reference MPEG-4 Spatial Audio Object Coding system, our system provides much cleaner separated speech signals (consistently 10-20 dB higher Signal to Interference Ratios), revealing strong potential for audio conference applications.

Introduction

Source separation aims at recovering I unobserved source signals si[n], i ∈ [1, I], from J observations of their mixture xj[n], j ∈ [1, J]. In this paper, we consider speech signals, and we address the stereo underdetermined configuration, where I > 2 speech signals have to be separated from only J = 2 channels. This is a difficult configuration, that cannot be processed by Blind Source Separation (BSS) / Independent Components Analysis (ICA) methods developed for (over)determined mixtures (J ≥ I) [START_REF] Comon | Handbook of Blind Source Separation -Independent Component Analysis and Applications[END_REF][START_REF]Independent Component Analysis[END_REF], and that is better addressed with techniques based on sparse Time-Frequency (TF) representations of audio signals [START_REF] Bofill | Underdetermined blind source separation using sparse representations[END_REF][START_REF] Yilmaz | Blind separation of speech mixtures via time-frequency masking[END_REF][START_REF] Araki | Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors[END_REF], or other a priori information on the source signals and the mixture process.

In [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF] we proposed an Informed Source Separation (ISS) system, based on audio signals sparsity and a two-step coderdecoder structure, and dedicated to music demixing. The coder corresponds to the music signal production level (e.g., music recording/mixing in studio) where the source signals are assumed to be available and the mixing process is controlled. The decoder corresponds to the personal music player, where only the stereo mix signal is available. Parameters that characterize the source signals and the mixing process are embedded into the mixture signal at the coder level, so that they can be retrieved at the decoder and exploited for source signals separation from the mix signal. For instance, the mixture process is Linear Instantaneous Stationary Stereo (LISS) (aka constant-gain stereo panning) and the side-information consists of i) the mixture matrix, and ii) the indexes of the two predominant sources in each TF bin as provided by "Oracle" estimation at the coder. The decoder uses those information to perform local mixture inversion in each TF bin. The separation performances obtained by this system are quite impressive: Signal to Distortion Ratio (SDR) [START_REF] Vincent | Performance measurement in blind audio source separation[END_REF] gains about 20 dB are obtained for all sources of 5-source music mixtures, enabling realistic applications such as generalized karaoke/soloing. Such ISS system also has a strong potential for audio conference applications, to separate and respatialize different simultaneous speakers. Therefore, in the present paper, and for the first time, we apply the ISS system to speech signals, and assess the separation performances for "difficult" mixtures of continuous speech with many speakers (up to 7). Also, we compare our system with the MPEG-4 Spatial Audio Object Coding (SAOC) system [START_REF] Engdegård | Spatial audio object coding (SAOC): The upcoming MPEG standard on parametric object based audio coding[END_REF], which is also a system dedicated to restitute audio sources ("audio objects") from downmix signals and side-information (composed of inter-object spatial cues). The technical positioning of our system with respect to SAOC is briefly discussed in Section 3. Note that SAOC applications also concern both music and speech demixing/remixing. This paper is organized as follows. The proposed system is described in Section 2. Results obtained for speech mixtures as well as comparison with SAOC are presented in Section 3, and Section 4 concludes the paper.

The ISS system

The ISS system used in this work is depicted in Fig. 1. It is similar to the one presented in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF]. However, the parameters of the ISS process are adapted to speech signals. Also, in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF], uncompressed (16-bit PCM) mixture signals were considered (and the side-information was embedded into the mix signal using a high-capacity watermarking technique [START_REF] Pinel | A high-capacity watermarking technique for audio signals based on MDCT-domain quantization[END_REF]). In the present framework, the speech mixture signal generally has to be transmitted in dedicated channels, hence we consider the use of both uncompressed (16-bit PCM) and compressed mixture signals (MPEG-2/4 AAC [START_REF]Information technology-Generic coding of moving pictures and associated audio information-Part 7: Advanced Audio Coding (AAC)[END_REF]). The side-information is assumed to be either embedded into the bitstream or transmitted on a dedicated channel.

MDCT x L [n] x R [n] X L [f,t] X R [f,t] s 1 [n] s 2 [n] s I [n] Oracle estimator AAC compr- ession S I [f,t] S 1 [f,t] I[f,

Time-frequency decomposition

The speech source signals are non-stationary, with quite large temporal and spectral variability, and they possibly strongly overlap in the time domain. Using TF representation is known to exhibit their natural sparsity, i.e., much lower overlapping of signals in the TF domain, a foundation for "sparse separation" methods [START_REF] Yilmaz | Blind separation of speech mixtures via time-frequency masking[END_REF][START_REF] Araki | Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors[END_REF]. A linear invertible transform is required, so that the source separation problem remains linear/instantaneous in the transformed domain. And preferably, the transform sparsifies signals maximally and suppresses the blocking artifact intrinsically. In this regard, we select the Modified Discrete Cosine Transform (MDCT [START_REF] Princen | Analysis/synthesis filter bank design based on time domain aliasing cancellation[END_REF]), a lapped orthogonal transform widely used in audio compression and processing. This transformation is applied on each source signal at the coder, and on each mixture channel at both coder and decoder, while the corresponding inverse transform (IMDCT) is used at the decoder to regenerate estimated time-domain source signals from separated MDCT coefficients (Fig. 1). We use here time frames of W =768 samples (approximately 48 ms for 16 kHz speech signals), with a 50% overlap between consecutive frames. This setting enables to follow the time-dynamics of speech signals while providing a frequency resolution of about 21 Hz suitable for the separation.

Local inversion of the mixture and sources selection

As in the blind method of [START_REF] Bofill | Underdetermined blind source separation using sparse representations[END_REF] and in our previous works [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF], the estimation of source signals is processed by a local inversion of the mixture signal. "Local" means that the process is considered for each TF bin [f, t], and at this level, only at most J = 2 sources are assumed to be contributing significantly to the mixture signal. Therefore, in the MDCT domain, the mixture

X[f, t] = A•S[f, t]
at each TF bin is assumed to be locally reduced to:

X[f, t] ≈ AI f t • SI f t [f, t], (1)
where I f t denotes the set of J = 2 most relevant sources at TF bin [f, t], i.e., the two source signals that locally "better explain" the mixture. AI f t represents the 2×2 mixing sub-matrix made with the Ai columns of A, i ∈ I f t (which is assumed to be invertible and well-conditioned). If Īft denotes the complementary set of non-active (or at least poorly active) sources at TF bin [f, t], the source signals are estimated by:

ŜI f t [f, t] = A -1 I f t • X[f, t] ŜĪ f t [f, t] = 0 . (2)
The side-information that is transmitted between ISS coder and decoder (in addition to the mix signal) consists of i) the coefficients of the mixing matrix A, and ii) the optimal combination of source indexes I f t for each TF bin. This contrasts with (semi-)blind separation methods where this information have to be estimated from the mix signal only, generally in two steps which can both be a challenging task and source of significant errors.

As for the mixing matrix, the number of coefficients to be transmitted is quite low in the present LISS configuration (only I fixed coefficients for each mixing configuration, if A is made of normalized column vectors). Therefore, the transmission cost of A is negligible compared to the transmission cost of I f t , and in the following we do not detail the encoding and transmission of A.

As for the source indexes, in the specific ISS framework, the optimal I f t is estimated using the source signals, the matrix A, and the mixture signals. This is done using an Oracle estimator, as introduced in [START_REF] Vincent | Oracle estimators for the benchmarking of source separation algorithms[END_REF] for providing upper bounds for the performances of source separation algorithms. Exploiting the reconstruction properties of the MDCT, the overall best separation in the time domain in the mean squared error (MSE) sense is obtained by finding the optimal combination of source signals at each TF bin separately [START_REF] Vincent | Oracle estimators for the benchmarking of source separation algorithms[END_REF]:

Ĩft = arg min I f t ∈P I i=1 Ŝi[f, t] -Si[f, t] 2 , (3)
where P represents the set of all possible combinations I f t and the I estimated source signals Ŝi(f, t) are provided by [START_REF]Independent Component Analysis[END_REF]. If I is limited to a reasonable number of sources, Ĩft can be found by exhaustive search, and coded with a very limited number of bits before being embedded into the mixture signal stream (see Section 2.4).

Separation from transmitted mixtures

The separation at the decoder consists of applying (2) using the MDCT coefficients X C [f, t] of the transmitted mix signal x C , instead of the coefficients of the original mix signal X[f, t]:

   ŜĨ f t [f, t] = A -1 Ĩft • X C [f, t] ŜĨ f t [f, t] = 0 . (4)
In the current system (Fig. 1), the mix signal at the output of the coder is either PCM or AAC bitstreams. In our previous studies, 16-bit PCM conversion has been shown to have negligible effects on separation performances. In contrast, audio compression is expected to perturb the MDCT coefficients of the mix signal significantly, depending on bitrate, and is thus likely to impair the separation performance. Also, the sparsity assumption may not hold perfectly, in which case, non-predominant but active sources will interfere as noises in the local inversion process. In Section 3, we report separation results for speech signals mixed with the LISS configuration and compressed with different settings.

Side-information coding and embedding

Instead of embedding the side-information (A and Ĩft ) into the mix signal waveform through a high-capacity watermarking technique [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF][START_REF] Pinel | A high-capacity watermarking technique for audio signals based on MDCT-domain quantization[END_REF], in the current system we transmit the sideinformation using metadata segments of the compressed binary stream. It can also be transmitted using a dedicated channel. This is similar to the spirit of SAOC [START_REF] Engdegård | Spatial audio object coding (SAOC): The upcoming MPEG standard on parametric object based audio coding[END_REF] (see Section 3). Since A is fixed and only needs to be transmitted once for the entire signal, we consider only Ĩft here. Suppose that I is fixed for all frames and frequencies for a given mix signal, with basic entropy-coding, the side-information occupies approximately 2fmax/Fs log 2 (I f t ) bits per sample, where fmax is the maximal frequency processed, and Fs is the sampling frequency of the mix signal. For example, for a 5-source mixture, we have 10 combinations of 2 active sources out of 5, leading to log 2 10 = 3.3 bit per sample, or 53 kbps for the sideinformation, given fmax = 8 kHz and Fs = 16 kHz.

This basic coding scheme can be refined by exploiting the fact that musical/speech sources also generally have some temporal sparsity. Before mixing, each source signal is then labeled into non-silent/silent sections, and for each MDCT frame, a Ibit code c1 is transmitted to provide the combination of nonsilent sources. For each frequency bin f , the estimation of Ĩft by ( 3) is then carried out only among the non-silent sources, leading to a lower bitrate. For example, if only 4 sources out of 5 are non-silent on a given frame, Ĩft only concerns 2 sources out of 4, then maximally log 2 (6) = 2.6 bits per sample. In addition, only c1 has to be transmitted if the number of nonsilent sources is lower or equal to 2. Such refinement has been shown to save up to 40% of bitrate depending on the number of sources and their degree of temporal sparsity. Finally, it appears that the distribution of Ĩft in the TF plane is highly structured, and many lossless coding strategies can be applied to the sideinformation.

In the present paper, we do not detail the format of the metadata segment in the compressed bitstream, and the corresponding embedding strategy. We, however, provide in Section 3 the typical ISS side-information bitrates obtained for our test signals (and we compare them with the SAOC side-information bitrates for the same test signals).

Experiments on speech mixtures 3.1. Experimental settings

The ISS system has been applied to mixtures of I = 3, 5, and 7 simultaneous talks, downmixed to stereo (2-channel) signals.

All test speech signals are 16-kHz signals taken from the TIMIT database [START_REF] Garofolo | Timit acoustic-phonetic continuous speech corpus[END_REF], concatenated and level-adjusted to form continuous (non-stop) 5s-clips with the same volume. In the mixed stereo signal, each source signal is panned to a different azimuth, governed by a constant mixing matrix A, to simulate speakers from different angles. To ensure good conditioning of each sub-matrix of A and spatial (perceived) separation of the speakers, we have

A = cos θ1 cos θ2 • • • cos θI sin θ1 sin θ2 • • • sin θI , (5)
where θ k = (2k -1)π/(4I). In such mix of dense simultaneous talks of the same volumes, we observed that in case of I = 3, normal people could barely understand who was saying what; in case of I = 7, the mix is very messy and close to cafeteria noises. So our experiments presented hard cases for the separation task, even for human beings. We evaluated the ISS system in terms of Signal to Interference Ratios (SIRs) and SDRs as defined in [START_REF] Vincent | Performance measurement in blind audio source separation[END_REF], with and without encoding the mixture signals by AAC. When encoding mixture stereo signals, we used the AAC codecs from Nero 1 . and bitrates were set to 16, 32, and 64 kbps employing the AAC Low Complexity (LC) profile [START_REF]Information technology-Generic coding of moving pictures and associated audio information-Part 7: Advanced Audio Coding (AAC)[END_REF] (this was to avoid spectral distortion resulting from parametric coding enabled at lower bitrates).

For comparison, we also evaluated the SIRs and SDRs of SAOC using the same source and mix signals. As mentioned in the introduction, SAOC is a system dedicated to restitute "audio objects" from downmix signals and side-information. SAOC is built on the former MPEG Surround or Spatial Audio coding (SAC) system [START_REF] Herre | MPEG surround-the ISO/MPEG standard for efficient and compatible multichannel audio coding[END_REF], which was initially designed for the respatialization of multichannel audio from downmix signals using interchannel spatial cues (transmitted as low-bitrate side-information). SAOC extends the respatialization concept of SAC to audio objects (e.g., musical instruments or individual speech signals), thus leading de facto to source separation through respatialization. However, because it was initially designed for spatialization, the SAOC decoder may not ensure sufficient separation quality for any kinds of audio scenes. In contrast, even if the overall spirit is close, our ISS system is designed specifically for source separation (of LISS mixtures) and uses specific parameters. Note that for those comparative experiments, we used the SAOC reference system provided on the MPEG website 2 , which may not be an optimized version. Note also that downmixing by matrices defined in ( 5) fits well the model of SAOC -sources are spatialized significantly and stably -a sweet point for SAOC.

Separation results

As an example, the stereo mix signals and speech signals separated by the ISS system are shown in Fig. 2 for I = 5 and bitrate = 32 kbps. They come from male and female American speakers with different accents.

The SIRs and SDRs obtained on our test signals are provided in Fig. 3 and Fig. 4 respectively. In Fig. 3, we find that mean (across sources) SIRs obtained with the ISS system are almost all higher than or close to 30 dB, even for the extreme case of 7 simultaneous speakers. This indicates that the sparsity assumption is well hold for multiple simultaneous speech signals. The SIRs are remarkably robust to compression: differences from PCM to AAC-64 kbps and then to AAC-32 kbps are generally small (however, a large decrease to 16 kbps, of about 10 dB). This implies that mix signal compression at relatively low bitrate, say 32 kbps, is sufficient to ensure good separation of sources. The ISS system outperforms the SAOC system by 10-25 dB in SIRs. This is also clearly verified by informal listening tests: cross-talks are much more apparent in SAOC. It should also be noted that the SIRs of SAOC are almost the same for PCM and AAC at different bitrates, but depend on the number of speakers, suggesting that SIRs in SAOC are more limited by the SAOC spatialization model than by AAC encoding distortion.

Compared to the SIRs, SDRs of both the ISS system and SAOC in Fig. 4 are 5-30 dB lower. Therefore, processing artifacts dominate the overall distortions. These artifacts come from both AAC encoding and source separation. And generally, the more numerous the speakers, the more dominant the artifacts from source separation, the less important the artifacts from compression. This is shown as SDRs of the ISS system drop about 7 dB from PCM to AAC-64 kbps for I = 3, but drop only about 2 and less than 1 dB for I = 5 and 7, respectively. For I = 7, AAC at 64 kbps is a good trade-off between source separation quality and mix bitrate, compared to PCM. Informal listening tests reveal that except for the case of I = 7 and bitrate = 16 kbps, speech signals separated by the ISS system are well intelligible (of course, they are of better quality when going toward smaller I and higher bitrates). For SAOC, intelligibility is ensured for all separated sources only when the mixtures are compressed at bitrate no less than 32 kbps (or not compressed) and the number of speakers is no more than 5. Demo sequences can be downloaded from http://www.gipsa-lab.inpg.fr/∼laurent.girin /demo/ISS-SAOC-speech.zip.

With basic entropy coding, the side-information bitrates of the ISS system are 15.6, 40.0, 56.9 kbps for I = 3, 5, and 7, respectively. These bitrates can be significantly reduced by taking the advantages of the structure of the indexes and redundancy among them. The side-information bitrates of the SAOC system are about 3/4 lower, thanks to its advanced entropy coding scheme.

Conclusions

We have extended the ISS system proposed in [START_REF] Parvaix | Informed source separation of underdetermined instantaneous stereo mixtures using source index embedding[END_REF] for the informed separation of uncompressed music signals to the case of uncompressed or compressed speech signals. Experiments with the extended system applied to continuous simultaneous talks of up to 7 speakers has demonstrated that local sparsity, the core assumption of the system, withstands dense mixtures of many speech signals that are impossible to distinguish for normal listeners. At different compression levels and with different numbers of speakers, the extended system is capable to separate stereo mix signals into intelligible individual speech signals, and maintains SIRs higher than 30 dB (except for one condition) while SDRs are dominated by encoding and separation artifacts. Therefore, reducing spectral artifacts due to the ISS processing is key to improving its performance. Also we will refine the entropy coding scheme of the indexes to reduce side-information bitrate and integrate the ISS system more closely into AAC to reduce computational complexity.

Figure 1 :

 1 Figure 1: Diagram of the ISS system.

1Figure 2 :

 2 Figure 2: Waveforms of the stereo mix (a), and the I = 5 separated signals (b)-(f) (ISS system, AAC 32 kbps).

2

  http://www.itscj.ipsj.or.jp/sc29/29w42911.htm

Figure 3 :

 3 Figure 3: Mean (across sources) SIRs of the ISS system and SAOC, with and without encoding mixtures by AAC. The horizontal bars are for minimum and maximum SIRs.

Figure 4 :

 4 Figure 4: Mean (across sources) SDRs of the ISS system and SAOC, with and without encoding mixtures by AAC. The horizontal bars are for minimum and maximum SDRs.

  t]

			MDCT
			MDCT
							Coding
			MDCT		PCM 16-bit	PCM mode		stream .wav stream side-info
	Mixing (LISS)						Embed-ding	AAC binary stream
	Mixing		MDCT			AAC mode
	matrix A
	side-.wav info		Decoding	I[f,t]			] ˆ1 t , [ f S	IMDCT	[ ˆ1 n s	]
	AAC binary stream	Demux AAC mode	AAC deco-ding	] Mixing matrix A [n x C L ] [n x C R	MDCT MDCT	2 × 2 inver -sion ˆt f , [ S I	]	IMDCT IMDCT	] ] ˆn [ ˆ2 n s [ s I

Acknowledgements

This project is supported by the French National Research Agency (ANR) -Grant CONTINT 2009 CORD 006.