Laurent Girin
email: laurent.girin@gipsa-lab.grenoble-inp.fr

Sharon Gannot

Xiaofei Li

Audio Source Separation into the Wild *

Keywords: Multichannel audio source separation, Beamforming for Audio signals, smart devices, hearing aids

This review chapter is dedicated to multichannel audio source separation in real-life environment. We explore some of the major achievements in the field and discuss some of the remaining challenges. We will explore several important practical scenarios, e.g. moving sources and/or microphones, varying number of sources and sensors, high reverberation levels, spatially diffuse sources, and synchronization problems. Several applications such as smart assistants, cellular phones, hearing aids and robots, will be discussed. Our perspectives on the future of the field will be given as concluding remarks of this chapter.

Introduction

Source separation is a topic of signal processing that has been of major interest for decades. It consists of processing an observed mixture of signals so that to extract the elementary signals composing this mixture. In the context of audio processing, it refers to the extraction of signals simultaneously emitted by several sound sources, from the audio recordings of the resulting mixture signal. It has major applications, going from speech enhancement as a front-end for telecommunication systems and automatic speech recognition, to demixing and remixing of music. Despite a recent progress using deep learning techniques, single-channel multi-source recordings are still regarded particularly difficult to separate. In this chapter we deal with audio source separation in the wild, and we address multichannel recordings obtained using multiple microphones in a natural environment, as opposed to mixtures created by mixing software which generally do not match the acoustics of real environments, e.g. music production in studio. Typically, we discuss such problems as having to separate the speech signals emitted simultaneously by different persons sharing the same acoustic enclosure, considering also ambient noise and other interfering sources such as a domestic apparatuses.

Even when using multichannel recordings, source separation in general is a difficult problem that belongs to the general class of inverse problem. As such, it is often ill-posed, in particular when the number of sensors used to capture the mixture signals is lower than the number of emitting sources. Consequently, in the signal processing community in general, and in the audio processing community in particular, the source separation problem has often been addressed within quite controlled configurations, i.e. "laboratory" studies, that are carefully designed to allow a proper evaluation protocol and an in-depth inspection of the behavior of the proposed techniques. As we will detail in this chapter, this often comes in contrast to robust, into-the-wild configurations, where the source separation algorithms are confronted with the complexity of real-world data, and thus "do not work so well". In this chapter, we describe such "limitations" of multichannel audio source separation (MASS) and make a review of the studies that have been proposed to overcome those limitations, trying to make MASS techniques progressively go from laboratories into the wild. 1Research in speech enhancement and speaker separation has followed two convergent paths, starting with microphone array processing (also referred to as beamforming) and blind source separation, respectively. These communities are now strongly interrelated and routinely borrow ideas from each other. Hence, in this chapter we discuss the two paradigms interchangeably. We will explore several important practical scenarios, e.g. moving sources and/or microphones, varying number of sources and sensors, high reverberation levels, spatially diffuse sources, and synchronization problems. Several applications such as smart assistants, cellular phones, hearing devices and robots, which have recently gained a growing research and industrial interest, will be discussed.

This chapter is organized as follows. In Section 2, we briefly present the fundamentals of multichannel audio source separation. In Section 3 we list the current major limitations of MASS that prevent a large deployment of MASS technique into the wild, and we present approaches that have been proposed in the literature to overcome these limitations.

Multichannel audio source separation

In this section, we briefly present the fundamentals of multichannel audio source separation. This presentation is limited to the basic material that is necessary to understand the following discussion on MASS into the wild. Indeed, the goal of this chapter is not to extensively present the theoretical foundations and principles of source separation and beamforming, even limited to the audio context: Many publications have addressed this issue, including books [START_REF]Independent Component Analysis[END_REF][START_REF] Divenyi | Speech separation by Humans and machines[END_REF][START_REF]Blind speech separation[END_REF][START_REF] Benesty | Microphone array signal processing[END_REF][START_REF] Comon | Handbook of Blind Source Separation -Independent Component Analysis and Applications[END_REF] and overview papers [START_REF] Cardoso | Blind signal separation: Statistical principles[END_REF][START_REF] O'grady | Survey of sparse and nonsparse methods in source separation[END_REF][START_REF] Gannot | A consolidated perspective on multimicrophone speech enhancement and source separation[END_REF].

Hundreds of multichannel audio signal enhancement techniques have been proposed in the literature over the last forty years along two historical research paths. Microphone array processing emerged from the theory of sensor array processing for telecommunications and it focused mostly on the localization and enhancement of speech in noisy or reverberant environments [START_REF] Gay | Acoustic signal processing for telecommunication[END_REF][START_REF]Microphone Arrays: Signal Processing Techniques and Applications[END_REF][START_REF] Benesty | Speech Enhancement[END_REF][START_REF] Loizou | Speech Enhancement: Theory and Practice[END_REF][START_REF]Speech processing in modern communication: Challenges and perspectives[END_REF], while MASS was later popularized by the machine learning community and it addressed "cocktail party" scenarios involving several sound sources mixed together [START_REF] O'grady | Survey of sparse and nonsparse methods in source separation[END_REF][START_REF]Blind speech separation[END_REF][START_REF] Pedersen | Convolutive blind source separation methods[END_REF][START_REF] Comon | Handbook of Blind Source Separation -Independent Component Analysis and Applications[END_REF][START_REF] Vincent | Probabilistic modeling paradigms for audio source separation[END_REF][START_REF] Vincent | From blind to guided audio source separation: How models and side information can improve the separation of sound[END_REF]. These two research tracks have converged in the last decade and they are hardly distinguishable today. Source separation techniques are not necessarily blind anymore and most of them exploit the same theoretical tools, impulse response models and spatial filtering principles as speech enhancement techniques.

The formalization of the MASS problem begins with the formalization of the mixture signal. The most general expression for a linear mixture of J source signals recorded by I microphones is:

x(t) = J j=1 y j (t) + b(t) ∈ R I , (1)
where y j (t) ∈ R I is the multichannel image of the j-th source signal s j (t) [START_REF] Sturmel | Linear mixing models for active listening of music productions in realistic studio conditions[END_REF], taking into account the effect of acoustic propagation from the position of the emitted source to the microphones (each entry y ij (t) of y j (t) is the image of s j (t) at microphone i). b(t) is a sensor noise term. In most studies on MASS, the effect of acoustic propagation from source j to microphone i is modelled as a linear time-invariant filter of impulse response a ij (t), and we have:

x(t) = J j=1 La-1 τ =0 a j (τ )s j (t -τ ) + b(t). (2)
The vector a j (τ ) contains all responses a ij (t) for i ∈ [1, I], which are assumed to have the same length L a for convenience. Depending on the application, the goal of MASS is to estimate either the source images y j (t) or the (monochannel) source signals s j (t) from the observation of x(t).

State-of-the-art MASS methods generally start with a time-frequency (TF) decomposition of the temporal signals, usually by applying the short-time Fourier transform (STFT) [START_REF] Crochiere | Multi-Rate Signal Processing[END_REF]. This is for two main reasons. First, model-based approaches can take advantage of the very particular sparse structure of audio signals in the TF plane [START_REF] Plumbley | Sparse representations in audio and music: From coding to source separation[END_REF]: A small proportion of source TF coefficients have a significant energy. Source signals are thus generally much less overlapping in the TF domain than in the time-domain, naturally facilitating the separation. Second, it is common to consider that at each frequency, the time-domain convolutive mixing process (2) is transformed by the STFT into a simple product between the source STFT coefficients and the discrete Fourier transform (DFT) coefficients of the mixing filter, see e.g. [START_REF] Parra | Convolutive blind separation of non-stationary sources[END_REF][START_REF] Yilmaz | Blind separation of speech mixtures via timefrequency masking[END_REF][START_REF] Winter | MAP-based underdetermined blind source separation of convolutive mixtures by hierarchical clustering and l1-norm minimization[END_REF][START_REF] Araki | Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors[END_REF][START_REF] Mandel | Model-based expectationmaximization source separation and localization[END_REF][START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF][START_REF] Ozerov | A general flexible framework for the handling of prior information in audio source separation[END_REF] and many other studies:

x(f, n) ≈ J j=1 a j (f )s j (f, n) + b(f, n) = A(f )s(f, n) + b(f, n), (3)
where x(f, n), s j (f, n) and b(f, n) are the STFT of x(τ ), b(τ ) and s j (τ ), respectively, and a j (f ) gathers the DFT of the entries of a j (τ ), known as the acoustic transfer functions (ATFs). The ATF vectors are concatenated in the matrix A(f ) and the source signals s j (f, n) are stacked in the vector s(f, n). In many practical scenarios, A(f ) is substituted by the respective relative transfer function (RTF) matrix, for which each column is normalized by its first entry. Under this normalization the first row of A(f ) is all '1's and the source signals s j (f, n) are substituted by their image on the first microphone. 2As further discussed in Section 3.3, ( 3) is an approximation that is valid if the length of the mixing filters impulse responses is shorter than the length of the STFT window analysis. In the literature and in the following this approximation is referred to as the multiplicative transfer function (MTF) approximation [START_REF] Avargel | On multiplicative transfer function approximation in the short-time Fourier transform domain[END_REF] or the narrowband approximation [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF].

MASS methods can then be classified into four (non-exclusive) categories [START_REF] Vincent | Probabilistic modeling paradigms for audio source separation[END_REF][START_REF] Gannot | A consolidated perspective on multimicrophone speech enhancement and source separation[END_REF]. Firstly, separation methods based on independent component analysis (ICA) consist in estimating the demixing filters that maximize the independency of separated sources [START_REF] Comon | Handbook of Blind Source Separation -Independent Component Analysis and Applications[END_REF][START_REF]Independent Component Analysis[END_REF]. TF-domain ICA methods have been largely investigated [START_REF] Smaragdis | Blind separation of convolved mixtures in the frequency domain[END_REF][START_REF] Parra | Convolutive blind separation of non-stationary sources[END_REF][START_REF] Nesta | Convolutive BSS of short mixtures by ICA recursively regularized across frequencies[END_REF]. Unfortunately, ICA-based methods are subject to the well-known scale ambiguity and source permutation problems across frequency bins [START_REF] Araki | The fundamental limitation of frequency domain blind source separation for convolutive mixtures of speech[END_REF], which must generally be solved as a post-processing step [START_REF] Ikram | A beamformer approach to permutation alignment for multichannel frequency-domain blind source separation[END_REF][START_REF] Sawada | A robust and precise method for solving the permutation problem of frequency-domain blind source separation[END_REF][START_REF] Sawada | Grouping separated frequency components by estimating propagation model parameters in frequency-domain blind source separation[END_REF]. In addition, these methods cannot be directly applied to underdetermined mixtures.

Secondly, methods based on sparse component analysis (SCA) and binary masking rely on the assumption that only one source is active at each TF point [START_REF] Roman | Speech segregation based on sound localization[END_REF][START_REF] Winter | MAP-based underdetermined blind source separation of convolutive mixtures by hierarchical clustering and l1-norm minimization[END_REF][START_REF] Araki | Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors[END_REF][START_REF] Mandel | Model-based expectationmaximization source separation and localization[END_REF] (most methods consider only one active source at each TF bin though in principle the SCA and ICA approaches can be combined by considering up to I active sources in each TF bin). These methods often rely on some sort of source clustering in the TF domain, generally based on spatial information extracted from prior mixing filter identification. Therefore, for this kind of methods, the source separation problem is often linked to the source localization problem (where are the emitting sources?).

Thirdly, more recent methods are based on probabilistic generative models in the STFT domain and associated parameter estimation and source inference algorithms [START_REF] Vincent | Probabilistic modeling paradigms for audio source separation[END_REF]. The latter are mostly based on the well-known expectationmaximization (EM) methodology [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF] and the like (i.e. iterative alternating optimization techniques). One popular approach is to model the source STFT coefficients with a complex-valued local Gaussian model (LGM) [START_REF] Févotte | Maximum likelihood approach for blind audio source separation using time-frequency Gaussian source models[END_REF][START_REF] Duong | Under-determined reverberant audio source separation using a full-rank spatial covariance model[END_REF][START_REF] Yoshioka | Blind separation and dereverberation of speech mixtures by joint optimization[END_REF][START_REF] Liutkus | Gaussian processes for underdetermined source separation[END_REF], often combined with a nonnegative matrix factorization (NMF) model [START_REF] Lee | Learning the parts of objects by non-negative matrix factorization[END_REF] applied to the source PSD matrix [START_REF] Févotte | Nonnegative matrix factorization with the Itakura-Saito divergence. With application to music analysis[END_REF][START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF][START_REF] Arberet | Nonnegative matrix factorization and spatial covariance model for under-determined reverberant audio source separation[END_REF][START_REF] Ozerov | A general flexible framework for the handling of prior information in audio source separation[END_REF], which is reminiscent of pioneering works such as [START_REF] Benaroya | Non negative sparse representation for Wiener based source separation with a single sensor[END_REF]. This allows one to drastically reduce the number of model parameters and (to some extent) to alleviate the source permutation problem. The sound sources are generally separated using Wiener filters constructed from the learned parameters. Such approach has been extended to super-Gaussian (or heavy-tailed) distributions to model the audio signal sparsity in the TF domain [START_REF] Mitianoudis | Audio source separation of convolutive mixtures[END_REF], as well as to a fully Bayesian framework by considering prior distributions for the (source and/or mixing process) model parameters [START_REF] Kounades-Bastian | An inverse-Gamma source variance prior with factorized parameterization for audio source separation[END_REF]. Note that, as for SCA/binary masking methods, the estimation of mixing parameters and the source separation itself are two different steps, often processed alternately within the EM principled methodology.

Methods belonging to the fourth category can be broadly classified as beamforming methods, which is roughly equivalent to linear spatial filtering. A beamformer is a vector w(f ) = [w 1 (f ), . . . , w I (f )]

T comprising one complex-valued weight per microphone, that is applied to x(n, f ). The output w H (f ) x(n, f ) can be transformed back into the time-domain by an inverse STFT. Beamformers originally referred to spatial filters based on the direction of arrival (DOA) of the source signal and were only later generalized to any linear spatial filters. DOA-based beamformers are still widely-used, especially when simplicity of the implementation and its robustness are of crucial importance. However, the performance of these beamformers is expected to degrade in comparison with modern beamformers that take the entire acoustic path into account [START_REF] Gannot | Signal enhancement using beamforming and nonstationarity with applications to speech[END_REF]. The beamformer weights are set according to a specific optimization criterion. Many beamforming criteria can be found in the general literature [START_REF] Van Trees | Detection, Estimation, and Modulation Theory, volume IV, Optimum Array Processing[END_REF]. In the speech processing community, the minimum variance distortionless response (MVDR) beamformer [START_REF] Gannot | Signal enhancement using beamforming and nonstationarity with applications to speech[END_REF], the maximum signal-to-noise ratio (MSNR) beamformer [START_REF] Warsitz | Blind acoustic beamforming based on generalized eigenvalue decomposition[END_REF], the multichannel Wiener filter (MWF) [START_REF] Doclo | GSVD-based optimal filtering for single and multimicrophone speech enhancement[END_REF], specifically its speech distortion weighted variant (SDW-MWF) [START_REF] Doclo | Speech distortion weighted multichannel Wiener filtering techniques for noise reduction[END_REF], and the linearly constrained minimum variance (LCMV) beamformer [START_REF] Markovich | Multichannel eigenspace beamforming in a reverberant noisy environment with multiple interfering speech signals[END_REF], are widely used.

One may state that, so far, the beamforming approach led to more effective industrial real-world applications than the "generic" MASS approach. This lies in the difference between a) enhancing a spatially fixed dominant target speech signal from background noise (possibly composed of several sources) and b) clearly separating several source signals, all considered as signals of interest, that are mixed with a similar power (the cocktail party problem). The first problem is generally simpler to solve than the second one. In other words, the second one can be seen as an extension of the first one. Anyway, as we will see now, problems of into-the-wild processing remain challenging in each case.

3 Making MASS go from labs into the wild

Moving sources and sensors

In a realistic into-the-wild scenario, sound sources are often moving. Sometimes they are moving slightly, e.g. the small movements of the head of a sat speaker. Sometimes they are moving a lot, e.g. a person speaking while walking in a room. Sensors can also move, e.g. microphones embedded within a mobile robot. In the same vein, the acoustic environment can also change over time, e.g. we close a window, an object is placed in between a source and the microphones, or the separation system has to operate in another room. All those changes imply changes in the acoustic propagation of sources to microphones, i.e. changes in the mixing filters. Yet the vast majority of MASS methods described in the signal processing literature deals with the assumption of fixed sources, fixed sensors and fixed environment, i.e. technically speaking the mixing filters are considered as time-invariant (at least over the duration of the processed recordings), as expressed in [START_REF] Miro | Speaker diarization: A review of recent research[END_REF]. Into-the-wild MASS methods should consider the more realistic case of time-varying mixtures corresponding to source-to-microphone channels that can change over time, which would account for possible source or microphone motions and environment changes. For example, in many Human-robot interaction scenarios, there is a strong need to consider mixed speech signals emitted by moving speakers and perturbed by reverberation that can change over time and by non-stationary ambient noise.

Studies dealing with moving sources or moving sensors or changing environment actually exist, but they are quite sparse compared to the large number of studies with time-invariant filters. Early attempts addressing the separation of time-varying mixtures basically consisted in block-wise adaptations of time-invariant methods: An STFT frame sequence is split into blocks, and a time-invariant MASS algorithm is applied to each block. Hence, block-wise adaptations assume time-invariant filters within blocks. The separation parameters are updated from one block to the next and the separation result over a block can be used to initialize the separation of the next block. Frame-wise algorithms can be considered as particular cases of block-wise algorithms, with single-frame blocks, and hybrid methods may combine block-wise and framewise processing. Notice that, depending on the implementation, some of these methods may run online.

Interestingly, most of the block-wise approaches use ICA, either in the temporal domain [START_REF] Koutras | Blind speech separation of moving speakers in real reverberant environments[END_REF][START_REF] Hild | Blind source separation of time-varying, instantaneous mixtures using an on-line algorithm[END_REF][START_REF] Aichner | On-line time-domain blind source separation of nonstationary convolved signals[END_REF][START_REF] Prieto | Blind source separation for time-variant mixing systems using piecewise linear approximations[END_REF] or in the Fourier domain [START_REF] Mukai | Robust real-time blind source separation for moving speakers in a room[END_REF][START_REF] Nakadai | Sound source separation of moving speakers for robot audition[END_REF]. In addition to being limited to overdetermined mixtures, block-wise ICA methods need to account for the source permutation problem, not only across frequency bins, as usual, but across successive blocks as well. Examples of block-wise adaptation of binary-masking or LGM-based methods are more scarce. As for binary masking, a block-wise adaptation of [START_REF] Araki | Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors[END_REF] was proposed in [START_REF] Loesch | Online blind source separation based on timefrequency sparseness[END_REF]. This method performs source separation by clustering the observation vectors in the source image space. As for LGM, [START_REF] Simon | A general framework for online audio source separation[END_REF] describes an online block-and frame-wise adaptation of the general LGM framework proposed in [START_REF] Ozerov | A general flexible framework for the handling of prior information in audio source separation[END_REF]. One important problem, common to all block-wise approaches, is the difficulty to choose the block size. Indeed, the block size must assume a good trade-off between local channel stationarity (short blocks) and sufficient data to infer relevant statistics (long blocks). The latter constraint can drastically limit the dynamics of either the sources or the sensors [START_REF] Loesch | Online blind source separation based on timefrequency sparseness[END_REF]. Other parameters such as the step-size of the iterative update equations may also be difficult to set [START_REF] Simon | A general framework for online audio source separation[END_REF]. In general, systematic convergence towards a good separation solution using a limited amount of signal statistics remains an open issue.

A more principled approach consists in modeling the mixing filter as a timevarying process and considering the MASS in the angle of an adaptive process, in the spirit of early works on adaptive filtering [START_REF] Widrow | Adaptive noise cancelling: Principles and applications[END_REF]. For example, an early iterative and sequential approach for speech enhancement in reverberant environment was proposed in [START_REF] Weinstein | Iterative and sequential algorithms for multisensor signal enhancement[END_REF]. This method used the EM framework to jointly estimate the desired speech signal and the required (deterministic) parameters, namely the speech auto-regressive coefficients, and the speech and noise mixing filters taps. Only the case of a 2 × 2 mixture was addressed. A subspace tracking recursive LCMV beamforming method for extracting multiple moving sources was proposed in [START_REF] Markovich-Golan | Subspace tracking of multiple sources and its application to speakers extraction[END_REF]. This method is applicable only to over-determined mixture.

As for under-determined time-varying convolutive mixtures, a method using binary masking within a probabilistic LGM framework was proposed in [START_REF] Higuchi | Underdetermined blind separation and tracking of moving sources based on DOA-HMM[END_REF]. The mixing filters were considered as latent variables following a Gaussian distribution with mean vector depending on the DOA of the corresponding source. The DOA was modeled as a discrete latent variable taking values from a finite set of angles and following a discrete hidden Markov model (HMM). A variational expectation-maximization (VEM) algorithm was derived to perform the inference, including forward-backward equations to estimate the DOA sequence.

In [START_REF] Kounades-Bastian | A variational EM algorithm for the separation of moving sound sources[END_REF][START_REF] Kounades-Bastian | A variational EM algorithm for the separation of time-varying convolutive audio mixtures[END_REF], the transfer function of the mixing filters were considered as continuous latent variables ruled by a first-order linear dynamical system (LDS) with Gaussian noise [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF], in the spirit of [START_REF] Gannot | On the application of the unscented Kalman filter to speech processing[END_REF]. This model was used in combination with a source LGM-with-NMF model, still to process underdetermined time-varying convolutive mixtures. This approach may be seen as a generalization of [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF] to moving sources/microphones. As in [START_REF] Higuchi | Underdetermined blind separation and tracking of moving sources based on DOA-HMM[END_REF], a VEM algorithm was developed for the joint estimation of the model parameters and inference of the latent variables. Here, a Kalman smoother was used for the inference of the time-varying mixing filters, which were combined with estimated source PSDs to build separating Wiener filters. This model can be more effective than the discrete DOA-dependent HMM model of [START_REF] Higuchi | Underdetermined blind separation and tracking of moving sources based on DOA-HMM[END_REF] in reverberant conditions, since the relationship between the transfer function and the source DOA can be quite complex, and Wiener filters are more general than binary masks.

A VEM approach for beamforming (hence, over-determined scenario) that is specifically designed for dynamic scenarios can be found in [START_REF] Malik | A Bayesian framework for blind adaptive beamforming[END_REF][START_REF] Schmid | Variational bayesian inference for multichannel dereverberation and noise reduction[END_REF][START_REF] Laufer | A Bayesian hierarchical model for speech enhancement[END_REF]. In these methods, the speech signal is modelled as a an LGM in the STFT domain and the RTFs as a first-order Markov model. The posterior distribution of the speech signal and the channel is recursively estimated in the E-step.

In comparison to the block-wise adaptation methodology described in [START_REF] Simon | A general framework for online audio source separation[END_REF],

explicit time-varying mixture models have the potential to exploit the information available within the whole sequence of input mixture frames. They were generally proposed in batch mode but they can be adapted to online processing, e.g. by replacing a Kalman smoother with a Kalman filter [START_REF] Weinstein | Iterative and sequential algorithms for multisensor signal enhancement[END_REF]. DOA estimation plays an important role in the design of beamforming methods. DOA tracking of multiple speakers based on a discrete set of angles and HMM is given in [START_REF] Roman | Binaural tracking of multiple moving sources[END_REF]. DOA estimation procedures for single source (or alternating sources) scenarios, which are based on variants of the recursive least squares (RLS) methodology, are presented in [START_REF] Dvorkind | Time difference of arrival estimation of speech source in a noisy and reverberant environment[END_REF]. Recursive versions of the EM procedure are utilized for multiple speakers tracking in [START_REF] Schwartz | Speaker tracking using recursive EM algorithms[END_REF]. A simple, yet effective method for localizing multiple sources in reverberant environments is the steered-response-power with phase transform (SRP-PHAT) [START_REF] Dibiase | Robust localization in reverberant rooms[END_REF]. The MUltiple Signal Classification (MUSIC) algorithm [START_REF] Schmidt | Multiple emitter location and signal parameter estimation[END_REF] or more specifically, the root-MUSIC variant, allow for fast adaptation of LCMV beamformers by exploiting instantaneous DOA estimates [START_REF] Thiergart | An informed LCMV filter based on multiple instantaneous direction-of-arrival estimates[END_REF][START_REF] Thiergart | An informed parametric spatial filter based on instantaneous direction-of-arrival estimates[END_REF].

Finally, we can mention that robots, as a moving platforms, open new opportunities and challenges for the sound source localization and separation tasks. An open source robot audition system, named 'HARK', is described in [START_REF] Nakadai | Design and implementation of robot audition system 'HARK'-Open source software for listening to three simultaneous speakers[END_REF]. The localization module is based on successive application of the MUSIC algorithm [START_REF] Schmidt | Multiple emitter location and signal parameter estimation[END_REF], while the separation stage uses a geometry-assisted MASS method [START_REF] Parra | Geometric source separation: Merging convolutive source separation with geometric beamforming[END_REF]. Bayesian methods for tracking multiple sources are also gaining interest in the literature, e.g. using particle filters [START_REF] Valin | Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering[END_REF][START_REF] Evers | Source tracking using moving microphone arrays for robot audition[END_REF] and probability hypothesis density (PHD) filters [START_REF] Evers | Localization of moving microphone arrays from moving sound sources for robot audition[END_REF]. Two recent European projects, the "Ears"3 project and the "Two!Ears"4 project, explored new algorithms for enhancing the auditory capabilities of humanoid robots [START_REF] Löllmann | Microphone array signal processing for robot audition[END_REF] and link them with decision and action [START_REF] Bustamante | An information based feedback control for audio-motor binaural localization[END_REF].

Varying number of (active) sources

In a realistic into-the-wild scenario, sound sources are often intermittent, i.e. they do not emit sounds all the time. As an example of major importance we can mention a natural conversational regime between several speakers that includes speech turns. Depending on the context and content of the conversation, the speech signals can have very low to very strong time overlap. The sound sources may not even be present in the scene all the time, e.g. a person that goes in and out of a room, occasionally speaking or turning on and off a sounding device. Yet, the vast majority of MASS methods described in the signal processing literature deals with the assumption of a fixed number of sources over time. In addition, this fixed number of sound sources is often assumed to be known and all sources are assumed to be continuously active, i.e. they emit during all the time of the processed recording sequence. The situation of the literature with this respect is similar to the previous section: A few studies with the number of active sources varying in time do exist but they are largely outnumbered by the studies with fixed number of constantly active sources.

One straightforward manner to address this problem is to proceed to the estimation of the number of sources present in the scene and/or the number of active sources as a pre-processing step before going into the separation problem. A method based on speech sparsity in the STFT domain is presented in [START_REF] Arberet | A robust method to count and locate audio sources in a multichannel underdetermined mixture[END_REF]. A variational EM approach for complex Watson mixture models is presented in [START_REF] Drude | Source counting in speech mixtures using a variational EM approach for complex Watson mixture models[END_REF]. In the beamforming context, identifying the number of speakers, as well as the number of available sensors, necessitates an update of the weights of the beamformer. An efficient implementation, based on low-rank update of correlation matrices, is presented in [START_REF] Markovich-Golan | Low-complexity addition or removal of sensors/constraints in LCMV beamformers[END_REF].

The detection of the number of active source and associating an "identity" (i.e. a label) to each detected source is related to the so-called diarization problem. Indeed, in speech processing, speaker diarization refers to the task of detecting who speaks and when in an audio stream [START_REF] Miro | Speaker diarization: A review of recent research[END_REF][START_REF] Vijayasenan | Multistream speaker diarization of meetings recordings beyond MFCC and TDOA features[END_REF]. In many (dialog) applications, the speakers are assumed to take distinct speech turns, i.e. they speak one after the other, and speaker diarization thus amounts to signal segmentation and speaker recognition. In a source separation context, the automatic detection of the number and "identity" of simultaneously active sources can thus be considered as an additional multisource diarization task to be considered jointly with the separation task, or within the separation task. Indeed, both processes are complementary: Knowing the source diarization is assumed to ease the separation process, for example by enabling to adapt the separation system to the actual number of active sources and to the speaker characteristics; in turn diarization is easier using separated source signals than using mixed source signals.

Processing of speech intermittency for MASS appears in [START_REF] Comon | Handbook of Blind Source Separation -Independent Component Analysis and Applications[END_REF] for the instantaneous mixing case. For convolutive mixtures, [START_REF] Ozerov | Factorial scaled hidden markov model for polyphonic audio representation and source separation[END_REF][START_REF] Higuchi | A unified approach for underdetermined blind signal separation and source activity detection by multichannel factorial hidden Markov models[END_REF] presented a framework for joint processing of MASS and diarization, where factorial Hidden Markov models were used to model the activity of the sources. Unfortunately, due to its factorial nature, the model does not account for correlations on the activity of different sources, i.e. the activities of the different sources are assumed independent with each other, which is questionable for natural conversations for example. Very recently, joint processing of the two tasks have been proposed in [START_REF] Kleijn | Robust and low-complexity blind source separation for meeting rooms[END_REF] (for over-determined mixtures) and in [START_REF] Kounades-Bastian | An EM algorithm for joint source separation and diarization of multichannel convolutive speech mixtures[END_REF][START_REF] Kounades-Bastian | Exploiting the intermittency of speech for joint separation and diarization[END_REF]. The models in [START_REF] Kounades-Bastian | An EM algorithm for joint source separation and diarization of multichannel convolutive speech mixtures[END_REF] and [START_REF] Kounades-Bastian | Exploiting the intermittency of speech for joint separation and diarization[END_REF] combine a diarization state model (that encodes the combination of active sources within a given set of maximum size N ) with the multichannel LGM+NMF model of [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF] and with the full-rank spatial covariance matrix model of [START_REF] Duong | Under-determined reverberant audio source separation using a full-rank spatial covariance model[END_REF], respectively. In contrast to [START_REF] Ozerov | Factorial scaled hidden markov model for polyphonic audio representation and source separation[END_REF][START_REF] Higuchi | A unified approach for underdetermined blind signal separation and source activity detection by multichannel factorial hidden Markov models[END_REF], modelling the activity of all sources jointly using a diarization state enables to exploit the potential correlations on speaker activity.

Note that estimating the number of active sources present in the scene is a good example of problem common to source separation and source localization. Strategies have been developed for the automatic detection of the number of sources within a (probabilistic) source localization framework, e.g. [START_REF] Fallon | Acoustic source localization and tracking of a time-varying number of speakers[END_REF][START_REF] Wang | An iterative approach to source counting and localization using two distant microphones[END_REF][START_REF] Li | Multiple-speaker localization based on direct-path features and likelihood maximization with spatial sparsity regularization[END_REF]. Obviously, such strategies may be exploited in or extended to source separation, as already explored in [START_REF] Roman | Speech segregation based on sound localization[END_REF][START_REF] Roman | Binaural tracking of multiple moving sources[END_REF].

Spatially diffuse sources and long mixing filters

The vast majority of current state-of-the-art MASS methods considers convolutive mixtures of sources, as expressed by (2): each source image within the mixture signal recorded at the microphones is assumed to be the result of the convolution of a small "point" source signal with the impulse response of the source-to-microphone acoustic path. This formulation implies that each sound source is assumed to be spatially concentrated at a single point of the acoustic space. This is fine to some extent for speech signals, but this is more questionable for "large" sound sources such as wind, trucks, or large musical instruments, which emit sound in a large region of space. In that later case, each source image is better considered as a spatially distributed source.

Moreover, if the source signal propagates in highly reverberant enclosures, the late tail of the room impulse response (RIR) is perceived as arriving from all directions. If the reverberation time T 60 , defined as the elapsed time until the reverberation level has decreased by 60 dB from its initial value, the reverberant sound field is said to be diffuse, homogenous and isotropic. The normalized spatial correlation between the received signal at two different microphones i and i , and at frequency f , is given in closed-form for spherical symmetry [START_REF] Cox | Spatial correlation in arbitrary noise fields with application to ambient sea noise[END_REF][START_REF] Kuttruff | Room acoustics[END_REF]:

Ω ii (f ) = E spat (r ij (f )r * i j (f )) E spat (|r ij (f )| 2 ) E spat (|r i j (f )| 2 ) = sin(2πf ii /c) 2πf ii /c (4)
where E spat denotes spatial expectation over all possible absolute positions of the sources and of the microphone array in the room, and ii denotes the distance between the microphones. A closed-form result also exists for cylindrical symmetry [START_REF] Cook | Measurement of correlation coefficients in reverberant sound fields[END_REF]. A simulator of both sound fields can be found in [START_REF] Habets | Generating sensor signals in isotropic noise fields[END_REF].

To address this problem, the authors of [START_REF] Duong | Under-determined reverberant audio source separation using a full-rank spatial covariance model[END_REF] proposed to use a full-rank (FR) spatial covariance matrix (SCM) for characterizing the spatial distribution of the source images (across channels), instead of the rank-1 matrix corresponding to the MTF model [START_REF] Ozerov | Multichannel nonnegative matrix factorization in convolutive mixtures for audio source separation[END_REF]. This FR-SCM model is assumed to represent diffuse sources better than the MTF approximation and it is compliant with the vision of a diffuse source as a sum of subsources with identical PSD distributed in a large region of the physical space. The model parameters are estimated using an expectation-maximization (EM) algorithm. Note that this approach does not attempt to explicitly model the mixture process but rather focuses on the properties of the resulting source images. The FR-SCM model was further used and improved in [START_REF] Arberet | Nonnegative matrix factorization and spatial covariance model for under-determined reverberant audio source separation[END_REF][START_REF] Duong | Spatial location priors for Gaussian model based reverberant audio source separation[END_REF].

Moreover, even for point sources, the processing of convolutive mixtures in the STFT domain is confronted to a severe limitation with respect to intothe-wild scenarios: The length of room impulse responses (RIRs) is in general (much) larger than the length of the STFT analysis window, which can severely question the validity of the approximation [START_REF] Araki | The fundamental limitation of frequency domain blind source separation for convolutive mixtures of speech[END_REF]. Typical values for the STFT window length for speech mixtures are within 16-64 ms, in order to adapt to the global non-stationarity / local stationarity of speech signals. At the same time, the typical T 60 reverberation time of usual home/office rooms is within 200-600 ms, and large meeting rooms or auditorium can have a T 60 larger than 1 s. The ratio between RIR length and STFT window length can thus easily be within 10-50 instead of being lower than 1. Therefore, (3) can be a quite poor approximation, even for moderately reverberant environments, if the sources are positioned further away from the microphones. 5 The MTF approximation is still widely used to address convolutive mixtures problems due to its practical interest: The fact that a time-domain convolutive mixture becomes an independent instantaneous mixture at each frequency bin f facilitates the technical development of solutions to the separation problem. While this can be a reasonable choice of model, we stress that the validity of the MTF approximation should be verified prior to its application. In some cases, a mixed MTF and full-rank models should be considered [START_REF] Duong | Under-determined reverberant audio source separation using a full-rank spatial covariance model[END_REF].

Here again, compared to the impressive amount of papers on MASS methods for convolutive mixtures based on (3), only a few have addressed solutions to overcome the limitation of the MTF approximation. Although they are only a few, these solutions can be classified into two general approaches: Methods mixing time-domain (TD) and TF-domain processing, and methods that totally remain in the TF-domain.

As for the first approach, the method of [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF] consists in modeling the sources in the TF domain while keeping a TD representation of the convolutive mixture using the inverse transform expression. The TD source signals are estimated using a Lasso optimization technique (hence the method is called W-Lasso for wideband Lasso) with a regularization term on STFT source coefficients to take into account source sparsity in the TF domain. In [START_REF] Arberet | Sparse reverberant audio source separation via reweighted analysis[END_REF] an improved W-Lasso with a re-weighted scheme is presented. W-Lasso achieves quite good source separation performance in reverberant environments, at the price of a tremendous computation time. Also, only semi-blind separation with known mixing filters was addressed in [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF], which is poorly satisfying with regards of going towards separation into the wild. A similar hybrid TD/TF approach was recently followed in [START_REF] Leglaive | Multichannel audio source separation: Variational inference of time-frequency sources from time-domain observations[END_REF][START_REF] Leglaive | Separating time-frequency sources from time-domain convolutive mixtures using non-negative matrix factorization[END_REF]. The source signals were represented using either the modified discrete cosine transform (MDCT), which is real-valued and critically sampled, or the odd-frequency STFT (OFSTFT). A probabilistic LGM + NMF model was used for the source coefficients, which were inferred from TD mixture observations using a VEM algorithm. This led to very interesting results, at the price of huge computation. Here also, most experiments were conducted in a semi-blind setup with known mixing filters since the joint estimation of the filters' impulse responses remains difficult.

As for the approaches that work totally in the TF domain, let us first mention that the authors of [START_REF] Duong | Under-determined reverberant audio source separation using a full-rank spatial covariance model[END_REF] have shown that, in addition to modeling diffuse sources, their method is able to circumvent to some extent the discussed limitations of the MTF approximation. Other methods have investigated TF mixture models more accurate than (3). Fundamentally, the time-domain convolution can be exactly represented as a two-dimensional filtering in the TF domain [START_REF] Gilloire | Adaptive filtering in subbands with critical sampling: analysis, experiments, and application to acoustic echo cancellation[END_REF]. This representation was used for linear system identification in [START_REF] Avargel | System identification in the short-time Fourier transform domain with crossband filtering[END_REF] as an alternative to MTF, under the name of cross-band filters (CBFs). Using CBFs, a source image STFT coefficient is represented as a summation over frequency bins of multiple convolutions between the input source STFT coefficients and the TF-domain filter impulse response, along the frame axis. This exact representation becomes an approximation when we limit the number of bins either in the frequency-wise summation or in the frame-wise convolution. In particular, considering only the current frequency bin, i.e. a unique convolution along the STFT frame axis, is a reasonable practical approximation, referred to as the convolutive transfer function (CTF) model [START_REF] Talmon | Relative transfer function identification using convolutive transfer function approximation[END_REF]. Using this CTF model, the mixture model ( 2) writes in the STFT domain:

x(f, n) = J j=1 Qa-1 n =0 a j (f, n )s j (f, n -n ) + b(f, n). (5)
Here, the i-th entry of a j (f, n), denoted a ij (f, n), is not the DFT of a ij (t) (nor is it its STFT), but it is its CTF. The CTF contains several STFT-frame-wise filter taps and its expression is a bit more complicated than the DFT, though easily computable from a ij (t), see [START_REF] Avargel | System identification in the short-time Fourier transform domain with crossband filtering[END_REF]. The full CBF representation was considered for solving the MASS problem in [START_REF] Badeau | Multichannel high-resolution NMF for modeling convolutive mixtures of non-stationary signals in the timefrequency domain[END_REF], in combination with a high-resolution NMF (HR-NMF) model of the source signal. A variational EM (VEM) algorithm was proposed to estimate the filters and infer the source signals. Unfortunately, due to the model complexity, this method was observed to perform well only in an oracle setup where both filters and source parameters are initialized from the individual source images. Therefore, in the current state-of-knowledge, the CBFs seem difficult to integrate into a realistic MASS framework and one has to resort to CTF-like approximations.

An MVDR beamformer, implemented in a generalized sidelobe canceller (GSC) structure, that utilizes the CTF model was proposed in [START_REF] Talmon | Convolutive transfer function generalized sidelobe canceler[END_REF]. It was shown to outperform the GSC beamformer which uses the MTF approximation [START_REF] Gannot | Signal enhancement using beamforming and nonstationarity with applications to speech[END_REF]. It can be noted that, as opposed to the full-rank model, the CTF approximation allows for coherent processing and can therefore implement an almost perfect null towards a point interfering source. The ability of the FR model to suppress the interference signal is limited by the number of microphones and their constellation and cannot exceed I 2 [START_REF] Parsons | Maximum directivity proof for three-dimensional arrays[END_REF] for fully diffuse signal.

Interestingly, the pioneering work [START_REF] Attias | New EM algorithms for source separation and deconvolution with a microphone array[END_REF] combined an STFT-domain convolutive model very similar to [START_REF] Arberet | A robust method to count and locate audio sources in a multichannel underdetermined mixture[END_REF] with a Gaussian mixture model (GMM) of source signals. In this paper, the STFT-domain convolution was intuited from empirical observations and was referred to as "subband filters" (no theoretical justification nor references were provided). Because of the overall complexity of the model (especially the large number of GMM components that is necessary to accurately represent speech signals), the author resorted to a VEM algorithm for parameters estimation. In [START_REF] Higuchi | Joint audio source separation and dereverberation based on multichannel factorial hidden Markov model[END_REF], an STFT-domain convolutive model also very similar to [START_REF] Arberet | A robust method to count and locate audio sources in a multichannel underdetermined mixture[END_REF] was used together with an HMM on source activity. However, the optimization method used to estimate the parameters and infer the source signal is quite complex.

A Lasso-type optimization applied to the MASS problem was considered in [START_REF] Li | Audio source separation based on convolutive transfer function and frequency-domain Lasso optimization[END_REF] within the CTF framework. More specifically, the 2 -norm model fitting term of Lasso was defined at each frequency bin with the STFT-domain convolutive mixture (5) instead of the TD convolutive mixture (2) as done in [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF]. In parallel, the 1 -norm regularizer of Lasso was kept so as to exploit the sparsity of TF-domain audio signals. Because the number of filter frames Q a in ( 5) is much lower than the length L a of the TD filter impulse response in (2), the computation cost in [START_REF] Li | Audio source separation based on convolutive transfer function and frequency-domain Lasso optimization[END_REF] is drastically reduced compared to [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF]. This was obtained at the price of quite moderate loss in separation performance, showing the good accuracy of the CTF approximation. However, as for [START_REF] Kowalski | Beyond the narrowband approximation: Wideband convex methods for under-determined reverberant audio source separation[END_REF], this was done only in a semi-blind setup with known filters. To address the use of CTF in a fully blind scenario, the mixture model ( 5) was plugged into a probabilistic framework with an LGM for source STFT coefficients in [START_REF] Li | An EM algorithm for audio source separation based on the convolutive transfer function[END_REF]. An exact EM algorithm was developed for joint estimation of the parameters (source parameters and CTF filter coefficients) and inference of the source STFT coefficients. The joint estimation of source STFT coefficients and CTF mixing filter coefficients was recently addressed as a pure optimization problem in [START_REF] Feng | Séparation aveugle de sources: de l'instantané au convolutif[END_REF].

Another attempt to address the problem of long filters is presented in [START_REF] Schwartz | Multi-microphone speech dereverberation and noise reduction using relative early transfer functions[END_REF]. In this work, the RTF is split into an early part which is coherently processed and a late part which is treated as an additive noise. This noise is reduced by a combination of an MVDR beamformer and a postfilter. In [START_REF] Schwartz | Nested generalized sidelobe canceller for joint dereverberation and noise reduction[END_REF], a nested GSC scheme was proposed that treats the long RIRs jointly as a coherent phenomenon, using CTF modelling, and as a diffused sound field. Different blocks of the proposed scheme use the different RIR models.

In parallel with the above attempts to model long mixing filters, as briefly mentioned in the previous sections, several more or less recent studies have considered the modeling of the mixing filters/process as latent variables, possibly in a fully Bayesian framework, either to better account for uncertainty in filter estimation or to introduce prior knowledge on these filters (for example the approximate knowledge of source DOA or the specific structure of room acoustic impulse responses). Because of room limitation, we do not describe those works and only add [START_REF] Cemgil | Variational and stochastic inference for Bayesian source separation[END_REF][START_REF] Leglaive | Multichannel audio source separation with probabilistic reverberation priors[END_REF][START_REF] Girin | On the use of latent mixing filters in audio source separation[END_REF] to the already cited references [START_REF] Ozerov | A general flexible framework for the handling of prior information in audio source separation[END_REF][START_REF] Higuchi | Underdetermined blind separation and tracking of moving sources based on DOA-HMM[END_REF][START_REF] Duong | Spatial location priors for Gaussian model based reverberant audio source separation[END_REF][START_REF] Kounades-Bastian | A variational EM algorithm for the separation of time-varying convolutive audio mixtures[END_REF].

It is clear from the above discussion that many models of the mixing filters are used in the literature. It is still in open question, which of the models is the most appropriate. Most probably, the answer to this question depends on the scenario.

Ad hoc microphone arrays

Classical microphone arrays usually consist of a condensed set of microphone mounted on a single device. Establishing wireless acoustic sensor networks (WASNs), comprising multiple cooperative devices (e.g., cellphone, tablet, hearing aid, smart watch) may increase the chances to find a subset of the micro-phones that is close to a relevant sound source. Consequently, WASNs may demonstrate higher separation capabilities than a single-device solution. The wide-spread availability of devices equipped with multiple microphones makes the vision closer to reality.

The distributed and ad hoc nature of WASNs arises new challenges, e.g. transmission and processing constraints, synchronization between nodes and dynamic network topology. Addressing these new challenges is a prerequisite for fully exploiting the potential of WASNs.

Several families of distributed algorithms, that only require the transmission of a fused version of the signals received by each node was proposed in [START_REF] Markovich-Golan | Optimal distributed minimum-variance beamforming approaches for speech enhancement in wireless acoustic sensor networks[END_REF]. The distributed adaptive node-specific signal estimation (DANSE) family of algorithms consists of distributed version of SDW-MWF [START_REF] Bertrand | Distributed adaptive node-specific signal estimation in fully connected sensor networks -part I: sequential node updating[END_REF] and LCMV beamformers [START_REF] Bertrand | Distributed node-specific LCMV beamforming in wireless sensor networks[END_REF]. A distributed version of the GSC beamformer is presented in [START_REF] Markovich-Golan | Distributed multiple constraints generalized sidelobe canceler for fully connected wireless acoustic sensor networks[END_REF]. A randomized gossip implementation of the delay and sum beamformer is presented in [START_REF] Zeng | Distributed delay and sum beamformer for speech enhancement via randomized[END_REF], and a diffusion adaptation method for distributed MVDR beamformer in [START_REF] O'connor | Diffusion-based distributed MVDR beamformer[END_REF]. The problem of synchronizing the clock drifts in several nodes is addressed in e.g. [START_REF] Wehr | Synchronization of acoustic sensors for distributed ad-hoc audio networks and its use for blind source separation[END_REF][START_REF] Pertilä | Passive temporal offset estimation of multichannel recordings of an ad-hoc microphone array[END_REF][START_REF] Wang | Correlation maximization-based sampling rate offset estimation for distributed microphone arrays[END_REF][START_REF] Cherkassky | Blind synchronization in wireless acoustic sensor networks[END_REF].

The full potential of ad hoc microphone arrays to separate source in the wild is yet to be explored.

Conclusions and Perspectives

In this review, we have presented several ways to make MASS and beamforming techniques go from laboratories to real-life scenarios. Laboratory studies are often based on a set of assumptions on the source signals and/or the mixture process that may not be totally realistic (e.g. static, point sources, and spatially stable microphone constellation). In this section, we will briefly explore a few families of devices that already work in real-life scenarios. We will conclude this section and the entire chapter by a perspective on the future of the research in the field.

In recent years, we have witnessed the penetration, in an accelerating pace, of smart audio devices to the consumer electronics market. These devices, designed to work in adverse conditions, include personal assistants embedded in smartphones, portable computers and most notably, smart loudspeakers, e.g. Amazon Echo ("Alexa"), 6 Microsoft Invoke (with "Cortana"), 7 Apple HomePod (with "Siri") 8 and Google Home [START_REF] Li | Acoustic modeling for Google Home[END_REF].

Basically, these smart loudspeakers demonstrate that tremendous progress has already been made in middle-range devices, capable of executing automatic speech recognition (ASR) engines in noisy environments. Smart loudspeakers are equipped with several microphones (six for Apple Homepod, seven for Ama-zon Echo and Microsoft Invoke, and only two microphones for Google Home). Algorithmically, these devices consist of a denoising (mostly using a steered beamformer), dereverberation and echo cancellation stages. The devices usually employ localization (or DOA estimation) algorithms to provide the steering direction, as an important prerequisite to the application of the beamformer. The acquired localization information is also used for indicating the direction of the detected source with respect to the device. As smart loudspeakers acquire and enhance a speech signal in a noisy and reverberant environment, they provide a living example of into-the-wild beamforming. Yet, their performance may be still limited to home scenarios with a predominant and reasonably spatially stable speaker relatively close to the device (as opposed to the above-mentioned adverse scenarios with several active and moving sources with low DRR).

Hearing aids [START_REF] Dillon | Hearing Aids[END_REF] are another example of successful application of MASS / beamforming technologies, aiming at speech quality and intelligibility improvement, as well as enhancing the spatial awareness of the hearing aid wearer (in binaural setting). Hearing devices impose severe real-time constraints on the applied algorithms (latency shorter than 10 ms). Moreover, robustness and reliability are of major importance to prevent potential hearing damage to the hearing impaired person. Binaural cue preservation can be obtained by calculating a common gain to both hearing devices [START_REF] Kamkar-Parsi | Instantaneous binaural target PSD estimation for hearing aid noise reduction in complex acoustic environments[END_REF] or by applying a beamformer that incorporates binaural information into the optimization criterion, e.g. MWF [START_REF] Marquardt | Theoretical analysis of linearly constrained multi-channel Wiener filtering algorithms for combined noise reduction and binaural cue preservation in binaural hearing aids[END_REF] or LCMV [START_REF] Hadad | The binaural LCMV beamformer and its performance analysis[END_REF]. Beamforming-based binaural processing is usually regarded computationally more expensive than the common gain approach. An important issue in designing a binaural enhancement algorithm is to determine the source of interest. In many cases, the beamformer is steered towards the look direction of the hearing aid wearer.

Most cellular phones are nowadays equipped with multiple microphones (3-4) and they usually work in adverse conditions demonstrating reasonable performance. A few systems already employ microphone networks, e.g. smart home and smart cities.

The quest for realistic solutions, capable of processing a large amount of sound sources in real-life environments and in dynamical scenarios of various character, still continues. Many of the theoretical and practical questions are still open and there are performance gaps to be filled for many scenarios such as under-determined mixtures with many simultaneously active sources, multiple moving sources and moving sensors (e.g. robots, cellular phones), highpower and non-stationary noise (e.g. from heavy machinery and drilling noise in mines), and binaural hearing (for both hearing impaired people and robots imitating the Human auditory system [START_REF] Lyon | Human and Machine Hearing: Extracting Meaning from Sound[END_REF]).

Recent years have witnessed a revolution in MASS techniques. Nowadays, deep learning solutions seem to be the new El Dorado for audio processing. Still, most studies deal with single-channel denoising / enhancement / separation algorithms [START_REF] Narayanan | Ideal ratio mask estimation using deep neural networks for robust speech recognition[END_REF][START_REF] Chazan | A hybrid approach for speech enhancement using MoG model and neural network phoneme classifier[END_REF][START_REF] Girgis | Combining mask estimates for single channel audio source separation using deep neural networks[END_REF][START_REF] Hershey | Deep clustering: Discriminative embeddings for segmentation and separation[END_REF][START_REF] Wang | Supervised speech separation based on deep learning: an overview[END_REF][START_REF] Luo | Deep clustering and conventional networks for music separation: Stronger together[END_REF]. More recently, multichannel processing solutions that employ deep learning [START_REF] Nugraha | Multichannel audio source separation with deep neural networks[END_REF][START_REF] Chazan | DNN-based concurrent speakers detector and its application to speaker extraction with LCMV beamforming[END_REF], as well as robust ASR systems [START_REF] Hori | Multi-microphone speech recognition integrating beamforming, robust feature extraction, and advanced NN/RNN backend[END_REF], have been proposed. Deep learning has also influenced the hearing aid industry [START_REF] Wang | Deep learning reinvents the hearing aid[END_REF] and the development of binaural algorithms [START_REF] Zhang | Deep learning based binaural speech separation in reverberant environments[END_REF]. An improved localization strategy that utilizes active head movements and deep learning is proposed in [START_REF] Ma | Exploiting deep neural networks and head movements for robust binaural localization of multiple sources in reverberant environments[END_REF]. Despite the impressive performance gains obtained by deep learning based speech processing approaches, the field is still in its infancy and major breakthroughs are expected in the foreseeable future.

As an outcome from this review, it is evident that a significant progress is still required for obtaining robust and reliable source separation in difficult reallife scenarios, especially under severe online constraints. We anticipate that future solutions will combine ideas from both the array processing / source separation and machine learning paradigms. As always, only such combined solutions, together with practical knowhow, are capable of advancing the already established solutions towards comprehensive audio source separation methods that work into-the-wild.

Of course, this is just a general view of the academic studies in the field as a whole. Some researchers in the field have considered with great attention the practical aspects of audio source separation techniques.

One can select other microphones as the reference microphone or use other methods of normalization. Selecting the reference microphone might have an impact on the MASS performance. This topic is out of the scope of this chapter.

https://robot-ears.eu/

http://twoears.eu/project/

Actually, the ratio between the coherent direct-path and the reverberation tail, the socalled direct-to-reverberant ratio (DRR) plays an important role examining the validity of MTF assumption.

https://www.slideshare.net/AmazonWebServices/designing-farfield-speechprocessing-systems-with-intel-and-amazon-alexa-voice-service-alx305-reinvent-2017

[START_REF] Arberet | Sparse reverberant audio source separation via reweighted analysis[END_REF] https://news.harman.com/releases/harman-reveals-the-harman-kardon-invokeTMintelligent-speaker-with-cortana-from-

microsoft 8 https://www.apple.com/homepod/


Angélique Villière

Gaëlle Arvisenet

Carole Prost

Thierry Serot

Angélique Villiere

Thierry Sérot

Assessment of ciders typicality characterisation through odorant volatile compounds

come   L'archive ouverte pluridisciplinaire

Introduction:

Apple cider is an alcoholic beverage obtained by fermentation of apple must. Unless volatile compounds of apple and apple juice have been extensively studied over the last decades, less information is available on odorant volatile compounds of apple cider. Gas chromatography/olfactometry (GC/O) analysis is a powerful way to determine key compounds of food aroma on condition that the odor of the extract is close to that of the food submitted to extraction. This similarity in odor is not obvious since different classes of compounds are preferentially extracted according to the method chosen. Therefore prior to GC/O analysis, a comparison of the odor of the food with the corresponding extract should be made. The objective of this work was to developpe a representative extraction method to analyse volatile compounds of ciders and to examine those responsible for the odorant perception of different ciders. Because of the undesirable odor of solvent and odors developed when heating to eliminate this solvent, only solvent-free extraction methods were tested.

Experimental:

Chemicals and materials

A commercial cider purchased in a supermarket was used for the development of a representative extraction method. Study of aroma compounds was conducted on three ciders selected among several according to their different organoleptic properties: a sweet cider from Normandy, a brut cider from the pays d'Auge and a pear cider. These ciders were verified to be perceived as different by discriminating tests performed with a panel of 36 naïve judges. Ciders were stored at -80 °C until analyses in 125 mL brown flasks sealed with Teflon/rubber screw caps. Prior analyses, vials were put at 37.5°C for 1h. Chemical standards (p98%) used for mass spectra identification of odorant compounds were purchased from Aldrich (St Quentin Fallavier, France).

Choice of a representative extraction method

For all extractions, 20 mL of cider were sampled except when otherwise mentioned. For static headspace extraction, vials (60 mL) containing the cider were sealed using Teflon/Silicone septa plastic caps (Varian, France). After equilibration at 37.5°C for 1 h, a 250 µL sample of headspace was withdrawn using gastight syringe and injected in the gas chromatograph (GC). Headspace SPME (HS SPME) extractions were conducted with DVB/CAR/PDMS, CAR/PDMS and PDMS fibres (Supelco, Inc., Bellefonte, PA). The sample in a 60 mL vial was equilibrated at 37.5°C in a thermostatic bath for 30 min, and then the fibre was introduced in the headspace for 10 min. Sample was maintened under stirring. The fibre was withdrawn and inserted into the injection port of the GC. Same fibres and procedure were used for aroma extraction by liquid SPME except that 60 mL of cider were placed in the 60 mL vial and the fibre was immersed in the cider. Headspace purge and trap was conducted with a concentrator (model LSC 2000; Teckmar Inc., Cincinnati, OH) equipped with a capillary interface, for cryofocusing, connected to the GC. Cider was introduced into a flask (50 mL) containing a stir bar. Temperature was maintained at 37.5 °C with a heating ring and the headspace of the sample was purged with helium during 5 min at 60 mL.min -1 . Volatile compounds extracted were swept into a Tenax trap at room temperature. After trap desorption, volatiles were cryofocused at -40 °C using carbon dioxide and thermally desorbed at 200 °C for 2 min to be transferred into the column. Same procedure was applied for bubbling purge and trap except that 60 mL of cider was introduced in the flask and the sample was bubbled with helium during 5 min.

Recovery of the extracts and evaluation of the extracts representativeness

Extracts were analyzed on a Varian Star 3400 GC (Varian, Palo Alto, CA) equipped with a DB Wax column (30 m length, 0.32 mm internal diameter, 0.5 µm film thickness, J&W Scientific, Folsom, CA). Detector temperature was set at 260°C and oven temperature program was set from 50 °C (5 min) to 150 °C at 5 °C.min -1 and from 150 °C to 250 °C (7 min) at 10 °C.min -1 . Helium was used as carrier gas at 2 mL.min -1 flow rate. The GC effluent was split 1:1 between the FID (T: 250 °C), to control the chromatographic pattern of the extract, and a 100mL glass syringe 1 . Twelve trained judges were asked to evaluate, on a 10 cm-scale, the proximity between the odour of each extract collected in the syringes and the original cider odour (representativeness). The reference (right anchor of the scale) corresponded to 0.05 mL of the original cider in a syringe. A hidden reference, containing 0.05 mL of the original cider, was also presented with the extracts. All syringes were covered with aluminium foil paper to avoid judges to know if they contained cider or gaseous extracts. Results are the means of the 12 judges' evaluations.

GC/O/MS analysis of 3 discriminated ciders

The GC/O system consisted of a 6890N GC (Hewlett-Packard Co., Palo Alto, CA) equipped with a FID, a mass detector (5973-Network), and a sniffing port ODP2 (Gerstel, Baltimore, MD). The GC effluent was split 1:1:1 between the FID, the mass detector and the sniffing port. Extracts obtained from the three ciders were injected into a DB Wax column (30 m length, 0.32 mm internal diameter, 0.5 µm film thickness, J&W Scientific) and volatile compounds were separated according to chromatographic conditions similar to those described above except for flow rate of helium (3 mL.min -1 ) and oven temperature program: 50 °C (2 min) to 80 °C at 3 °C.min -1, from 80 °C to 105 °C at 5 °C.min -1 and from 105 °C to 240 °C (1 min) at 10 °C.min -1 . Mass spectra were recorded in electron impact mode (70 eV) between 33 and 300 m/z mass range at a scan rate of 2.7 scan.sec -1 . Compounds identification was based on a comparison of mass spectra with those of MS spectra database (Wiley 6) and of standard molecules injected in the same conditions. Coincidence of odour description by judges and literature data could be used to confirm identification of odorant compounds. Olfactometry was conducted with 10 trained judges. GC effluent was carried to the sniffing port using deactivated capillary column heated at 200 °C and supplied with humidified air at 40 C. Judges were asked to assign odor descriptor to each odorant area detected. Results were expressed in frequency detection and odorant areas that were not perceived by at least 3 judges were not taken into account.

Results:

Representativeness of the extracts Representativeness scores of the extracts obtained by the different methods ranged from 2.5/10 for the extracts obtained by liquid SPME with a PDMS fibre, to 6.5/10 for those obtained by HS SPME with a CAR/PDMS fibre (Table 1). The extract obtained by HS SPME with a CAR/PDMS fibre was the most representative of the original cider odour. Multiple comparisons of means showed that, contrary to other extracts, no significant difference was perceived between the odour of this extract and the odour of the hidden reference. Thus, these extraction conditions were applied to investigate the aroma typicality of discriminated ciders.

Olfactometric analyses of 3 discriminated ciders

More than 100 volatiles were detected in the ciders whereas only 50 were detected in previous studies 2,3 . A majority of them was esters (~60) and alcohols (~20). 43 odorant areas (OA) were pointed by judges in the sweet cider from Normandy and 24 in the pays d'Auge and pear ciders (Figure 1). These OA were either associated with a single volatile compound or with several coeluted volatile compounds. 15 OA, which were among the most potent, were common to the 3 ciders (numbered 1 to 15) however their frequency of detection differ from a cider to another. 7 OA were associated to esters (1, 3-5, 7, 10, 13), and were described with fruity or sweety notes. The others were described as floral, green, earthy or metallic. Few OA were characteristic of one cider. Six were found in the sweet cider from Normandy. There were associated with ethanal (fruity), 1-pentanol (fruity, sweet), heptyl acetate (metallic, earthy), hexyl butanoate (cocoa) and hexyl hexanoate (sweet) and one OA (rancid) could not be associated with a chromatographic peak. Two OA were detected only in extract from the pear cider and were associated to ethyl crotonate (sweet) and -citronellol (floral, fruity). No specific OA was found in the cider from the pays d'Auge. These results suggest that both OA specific to a cider and the proportions of major OA common to all ciders, were involved in the ciders odorant characteristics.

Conclusion

These results provide elements to understand the typicality of ciders. Further studies, with a larger selection of ciders, will now be necessary to confirm if these aromatic profiles can explain ciders odorant typicality. Then a connection with the soil and the know-how of a region would be interesting to investigate.

Figure 1 .

 1 Figure 1. Olfactometric pattern of 3 discriminated ciders: a sweet cider from Normandy, a brut cider from the pays d'Auge and a pear cider.

Table 1 .

 1 Representativeness mean scores of the extracts -Letters represented results of multiple comparison of means test.

	Extraction method	Representativeness mean score
	Liquid SPME PDMS fibre	2.5 e
	Headspace purge and trap	2.9 de
	Bubbling purge and trap	3.8 cde
	Liquid SPME Car/PDMS fibre	3.9 cde
	Liquid SPME DVB/Car/PDMS fibre	4.4 cde
	Headspace SPME DVB/CAR/PDMS fibre	4.5 cd
	Headspace SPME PDMS fibre	4.9 bc
	Headspace SPME CAR/PDMS fibre	6.5 ab
	Hidden reference	8.5 a