Prerak Srivastava
email: prerak.srivastava@inria.fr

Antoine Deleforge
email: antoine.deleforge@inria.fr

Emmanuel Vincent
email: emmanuel.vincent@inria.fr

BLIND ROOM PARAMETER ESTIMATION USING MULTIPLE MULTICHANNEL SPEECH RECORDINGS

Knowing the geometrical and acoustical parameters of a room may benefit applications such as audio augmented reality, speech dereverberation or audio forensics. In this paper, we study the problem of jointly estimating the total surface area, the volume, as well as the frequency-dependent reverberation time and mean surface absorption of a room in a blind fashion, based on two-channel noisy speech recordings from multiple, unknown source-receiver positions. A novel convolutional neural network architecture leveraging both single-and inter-channel cues is proposed and trained on a large, realistic simulated dataset. Results on both simulated and real data show that using multiple observations in one room significantly reduces estimation errors and variances on all target quantities, and that using two channels helps the estimation of surface and volume. The proposed model outperforms a recently proposed blind volume estimation method on the considered datasets.

INTRODUCTION

Audio augmented reality (AAR) has received increased interest in the recent years [START_REF] Valimaki | Assisted listening using a headset: Enhancing audio perception in real, augmented, and virtual environments[END_REF]. A key task in AAR is to simulate virtual sound sources that are consistent with the user's environment via, e.g., a hear-through headset. To achieve this in an immersive way, a major hurdle is to estimate the relevant acoustical parameters of the room the user evolves in. The notion of reverberation fingerprint (RF) was introduced in [START_REF] Jot | Augmented reality headphone environment rendering[END_REF] as a compact way to characterize rooms for realistic binaural rendering on AAR headphones. The RF consists of the room's volume V in m 3 and its reverberation time per octave band RT60(b) in s, where b ∈ B = {125, 250, 500, 1k, 2k, 4k} Hz. Under ideal diffuse sound field conditions, these parameters are related via Sabine's well-known formula [START_REF] Eyring | Reverberation time in "dead" rooms[END_REF]

RT60(b) ≈ 0.16 V ᾱ(b) • S (1)
where S = K i=1 Si is the total area of the room's K surfaces in m 2 and ᾱ(b) = K i=1 αi(b)Si/S is the area-weighted mean absorption coefficient in octave band b ∈ B.

An attractive research direction to retrieve such parameters is to estimate them solely from audio recordings of unknown sound sources in the room, using microphones placed on the headset. In [START_REF] Murgai | Blind estimation of the reverberation fingerprint of unknown acoustic environments[END_REF], an algorithm to blindly estimate the RF based on the decay envelope of a single-channel clean speech signal was proposed and showed encouraging results on simulated data. In the same year, the authors of [START_REF] Kataria | Hearing in a shoebox: binaural source position and wall absorption estimation using virtually supervised learning[END_REF] trained a statistical model on a large dataset generated from simulated room impulse responses (RIRs) to blindly estimate the position of a broadband source and the mean absorption coefficient of walls above 1 kHz using a binaural receiver and interchannel cues. Experiments were limited to a fixed room geometry, a fixed receiver position, and simulated data. Later, [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] paved the way for blind room volume estimation from single-channel noisy speech using a convolutional neural network trained using both simulated and real RIRs. The reported results showed that this method can estimate a broad range of volumes within a factor of 2 on real data from the Acoustic Characterization of Environments (ACE) challenge [START_REF] Eaton | Estimation of room acoustic parameters: The ACE challenge[END_REF]. Following the ACE challenge, a number of neuralnetwork-based methods were proposed to blindly estimate reverberation times and direct-to-reverberant ratios from speech signals [START_REF] Xiong | Joint estimation of reverberation time and direct-to-reverberation ratio from speech using auditory-inspired features[END_REF][START_REF] Gamper | Blind reverberation time estimation using a convolutional neural network[END_REF][START_REF] Chai | Blind estimation of reverberation time using binaural complex ideal ratio mask[END_REF][START_REF] Bryan | Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation[END_REF]. In particular, [START_REF] Bryan | Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation[END_REF] proposes a method to generate augmented training datasets from real RIRs. These works estimate broadband values rather than frequency-dependent ones, which may limit the rendering realism in the context of AAR. They also all use single-channel signals. As an interesting alternative approach, [START_REF] Su | Acoustic matching by embedding impulse responses[END_REF] proposes to learn a latent representation referred to as room embedding, which is then used to condition a waveform-to-waveform network that converts a signal recorded in one environment to another. However, this approach cannot be straightforwardly applied to spatial AAR, whereas explicitly knowing room parameters offers more flexibility in binaural rendering. Moreover, explicit room parameters may be useful in themselves for other applications such as speech dereverberation [START_REF] Wu | A reverberation-time-aware DNN approach leveraging spatial information for microphone array dereverberation[END_REF] or audio forensics [START_REF] Moore | Roomprints for forensic audio applications[END_REF].

In this paper, we push these recent research efforts further by proposing a blind method to jointly estimate the total surface area S, the volume V , the reverberation times RT60(b) and the mean absorption coefficients ᾱ(b) in all octave bands b ∈ B. On top of that, we investigate whether using multichannel input and/or fusing estimates across multiple source-receiver positions inside the same room help. To this end, we propose a new convolutional neural network architecture that combines single-and inter-channel features in a joint embedding layer, and uses a likelihood-based loss function that yields adaptive variance estimates, which allow us to fuse multiple independent observations of the same room in a statistically principled way. The model is trained on a carefully-crafted realistic simulated dataset and tested on both simulated and real wet speech recordings1 .

SIMULATED DATASET

To train and validate our approach, a large dataset of noisy speech signals annotated with target room acoustic parameters is created. First, synthetic RIRs are generated using Roomsim [START_REF] Schimmel | A fast and accurate "shoebox" room acoustics simulator[END_REF]. Roomsim is a hybrid shoe-box room acoustic simulator combining the imagesource method [START_REF] Allen | Image method for efficiently simulating small-room acoustics[END_REF] to simulate specular reflections that dominate the early part of RIRs, and the diffuse-rain method [START_REF] Schimmel | A fast and accurate "shoebox" room acoustics simulator[END_REF] to model scattered reflections using stochastic ray-tracing. Simulations are run using a sampling frequency of 48 kHz, a reflection order of 10 for the image-source method and 2000 rays for the diffuse-rain method. 20,000 rooms are simulated with length, width and height drawn uniformly at random in [START_REF] Eyring | Reverberation time in "dead" rooms[END_REF][START_REF] Chai | Blind estimation of reverberation time using binaural complex ideal ratio mask[END_REF] m, [START_REF] Eyring | Reverberation time in "dead" rooms[END_REF][START_REF] Chai | Blind estimation of reverberation time using binaural complex ideal ratio mask[END_REF] m and [2.5, 4] m, respectively, resulting in S ∈ [48, 360] m 2 and V ∈ [18, 400] m 3 . For each room, 5 RIRs corresponding to different source-receiver positions are generated. Omnidirectional sources and receivers are used and placed uniformly at random at least 30 cm from each surface and from each other. The receiver is a two-microphone array placed parallel to the floor with an aperture of 22.5 cm, that is a typical head or headset width. For each room, a single scattering coefficient drawn uniformly at random in [0.2, 1] is used on all surfaces and octave bands for the diffuse-rain method. The absorption coefficients of the 6 surfaces in 6 octave bands are sampled using the reflectivity-biased sampling strategy proposed in [START_REF] Bastien | Mean absorption coefficient estimation from impulse responses: Deep learning vs. Sabine[END_REF]. Each αi(b) is drawn uniformly at random inside ranges based on measured databases of commonly encountered surface materials, while giving each surface a 50% probability of having a frequency-independent reflective profile (αi(b) < 0.12). As showed in [START_REF] Bastien | Mean absorption coefficient estimation from impulse responses: Deep learning vs. Sabine[END_REF] The obtained RIRs are downsampled to 16 kHz and convolved with random speech excerpts from the LibriSpeech [START_REF] Panayotov | Librispeech: An ASR corpus based on public domain audio books[END_REF] corpus. The resulting 3 s two-channel reverberated signals are then corrupted with both static microphone noise, i.e., independent additive white Gaussian noise on each channel, and spatially-diffuse babble noise, i.e., speech-shaped noise convolved with the late part (>50 ms) of a random RIR in the room. For the noise levels to be realistic, signals from sources that are placed further away from the receiver should exhibit lower signal-to-noise ratios (SNRs). To achieve this, for each room, we first generate a reference signal using a random speech source placed 1 meter in front of a receiver (not used in the final dataset). Static and diffuse noise levels for this signal are set to obtain SNRs drawn uniformly at random in [70, 90] dB and [30, 60] dB, respectively (different levels for each source-receiver position in the room are set). These noise levels are then kept fixed for the final mixtures, irrespective of the distance from the speech source to the receiver. This resulted in an overall SNR range of [-10, 65] dB across the dataset.

NEURAL NETWORK MODEL

Proposed Architecture, Cost Function, and Fusion Method

The proposed neural network architecture is depicted in Fig. 1. Single-channel (SC) and inter-channel (IC) features are extracted from the time-domain two-channel input signal x in the form of spectrograms. We use the short-time Fourier transform (STFT) with 96 ms Hann windows and 50% overlap to obtain a complex spectrogram {Xi(f, t)} F,T f,t=1 for each channel i, with F = 769 positive frequency bins and T = 63 time frames for a 3 s input signal (our architecture works on arbitrary input length). This choice performed best among 32, 64, 96 and 128 ms windows in our preliminary experiments. Then, SC features are computed as

|X1(f, t)|, which performed better than |X1(f, t)| 2 , |X1(f, t)| 1/2 , or log |X1(f, t)|.
IC features are obtained by concatenating interchannel level differences (ILD) and phase differences (IPD):

ILD(f, t) = log |X1(f, t)| -log |X2(f, t)|, (2)
IPD(f, t) = Re, Im X1(f, t)X * 2 (f, t) |X1(f, t)X * 2 (f, t)| . (3)
These features are then processed through 1D convolutional blocks (1D-Conv), which were recently proposed in the Conv-TasNet architecture in the context of speech separation [START_REF] Luo | Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation[END_REF]. These blocks consist of separable convolutions (depth-wise and point-wise) intertwined with rectified linear unit (ReLU) activations and followed by layer normalization [START_REF] Ba | Layer normalization[END_REF]. The latter proved to be crucial in our experiments, as it creates scale-invariant representations. For SC features, three 1D-Conv blocks with increasing dilation factors along the frequency-axis and a kernel size of 11 are used, while only one block is used on IC features, as this showed to give best results. The obtained representations are concatenated along the frequency axis and average-pooled along the time-axis to yield a time-independent, 1248-dimensional embedding vector. The embedding vector is finally passed through 3 fully-connected layers of respective dimensions 96, 48 and 28 to obtain 2 × D = 14 outputs consisting of the estimated room parameters µ θ (x) ∈ R D and the estimated variances σ 2 θ (x) ∈ R D (or uncertainties) on these parameters. The network parameters θ are optimized by minimizing the following Gaussian negative log-likelihood loss function:

L θ (x, y) = -log p θ (y|x) = -log N (y; µ θ (x), σ 2 θ (x)) c = 1 2 D d=1 log σ 2 d,θ (x) + (y d -µ d,θ (x)) 2 σ 2 d,θ (x) (4)
where y ∈ R D denotes the ground truth room parameters. A benefit of this approach is that it adaptively weights errors on individual parameters. The estimated variances can also be used to fuse estimated obtained from J independent observations {xj} J j=1 of the same room using the following formula derived from Bayes' rule:

p θ (y d |x = [x1, . . . , xJ ]) = N (y d ; μd,θ (x), 1/γ 2 d,θ (x)) (5) with μd,θ (x) = J j=1 γ 2 d,θ (xj) γ2 d,θ (x) µ d,θ (xj), γ2 d,θ (x) = J j=1 γ 2 d,θ (xj)
where

γ 2 d,θ (xj) = 1/σ 2 d,θ (xj)
is the estimated precision for observation xj and μd,θ (x) is the fused estimate.

To avoid issues due to scale differences, the ground truth parameters are normalized at training time by dividing them by their standard deviations over the training set, which are saved and multiplied with the network output at test time. The network is trained using the ADAM optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a learning rate of 10 -4 and a batch size of 120. A dropout rate of 0.2 and 0.4 was used in convblocks and in fully connected layers to avoid over-fitting. We used a patience of 15 epochs on the validation set for early stopping. Our models generally converged in 100-150 epochs.

Other Tried Architectures

In addition to 1D-Conv blocks, Conv-TasNet introduced the idea of learnable input filters [START_REF] Luo | Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation[END_REF]. We tried replacing the STFT by such learnable filters to estimate our SC features, but this yielded worse performance. We also tried an architecture inspired by x-vectors, a state-of-the-art model for speaker recognition [START_REF] Snyder | X-vectors: Robust DNN embeddings for speaker recognition[END_REF], but this did not perform well on our data. We experimented with replacing the proposed maximum-likelihood loss (4) by a simple mean squared error loss. This yielded significantly higher errors while losing the benefit of easily fusing multiple observations in one room. Estimating log(V ) and log(S) instead of V and S as suggested in [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] did not improve results on our data, most likely because the ranges we consider for these quantities are smaller. Finally, instead of estimating ᾱ, RT60, V and S jointly in a multi-task fashion, we trained the same architecture to estimate each of these quantities individually. The obtained results were identical, while increasing the number of model parameters and training time by a factor of 4, which suggests that our proposed network achieves efficient weight sharing.

EXPERIMENTS AND RESULTS

Simulated Data

We first evaluate the proposed approach on the 2,000 unseen rooms of the simulated test set (see Section 2), using between 1 and 5 twochannel 3-s noisy speech recordings at different source-receiver positions per room, based on the fusion method in [START_REF] Kataria | Hearing in a shoebox: binaural source position and wall absorption estimation using virtually supervised learning[END_REF]. Two variants of Figure 2: Mean absolute error achieved on simulated data by [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] vs. the proposed model with one-or two-channel inputs, as a function of the number of source-receiver positions fused in each room. our model are compared: the full two-channel architecture depicted in Fig. 1, and the same architecture without the upper inter-channel processing part of the network, i.e., using only one channel. These variants are compared to our implementation of the single-channel, single-position blind room volume estimation method of [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] trained on the same data. We could not find competing methods for blind total surface estimation or blind frequency-dependent reverberation time and mean absorption coefficient estimation in the literature. The metric used is the mean absolute error on each room parameter.

As can be seen in Fig. 2, increasing the number of fused observations per room significantly reduces errors on all parameters. Using five positions and two channels, mean absolute errors of 0.052 for ᾱ (training range [0.02, 0.6]), 0.18 s for RT60 (training range [0.2, 3.2]), 42 m 2 for S (training range [48,360]) and 54 m 3 for V (training range [START_REF] Panayotov | Librispeech: An ASR corpus based on public domain audio books[END_REF]400]) are obtained. The proposed model significantly outperforms the one in [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] for volume estimation, reducing the error by 13% using one channel and one observation, and by 31% using two channels and five observations. Interestingly, we observe that using two channels instead of one significantly reduces surface and volume estimation error without significantly impacting mean absorption and reverberation time estimation, according to 95% confidence intervals. This may be interpreted by the fact that the latter mostly govern the late, spatially-diffuse regime of RIRs, and hence should have limited correlation with inter-channel cues that mostly capture spatial characteristics. Conversely, S and V are inherently spatial quantities as they relate to the room's geometry and hence early echoes, which do correlate with IC cues [START_REF] Di Carlo | MIRAGE: 2D source localization using microphone pair augmentation with echoes[END_REF][START_REF] Di Carlo | dEchorate: a calibrated room impulse response database for echo-aware signal processing[END_REF].

We hence carry an additional experiment to study whether the improvement observed when using two channels is truly explained by concatenating SC and IC representations at the embedding layer (see Fig. 1), or is only due to doubling the number of signal samples in a given source-receiver position, which mitigates noise. Four different embeddings are compared, namely (i) the SC representation of one speech signal (1mic, 1sig), (ii) the average of the SC As can be seen in Table 1, the only model which significantly improves surface and volume estimation is the one using IC features, despite the fact that the 2nd and 4th models also benefit from the same number of input signal samples.

So far, the reported errors on ᾱ and RT60 were averaged over all 6 octave bands. Table 2 reports detailed errors per octave band. As can be seen, the mean absorption is well estimated in all octave bands, while reverberation-time errors steadily decrease by a factor of 3 from 125 Hz to 4 kHz. This could be explained by the fact that less information is available in the narrower, lower octave bands.

Real Data

To check how well our method generalizes to real data, we use the recently released dEchorate dataset [START_REF] Di Carlo | dEchorate: a calibrated room impulse response database for echo-aware signal processing[END_REF], which contains wet speech recordings made inside the acoustic lab at Bar-Ilan University. The lab is a shoe-box room of size 5.7 × 6 × 2.4 m (S = 125 m 2 , V = 82 m 2 ) where each of the walls, floor and ceiling can be set either to either a reflective mode or an absorbent mode. 5 arrays of 6 omnidirectional microphones and 6 directional loudspeakers are placed inside the room, yielding 5 × 6 = 30 multi-channel speech recordings per room configuration. For each room, ground truth ᾱ and RT60 values are provided for the four octave bands from 500 Hz to 4 kHz. Hence, results in lower octave bands are omitted here. In our experiments, we use 3 × 30 = 90 3-s speech recordings corresponding to the 2-channel sub-arrays with aperture 22.5 cm and to the 3 room configurations involving 3 or more reflective surfaces, as these most closely match the considered scenario. For these rooms, ᾱ(b) ranges from 0.16 to 0.35 and RT60(b) from 0.25 to 0.66 s. Table 3 reports mean absolute errors using the proposed approach with or without IC features. We report results using either 1 or 5 source positions and a fixed receiver. For the latter, we exclude one out of the 6 available source positions for each test, so that there are 90 tests in each case. The mean absolute volume estimation error using the single-channel, single-position approach of [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] is reported as well. Encouragingly, errors obtained with our approach are of comparable orders to those obtained on simulated data. In the single-channel, single-position case, volume estimation errors obtained with our model are comparable to [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF]. For RT60, S and V , we are able to reproduce the observation that increasing the number of source-receiver positions significantly decreases errors, at least using IC features. We also observe again that using IC features significantly improves the estimation of S and V . The errors obtained on ᾱ are less consistent than those on simulated data, which may be due to the difficulty of reliably annotating mean absorption coefficients in real rooms.

Finally, Table 4 shows the standard deviations of estimated values by the same methods over the 30 recordings from the room with 3 reflective surfaces. Encouragingly, the relatively low standard deviations reveal the ability of the models to provide parameter estimates that are stable within a room, and do not depend much on the source-receiver position. Moreover, it can be seen that, as expected, using five observations in a room instead of one systematically decreases the standard deviation of estimates.

CONCLUSION

This study revealed that using inter-channel cues can significantly improve the blind estimation of a room's volume and surface from noisy speech, while for estimating reverberation and absorption parameters, a single channel is sufficient. It also highlights that fusing multiple measurements reduces estimation errors and variances on all parameters. Finally, we showed that a system trained on a carefully simulated training set offers reasonable generalization capabilities to real data. Future work will include the use of dataaugmentation and domain adaptation techniques to improve real data results, extensions to binaural or ambisonic receivers, and the joint estimation of local parameters such as the positions and properties of the source and individual surfaces in the room, with the help of early acoustic echoes.

ACKNOWLEDGMENTS

This work was made with the support of the French National Research Agency through project HAIKUS "Artifical Intelligence applied to augmented acoustic scenes" (ANR-19-CE23-0023). Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr). We thank the authors of [START_REF] Genovese | Blind room volume estimation from singlechannel noisy speech[END_REF] for their helpful advice.

,

Figure 1 :

 1 Figure 1: Diagram of the proposed neural network architecture.

  , this yields more realistic and more diverse distributions of absorption and reverberation times (RT60(b) ∈ [0.2, 3.2] s and ᾱ(b) ∈ [0.02, 0.6] in our dataset) than, e.g., by sampling each αi(b) uniformly in [0, 1].

  The 5×20k = 100k two-channel noisy speech signals are divided into training, validation and test sets of respective size 80k, 10k and 10k with no room or speech-signal overlap between them. Each room is annotated with the 14 target parameters, namely, 6 mean absorption coefficients and 6 reverberation times in all octave bands, the total surface area S and the volume V . For each room and each octave band, a unique reverberation time is estimated by taking the median value over the 5 source-receiver positions available. The values are obtained by linear regression over the -5 dB to -25 dB decay of Schroeder curves[START_REF] Schroeder | New method of measuring reverberation time[END_REF].

Table 1 :

 1 Shaded areas indicate 95% confidence intervals. Mean absolute error achieved on simulated data for one source-receiver position using different inputs and features. Bold numbers indicate the best statistically significant result per column based on 95% confidence intervals, when there is one.

	Input	Feature	ᾱ	RT 60 (s)	S (m 2 ) V (m 3 )
	1mic, 1sig	SC	0.055	0.225	55.0	68.8
	1mic, 2sig	SC	0.058	0.222	55.4	69.7
	2mic, 1sig	SC	0.057	0.221	54.0	67.7
	2mic, 1sig	SC+IC	0.055	0.236	49.9	63.7

Table 2 :

 2 Mean absolute errors on ᾱ(b) and RT60(b) in the 6 octave bands achieved by the proposed model with two channels and 1 or 5 source-receiver positions per room on simulated data.

	Octave bands	ᾱ	1 position RT 60 (s)	ᾱ	5 positions RT 60 (s)
	125 Hz	0.056	0.392	0.051	0.320
	250 Hz	0.060	0.305	0.055	0.249
	500 Hz	0.057	0.228	0.051	0.170
	1 kHz	0.053	0.188	0.050	0.146
	2 kHz	0.054	0.165	0.051	0.122
	4 kHz	0.052	0.139	0.049	0.106
	representations of two speech signals (1mic, 2sig), (iii) the concate-
	nation of the SC representations of the two individual channels of
	one speech signal (2mic, 1sig) and (iv) the proposed concatenation
	of SC and IC representations. Speech signals are always 3-s long.

Table 3 :

 3 Mean absolute error achieved over 3 rooms from the real dEchorate dataset. Bold numbers indicate the best statistically significant result per column, based on 95% confidence intervals.

	Method Features	# pos	ᾱ	RT 60	S	V
	[6]	SC	1	-	-	-	10.0
	Ours	SC	1	0.030	0.161	27.2	31.8
	Ours	SC	5	0.024	0.090	19.6	23.0
	Ours	SC+IC	1	0.031	0.100	34.7	39.7
	Ours	SC+IC	5	0.015	0.054	16.5	18.9

Table 4 :

 4 Standard deviation of parameter estimates for room "011100" of the real dEchorate dataset.

Our code to reproduce this paper is available at github.com/ prerak23/RoomParamEstim.

October 17-20, 2021, New Paltz, NY