Théo Mariotte

Anthony Larcher

Silvio Montrésor

Jean-Hugh Thomas

Channel-Combination Algorithms for Robust Distant Voice Activity and Overlapped Speech Detection

Keywords: speaker diarization, distant speech, microphone array, voice activity detection, overlapped speech detection, channel-number agnostic

Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD) are key pre-processing tasks for speaker diarization. In the meeting context, it is often easier to capture speech with a distant device. This consideration however leads to severe performance degradation. We study a unified supervised learning framework to solve distant multi-microphone joint VAD and OSD (VAD+OSD). This paper investigates various multi-channel VAD+OSD front-ends that weight and combine incoming channels. We propose three algorithms based on the Self-Attention Channel Combinator (SACC), previously proposed in the literature. Experiments conducted on the AMI meeting corpus exhibit that channel combination approaches bring significant VAD+OSD improvements in the distant speech scenario. Specifically, we explore the use of learned complex combination weights and demonstrate the benefits of such an approach in terms of explainability. Channel combination-based VAD+OSD systems are evaluated on the final back-end task, i.e. speaker diarization, and show significant improvements. Finally, since multi-channel systems are trained given a fixed array configuration, they may fail in generalizing to other array setups, e.g. mismatched number of microphones. A channel-number invariant loss is proposed to learn a unique feature representation regardless of the number of available microphones. The evaluation conducted on mismatched array configurations highlights the robustness of this training strategy.

I. INTRODUCTION

S PEAKER diarization answers the question Who spoke and when? from an audio stream. Although much research has been conducted on speaker diarization, it remains a challenging automatic speech processing task as proven by the numerous organized challenges [START_REF] Ryant | The Third DIHARD Diarization Challenge[END_REF][START_REF] Yu | M2met: The ICASSP 2022 multi-channel multiparty meeting transcription challenge[END_REF]. Most diarization systems rely on pipeline architectures based on the segmentation of the audio signal and the clustering of these segments to group them by speaker [START_REF] Park | A review of speaker diarization: Recent advances with deep learning[END_REF]. In the segmentation process, Voice Activity Detection (VAD) and Overlapped Speech Detection Manuscript created in September 2022. The work was conducted at LAUM and LIUM, Le Mans Université, France. This project has received funding from LMAC grant and the European Union's Horizon 2020 research and innovation program under the Marie Sklodowska-Curie grant agreement No 101007666, the Agency is not responsible for these results or use that may be made of the information. This work was performed using HPC resources from GENCI-IDRIS (Grant 2022-AD011012565). Théo Mariotte, Jean-Hugh Thomas, and Silvio Montrésor are with the Laboratoire d'Acoustique de l'Université du Mans (LAUM), IA-GS, UMR CNRS 6613. Théo Mariotte and Anthony Larcher are with the Laboratoire d'Informatique de l'Université du Mans (LIUM), Institut Claude Chappe. Le Mans Université, 72000 Le Mans, France (e-mail: anthony.larcher@univ-lemans.fr) (OSD) are two essential sub-tasks. VAD consists in detecting speech segments from non-speech (e.g. silence, background noise...) in an audio stream. In most approaches, this task is the first segmentation step in the pipeline [START_REF] Park | A review of speaker diarization: Recent advances with deep learning[END_REF][START_REF] Bredin | Pyannote.Audio: Neural Building Blocks for Speaker Diarization[END_REF]. Overlapping speech appears when several speakers are simultaneously active. These events cause severe performance degradation in speaker diarization [START_REF] Garcia Perera | Speaker Detection in the Wild: Lessons Learned from JSALT 2019[END_REF]. OSD consists of detecting segments that contain at least two simultaneous active speakers. Then, speakers can be assigned to the overlap segments either with a heuristic approach [START_REF] Otterson | Efficient use of overlap information in speaker diarization[END_REF][START_REF] Landini | Analysis of the but diarization system for voxconverse challenge[END_REF] or by using the posterior matrix in case of Variational Bayes (VB) clustering [START_REF] Landini | Analysis of the but diarization system for voxconverse challenge[END_REF][START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF]. OSD leads to significative improvements in speaker diarization [START_REF] Garcia Perera | Speaker Detection in the Wild: Lessons Learned from JSALT 2019[END_REF][START_REF] Landini | Analysis of the but diarization system for voxconverse challenge[END_REF], which makes this task essential for speaker diarization.

VAD and OSD can be processed separately in the diarization pipeline. They can also be extended to speaker counting [START_REF] Stöter | Countnet: Estimating the number of concurrent speakers using supervised learning[END_REF], which detects the number of active speakers in an audio segment. This paper tackles joint VAD and OSD (VAD+OSD). It consists of the detection of non-speech (N spk = 0), speech (N spk = 1) and overlapping speech (N spk ≥ 2) segments with a single model. N spk is the number of active speakers.

In the multi-speaker context, such as meetings, it is often easier to record speech signals using a single distant device instead of asking each participant to carry an individual microphone. Although a distant device offers practical benefits, the captured speech signal is more likely to be corrupted by background noise and reverberation [START_REF] Wölfel | Distant speech recognition[END_REF]. This degradation tends to lower the performance of automatic speech processing systems [START_REF] Maciejewski | Characterizing performance of speaker diarization systems on far-field speech using standard methods[END_REF]. To tackle this issue, it is a common practice to use multi-microphone devices such as microphone arrays. These devices perform a spatial sampling that captures additional information about the spatial distribution of the soundfield [START_REF] Li | Multiple-speaker localization based on direct-path features and likelihood maximization with spatial sparsity regularization[END_REF][START_REF] He | Neural network adaptation and data augmentation for multi-speaker direction-of-arrival estimation[END_REF]. The resulting signal is composed of multiple channels. Some specific algorithms can be applied to take advantage of this spatial information implicitly contained in multi-channel signals [START_REF] Wölfel | Distant speech recognition[END_REF][START_REF] Benesty | Microphone Array Signal Processing[END_REF][START_REF] Benesty | Design of Circular Differential Microphone Arrays[END_REF].

Furthermore, the spatial distribution of the sound field is tightly related to the activity of the speakers [START_REF] Sivasankaran | Localization guided speech separation[END_REF]. The use of multi-channel audio signals, which contain such information, might be beneficial to VAD and OSD. A few works have been conducted towards robust distant VAD and OSD [START_REF] Chen | Multi-channel overlapped speech recognition with location guided speech extraction network[END_REF][START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF][START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF][START_REF] Mariotte | Microphone Array Channel Combination Algorithms for Overlapped Speech Detection[END_REF]. This paper investigates the use of several channel-combination algorithms as joint VAD+OSD front-ends.

A. Related work

Early studies on VAD were based on the extraction of features from the audio signal such as energy [START_REF] Chengalvarayan | Robust energy normalization using speech/nonspeech discriminator for german connected digit recognition[END_REF][START_REF] Woo | Robust voice activity detection algorithm for estimating noise spectrum[END_REF], cross-correlation [START_REF] Kristjansson | Voicing features for robust speech detection[END_REF], zero-crossing rate [START_REF] Ghaemmaghami | Noise robust voice activity detection using features extracted from the time-domain autocorrelation function[END_REF] or linear predictive coding [START_REF] Nemer | Robust voice activity detection using higher-order statistics in the LPC residual domain[END_REF]. Acoustic features have also been used to train statistical models such as Hidden Markov Models (HMM) or Gaussian Mixture Model (GMM) [START_REF] Sarikaya | Robust detection of speech activity in the presence of noise[END_REF][START_REF] Sohn | A statistical model-based voice activity detection[END_REF][START_REF] Pfau | Multispeaker speech activity detection for the ICSI meeting recorder[END_REF][START_REF] Ng | Developing a speech activity detection system for the DARPA RATS program[END_REF]. With the recent advances in supervised learning and deep neural networks, novel approaches have emerged. Early architectures were based on multi-layer perceptron (MLP) [START_REF] Ryant | Speech activity detection on youtube using deep neural networks[END_REF]. More advanced neural models such as Convolutional Neural Networks (CNN) [START_REF] Thomas | Analyzing convolutional neural networks for speech activity detection in mismatched acoustic conditions[END_REF] or Long Short Time Memory (LSTM) recurrent neural networks [START_REF] Gelly | Optimization of RNN-based speech activity detection[END_REF][START_REF] Lavechin | End-to-end domain-adversarial voice activity detection[END_REF] have then been proposed, drastically improving the VAD performance compared to signal-based or statistical approaches.

Similarly to VAD, early research on OSD mostly focuses on HMM/GMM classifiers [START_REF] Boakye | Improved overlapped speech handling for speaker diarization[END_REF][START_REF] Vipperla | Speech overlap detection and attribution using convolutive non-negative sparse coding[END_REF][START_REF] Charlet | Impact of overlapping speech detection on speaker diarization for broadcast news and debates[END_REF]. Some approaches show diarization improvement by detecting and removing overlapping speech segments during the clustering step [START_REF] Boakye | Improved overlapped speech handling for speaker diarization[END_REF][START_REF] Sajjan | Leveraging LSTM models for overlap detection in multi-party meetings[END_REF]. Recent OSD approaches are mostly based on deep neural networks. LSTM neural networks were first applied to OSD in [START_REF] Geiger | Detecting overlapping speech with long short-term memory recurrent neural networks[END_REF] and have become widespread, improving the detection compared to classical machine learning approaches [START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF][START_REF] Sajjan | Leveraging LSTM models for overlap detection in multi-party meetings[END_REF]. Alternatively, some authors use convolutional neural networks [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF][START_REF] Kunešová | Detection of overlapping speech for the purposes of speaker diarization[END_REF][START_REF] Jung | Three-Class Overlapped Speech Detection Using a Convolutional Recurrent Neural Network[END_REF], such as the Temporal Convolutional Network (TCN) [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF][START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF]. In [START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF], Bullock et al. show promising diarization performance improvement by detecting overlapping segments with learnable features and a LSTM-based system. The authors also propose an algorithm to assign the overlapping speech segments to the appropriate speaker, thus improving diarization performance. Most VAD and OSD have a similar architecture and are formulated as binary frame classification tasks. A few studies explore the joint training of both tasks [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF][START_REF] Jung | Three-Class Overlapped Speech Detection Using a Convolutional Recurrent Neural Network[END_REF]. Specifically, Cornell et al. [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF] show that both LSTM-and TCN-based architectures are suitable for joint VAD+OSD and speaker counting.

Research on VAD and OSD is mostly focused on closetalk speech signals, i.e. when the speaker is close to the microphone. However, these conditions are rarely met in the meeting context since it is more convenient to capture the scene with a single distant device than requiring speakers to carry individual microphones. Few studies have been conducted on distant multichannel VAD and OSD. In [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF], the authors propose a TCN-based architecture for distant speaker counting. This work is extended in [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF] by adding spatial features extracted from multi-microphone signals. The authors show a significative performance gain on both VAD and OSD by using spatial features.

Multichannel speech processing concerns various speech processing tasks such as speech recognition [START_REF] Park | Robust multi-channel speech recognition using frequency aligned network[END_REF][START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF][START_REF] Watanabe | CHiME-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings[END_REF], speech separation [START_REF] Yoshioka | Multi-microphone neural speech separation for far-field multi-talker speech recognition[END_REF][START_REF] Gu | Enhancing end-to-end multi-channel speech separation via spatial feature learning[END_REF], and speaker diarization and recognition [START_REF] Watanabe | CHiME-6 challenge: Tackling multispeaker speech recognition for unsegmented recordings[END_REF][START_REF] Taherian | Robust speaker recognition based on single-channel and multi-channel speech enhancement[END_REF]. The spatial information contained in these signals is often exploited through spatial filtering -beamformingwhich consists of weighting and combining channels. Thus, the output of the spatial filter is single-channel but steered in a given angular direction. Early beamforming approaches are mostly based on signal processing methods [START_REF] Benesty | Microphone Array Signal Processing[END_REF][START_REF] Benesty | Design of Circular Differential Microphone Arrays[END_REF][START_REF] Souden | On optimal frequency-domain multichannel linear filtering for noise reduction[END_REF][START_REF] Applebaum | Adaptive arrays[END_REF][START_REF] Anguera | Acoustic beamforming for speaker diarization of meetings[END_REF] while latest algorithms exploit neural networks [START_REF] Park | Robust multi-channel speech recognition using frequency aligned network[END_REF][START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF][START_REF] Li | Neural network adaptive beamforming for robust multichannel speech recognition[END_REF][START_REF] Minhua | Frequency domain multi-channel acoustic modeling for distant speech recognition[END_REF][START_REF] Cornell | Learning filterbanks for end-to-end acoustic beamforming[END_REF]. In [START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF], Gong et al. propose a straightforward way to compute combination-weights based on a self-attention [START_REF] Vaswani | Attention is all you need[END_REF] module and Short-Time Fourier Transform (STFT) magnitude features.

The performance of multi-microphone algorithms tends to highly rely on the array geometry used during training. A few studies have been conducted on array-agnostic or channelnumber-agnostic algorithms in the context of speech separation [START_REF] Taherian | Robust speaker recognition based on single-channel and multi-channel speech enhancement[END_REF][START_REF] Luo | End-to-end microphone permutation and number invariant multi-channel speech separation[END_REF]. Both approaches improve the robustness of the system to array mismatch but require more model parameters to train.

B. Contributions

Building on these previous works, this paper proposes several channel-combination algorithms as joint VAD and OSD (VAD+OSD) front-ends. These channel combination algorithms are based on the Self Attention Channel Combinator (SACC) [START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF]. The main motivation behind this work is to include the phase information in the combination-weight estimation process. A first extension is proposed by replacing the STFT with a learnable analytic filter bank [START_REF] Pariente | Filterbank design for end-to-end speech separation[END_REF] in the original SACC algorithm. In a second approach, following the work conducted in [START_REF] Mariotte | Microphone Array Channel Combination Algorithms for Overlapped Speech Detection[END_REF], we explore two architectures in the complex domain: EcSACC and IcSACC. The former processes the STFT parts separately but doubles the number of parameters. The latter uses a single self-attention module to process both parts. These extensions show similar performance as state-ofthe-art approaches while being more explainable by design. The explainability features offered by complex extensions are demonstrated by the visualization of the beampattern. This leads to a better understanding of the use of spatial information in such systems. The impact of the proposed segmentation algorithms is evaluated on the speaker diarization task using the VBx system [START_REF] Landini | Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks[END_REF]. Results show that the proposed extensions offer competitive performance considering standard beamforming and SACC. Finally, systems trained on multichannel data often suffer performance degradation in case of a mismatch in the array geometry between training and evaluation sets. We propose a channel-number invariant loss to facilitate the model generalization under mismatched array configuration. To the best of our knowledge, no similar training procedure has yet been applied to VAD and OSD. The source code will be made available in a large-scale diarization toolkit to be published soon.

The paper is organized as follows. Section II formulates the joint VAD+OSD task. Section III introduces the proposed algorithms. The systems and baselines are described in Section IV, followed by the experimental protocol detailed in Section V. Section VI evaluates the VAD and OSD results while speaker diarization results are described in Section VII. The analysis of the spatial information used by complex systems is conducted in Section VIII. Finally, the channel-number invariant loss is introduced and evaluated in Section IX before drawing conclusions and perspectives in Section X.

II. VOICE ACTIVITY AND OVERLAPPED SPEECH DETECTION

This paper focuses on two speech segmentation sub-tasks, namely Voice Activity Detection (VAD) and Overlapped Speech Detection (OSD). The former aims to detect speech segments in the signal, while the latter aims to find segments in which at least two speakers are simultaneously active. VAD and OSD can be solved within the same framework, which is detailed in the following sections.

A. General formulation

Let X = [X 0 , . . . X t , . . . X T -1 ] ∈ R F ×T be a sequence of feature vectors where F is the number of features, T the number of time frames and t the time frame index. This sequence is extracted from the raw audio signal x ∈ R C×N , with C being the number of channels (i.e. microphones) and N the number of samples. Feature extraction can be defined as a function g : x → X, which maps the raw input signal to a sequence of feature vectors X ∈ R F ×T . The g function might be handcrafted or trained end-to-end with the sequence-modeling model described below. As proposed in [START_REF] Bredin | Pyannote.Audio: Neural Building Blocks for Speaker Diarization[END_REF][START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF], the sequence of features verifies T < N to reduce the computational cost. Note that the g function depends on the number of channels in the input signal x. The feature extraction differs between the single-channel, C = 1, and the multi-channel scenario, C > 1.

Let y = [y 0 , . . . y t , . . . y T -1 ] ∈ R T be a sequence of aligned reference binary labels. VAD and OSD are solved by optimizing the parameters θ of a model f : X, θ → ŷ which maps the feature sequence to a sequence of predicted labels ŷ = [ŷ 0 , . . . ŷt , . . . ŷT -1 ] ∈ R T .

B. Labelling procedure

VAD and OSD are solved jointly following a 3-class classification problem following [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF][START_REF] Jung | Three-Class Overlapped Speech Detection Using a Convolutional Recurrent Neural Network[END_REF]. The neural model is trained to predict a sequence ŷ from the raw input signal x based on the manual annotations y.

To solve VAD and OSD tasks, the frame-level annotation verifies y t ∈ {0, 1, 2}. y t = 0 refers to a non-speech frame (N spk = 0). y t = 1 indicates that the t-th frame contains speech from a single active speaker (N spk = 1). y t = 2 refers to a frame containing at least two simultaneously active speakers (N spk ≥ 2).

Since overlapping speech is a rare event, the classes are unbalanced [START_REF] Lebourdais | Overlaps and gender analysis in the context of broadcast media[END_REF]. The class balance can be improved by artificially generating additional overlapped data by combining single-speaker utterances from other datasets [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF][START_REF] Kunešová | Detection of overlapping speech for the purposes of speaker diarization[END_REF] or random segments of the training data at training time [START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF]. Here, we consider the second data augmentation strategy. and multiple active speakers p(N spk ≥ 2|x). VAD prediction is the combination of the N spk = 1 and the N spk ≥ 2 outputs, while OSD prediction is determined from the N spk ≥ 2 output only.

III. MULTI-CHANNEL FEATURE EXTRACTION

In the meeting context, capturing distant speech signals offers practical benefits by preventing the participants from carrying individual microphones. Distant speech signals however, tend to degrade the performance of automatic speech processing systems [START_REF] Maciejewski | Characterizing performance of speaker diarization systems on far-field speech using standard methods[END_REF]. We investigate several multi-channel frontends to tackle distant VAD and OSD. The proposed approaches consist of weighting and combining the incoming channels to generate an enhanced single-channel feature sequence. All the implemented algorithms are inspired by the Self Attention Channel Combinator (SACC) [START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF]. Figure 2 depicts the global approach for multi-channel feature extraction. Specific weight estimation algorithms are presented in the following sections.

A. Self Attention Channel Combinator

The Self-Attention Channel Combinator (SACC) [START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF] was developed in the context of multi-channel speech recognition as an alternative to neural network-based beamforming [START_REF] Li | Neural network adaptive beamforming for robust multichannel speech recognition[END_REF]. This approach consists of estimating time-varying weights, referred to as combination weights, to weight and to combine channels captured by a set of microphones. The weight estimation is performed in the Short-Time Fourier Transform (STFT) domain using a self-attention module [START_REF] Vaswani | Attention is all you need[END_REF]. The model is thus supposed to focus attention on the microphones containing relevant information for the back-end task.

Let Y ∈ C C×T ×K be the multichannel STFT of the input signal x, with K being the number of frequency bins. The SACC algorithm computes combination-weights, w ∈ R C×T ×1 , from the magnitude of the multichannel STFT using self-attention [START_REF] Vaswani | Attention is all you need[END_REF]. Let q, k, and v be three linear transformations, mapping the STFT log-magnitude log(∥Y∥) to the query and the key Q, K ∈ R C×T ×D , and the value V ∈ R C×T ×1 . The combination weights are determined as follows:

w = softmax softmax QK T √ D V , (1)
with D being the output space dimension of the query Q and the key K. Note that the transpose operator . T is applied framewise. Each C × T matrix is transposed for each time frame. The last softmax activation applies to the channel dimension and constrains the weights to be within the interval [0, 1]. Mean and Variance Normalization (MVN) is applied to each frequency bin before feeding the self-attention module. MVN reduces the data variation range to make combinationweights learning easier. The combined time-frequency representation Y att ∈ R 1×T ×K is finally obtained as the weighted sum of the different channels:

Y att = C c=1 w ⊙ ∥Y∥, (2)
with ⊙ being the element-wise product. The combined STFT, Y att , is finally converted to the log-mel scale using F mel scale filters to obtain the feature vector X ∈ R T ×F . The process of the SACC algorithm is presented in Figure 3(a). This model is referred to as SACC+ST F T .

B. Analytic SACC

In the original SACC implementation [START_REF] Gong | Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-Field Speech Recognition[END_REF], the weights are computed from the STFT magnitude. The phase information contained in the STFT is lost at the beginning of the process. This information might however be relevant for the weights estimation process since it informs on the time delay between channels. Including the phase-related information may improve the weight estimation procedure.

To implicitly include phase information during weight estimation, we propose to replace the STFT with a learnable filter bank. The input signal is decomposed into several frequency bands with a time domain filter bank. Filters are implemented as a set of mono-dimensional convolutional layers optimized in an end-to-end manner with the back-end task, i.e. VAD and OSD. Learning consistent filters from free convolutional layers may however be difficult when dealing with high-order filters. To counteract this limitation, learnable analytical filters have been proposed in the context of speech separation [START_REF] Pariente | Filterbank design for end-to-end speech separation[END_REF].

Let h(n) ∈ R L be the impulse response of a real filter with n being the sample index and L its length. The analytical version of this filter can be computed through the Hilbert transform H as follows:

h analytic (n) = h(n) + jH{h(n)}, (3)
where j = √ -1. In practice, the real and imaginary parts of h analytic (n) are concatenated before computing the combination weights.

In the analytical version of the SACC model, the STFT is replaced by a set of analytical filters. The filter bank is shared across channels to reduce the number of trainable parameters. The output STFT-like representation is then forwarded through the self-attention module as depicted in figure 3(b). This model is referred to as SACC+A.

C. Complex SACC

Another approach to take the phase information into account is to integrate the phase of the STFT in the combination-weight estimation procedure. The combination weights thus belong to the complex space, i.e. w ∈ C C×T ×1 . This approach was proposed in [START_REF] Mariotte | Microphone Array Channel Combination Algorithms for Overlapped Speech Detection[END_REF] to learn combination weights that are closer to the conventional beamforming techniques. Furthermore, this formulation allows us to better interpret the weights learned by the model as shown in section VIII.

This paper investigates two formulations to estimate the complex combination weights. The first computes the magnitude and the phase of the weights separately using two distinct self-attention modules applied to the magnitude (Mag) and the phase (ϕ) of the STFT respectively. The second estimates the complex weights using a single self-attention module applied to the concatenation of the magnitude and the phase. These approaches are respectively referred to as Explicit complex SACC (EcSACC) and Implicit complex SACC (IcSACC).

Magnitude + MVN Combination weights Multichannel STFT V K Q Scaled dot prod. Softmax Softmax (a) Softmax Softmax V K Q Scaled dot prod. Softmax V K Q Scaled dot prod.
1) Explicit cSACC: EcSACC consists of computing the magnitude and the phase of the combination weights separately. Each component of these weights is thus computed explicitly. The system flowchart is presented in figure 3(c). Let ∥Y∥ and ∠Y be the magnitude and the phase of the multichannel STFT. The weights associated with each part of the STFT are computed as follows:

w M ag = SA M ag (∥Y∥) , w ϕ = SA ϕ (∠Y) , (4)
where SA M ag and SA ϕ represent the self-attention modules defined in equation ( 1) applied to the magnitude and the phase respectively. The magnitude w M ag and phase w ϕ of the weights are then applied to the associated parts of the multichannel STFT:

Y att = (w M ag ⊙ ∥Y∥) e j(2πw ϕ +∠Y) . (5)
The phase weights verify w ϕ ∈ [0, 1] with the softmax normalization in Eq.( 1). The 2π factor ensures that the phase correction covers the entire unit circle. Note that there is no weight sharing between the two self-attention modules to enable the model to learn two distinct representations. The same weight-estimation procedure can be applied to the real (R) and the imaginary (I) parts of the STFT.

2) Implicit cSACC: EcSACC processes the magnitude and the phase separately, which prevents the model from learning a relationship between the two representations. Furthermore, it requires learning two self-attention modules which doubles the number of trainable parameters in the feature extraction. The IcSACC formulation can learn cross-relationships between each part of the STFT while keeping the same number of trainable parameters as SACC. The IcSACC approach is presented in figure 3(d). It consists of a single self-attention module that estimates the combination weights from the concatenation of the magnitude and the phase of the STFT:

w = SA (cat(∥Y∥, ∠Y)) , (6)
where cat(•, •) represents the concatenation operator. The two STFT parts are concatenated over the channel dimension. The resulting vector w contains the concatenation of the magnitude and the phase of the weights. The channel combination is then performed following equation [START_REF] Garcia Perera | Speaker Detection in the Wild: Lessons Learned from JSALT 2019[END_REF].

IV. SYSTEMS AND BASELINES

This section describes the baseline systems along with the hyperparameters used for each architecture.

A. Feature extraction configuration SACC-based models are compared to two single-channel baselines. The first is trained and evaluated on the close talk data captured by AMI headset-mix while the second is trained and evaluated on the first channel of Array 1 (single distant microphone). The close-talk baseline features an upper-bound score, considering that distant performance tends to be degraded [START_REF] Maciejewski | Characterizing performance of speaker diarization systems on far-field speech using standard methods[END_REF]. A multi-channel baseline, based on Minimum Variance Distortion-less (MVDR) beamforming, is also considered. The following sections describe the feature extraction procedure for each system. The length of the feature vector has been optimized separately for each feature extractor. Thus, the number of features F differs from one system to another.

1) Single channel:

The Mel Frequency Cepstral Coefficients (MFCC) are used as single-channel features. They have been extensively used for VAD and OSD in the literature [START_REF] Bredin | Pyannote.Audio: Neural Building Blocks for Speaker Diarization[END_REF][START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF]. In this study, 20 MFCC are extracted from the raw audio signal using a 25ms window with a 10ms shift. The first ∆ and second ∆∆ derivatives are also computed. The first coefficient is removed resulting in a feature vector composed of F = 59 coefficients.

2) MVDR Beamformer: To compare SACC-based approaches to standard beamforming algorithms, the Minimum Variance Distortion-less (MVDR) beamformer proposed in [START_REF] Souden | On optimal frequency-domain multichannel linear filtering for noise reduction[END_REF] is also implemented. This method requires estimating the noise covariance matrix. Since we are dealing with realistic speech conditions, the clean speech of the source is unknown. We use the Coherent-to-Diffuse Ratio (CDR) under diffuse noise and unknown DOA conditions -see [START_REF] Schwarz | Coherent-to-Diffuse Power Ratio Estimation for Dereverberation[END_REF] Eq. ( 25)to estimate the time-frequency masks. The noise covariance matrix is then estimated from the masked signal. Beamformer weight estimation and channel combination are performed in the STFT domain. Similarly to STFT-based SACC, the resulting spectrogram is converted to the Mel scale using F = 64 filters.

3) Self-Attention Channel Combinators: Several multichannel front-ends are considered in this study. The STFTbased features -SACC, EcSACC, and IcSACC -are extracted using 25ms windows with a 10ms shift. For each SACC variant, the hidden size of the self-attention module is set to D = 256. The output spectrograms of SACC+ST F T , EcSACC, and IcSACC are converted to the Mel scale with F = 64 filters. In the SACC+A model, features are extracted with 32 analytic filters. The concatenation of the real and imaginary parts of the filter's output results in a vector of F = 64 features. The kernel of the convolution layer is designed to fit the STFT parameters with a window size of 25ms and a shift of 10ms.

B. Sequence modeling architecture

The sequence of features extracted from the raw audio signal is processed by a sequence modeling network to predict VAD+OSD labels. To better evaluate the impact of each feature extractor on each task performance, two sequence modeling architectures are considered. The first is based on Bidirectional Long Short-Term Memory (BLSTM) while the second is a Temporal Convolutional Network (TCN). Architecture details are presented in the following sub-sections.

1) BLSTM: The former sequence modeling architecture composed of two stacked BLSTM layers is composed of 256 cells similar to [START_REF] Bredin | Pyannote.Audio: Neural Building Blocks for Speaker Diarization[END_REF][START_REF] Bullock | Overlap-Aware Diarization: Resegmentation Using Neural End-to-End Overlapped Speech Detection[END_REF]. The output sequence is post-processed using a three-layer feed-forward network (FFN) with output sizes L 1 = 128, L 2 = 128, and L 3 = 2 respectively. The last layer outputs the pseudo-probability of each class. FFN layers are followed by a tanh activation function except for the last one. A softmax activation is applied to the output logits to compute the classification scores.

2) TCN: The Temporal Convolutional Network (TCN) has been proposed as an alternative to BLSTM for VAD and OSD [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF]. This architecture is composed of causal convolutional layers with residual connections. The TCN uses dilated convolutions to benefit from a large temporal context. Our system features the same TCN architecture as [START_REF] Cornell | Detecting and Counting Overlapping Speakers in Distant Speech Scenarios[END_REF]. A layer normalization step is first applied, followed by a 1Dconvolution bottleneck layer which projects the sequence of features from F dimensions to 64 dimensions. The bottleneck output is then processed by 5 1-D convolution layers of 128 hidden channels, with exponentially increasing dilatation. This block is repeated 3 times. A residual connexion is added at the output of each block. Finally, a 1-D convolutional layer projects the hidden sequence to the classification space (3 dimensions).

V. EXPERIMENTAL PROTOCOL Our experiments aim to evaluate how each multi-channel front end impacts VAD and OSD performance. This section introduces the dataset and the evaluation protocol.

A. Dataset

Experiments are conducted on the AMI meeting corpus [START_REF] Mccowan | The AMI meeting corpus[END_REF]. This dataset features about 100 hours of meetings recorded under realistic conditions. Audio signals have been captured using different devices. In this work, the signals recorded by headset microphones, referred to as headset-mix, are considered as a close-talk reference. Multichannel signals acquired by the 8-microphone array placed in the center of the table referred to as Array 1, are used as distant speech material. This device consists of a uniform circular array (UCA), with r = 0.1m radius. Distant audio signals are also captured by a 4-microphone array, referred to as Array 2. Both 8and 4-microphone arrays are considered to evaluate channelnumber invariant systems (section IX). This last device is either circular or linear depending on the meeting session.

Framewise VAD+OSD labels are extracted from the speaker diarization annotation provided in the AMI dataset. Overlapping speech labels are obtained where two annotated segments overlap. The data is divided into training, development, and test sets based on the protocol proposed in [START_REF] Landini | Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks[END_REF], which guarantees different speakers in each set. 10% of the training set is reserved for validation to determine the best-performing system. To reduce the computational cost, the label sequence is sampled at 100Hz, while the audio signal has a 16kHz sampling rate. The training stops after five epochs if there is no improvement in the target validation metric (F1-score). Since VAD+OSD is formulated as a 3-class classification problem, cross-entropy L CE is used as a training objective. A channelnumber invariant loss L inv can also be added as detailed in section IX. Model optimization is performed using the ADAM optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF] with a 10 -3 learning rate. The validation is conducted on batches of 64 segments from the validation set. The best model is selected as the one obtaining the higher F1-score on the validation set.

The evaluation is conducted on the development (Dev) and evaluation (Eval) sets of the AMI meeting corpus. The inference is performed on a 2s sliding window with a 0.5s shift. Predictions are averaged on the overlapped part of the sliding windows. For each frame, the predicted class is determined by finding the argmax among the pseudo-probabilities. The detected segments are saved in the Rich Transcription Time Marked (RTTM) format which allows the evaluation of VAD and OSD with the reference segmentation. VAD is evaluated using the false alarm rate (FA), the miss detection rate (Miss), and the sum of these two metrics: the Segmentation Error Rate (SER). OSD is evaluated using precision, recall, and F1-score.

VI. SEGMENTATION RESULTS

This section presents the VAD and OSD performance of both BLSTM and TCN architectures with each front end. Results are presented on the development and evaluation sets of the AMI corpus.

A. BLSTM results

Table I presents the VAD and OSD performance of the 3-class BLSTM system. The close-talk model reaches an OSD F1-score of 70.1% and 69.2% on Dev and Eval sets, respectively. This model achieves a VAD SER of 6.66% and 5.94% on these two subsets. This score is the upper bound baseline with the BLSTM sequence modeling. Single-channel distant speech degrades the performance, with 64.0% and 62.5% OSD F1-score. VAD performance is also lower, e.g. 7.06% on the evaluation set. This shows how distant speech conditions degrade the segmentation. The SDM acts as a lower-bound reference.

Considering multichannel processing such as MVDR drastically improves distant OSD with an F1-score of 66.9% on the Eval. VAD also improves with a 6.55% evaluation SER. The original SACC model shows better performance, with a 67.4% OSD F1-score and a 6.40% VAD SER on the evaluation subset. It offers the best VAD and OSD performance on the evaluation set with the BLSTM architecture.

Among the proposed models, the SACC+A degrades the detection performance on both OSD (64.5% F1-score Eval) and VAD (6.71% SER Eval) compared to SACC. The performance however remains higher than the SDM scenario. IcSACC shows similar OSD performance as A on the development The VAD performance of this system is slightly better than the SDM with 6.73% on the evaluation set. The EcSACC formulation shows similar OSD performance as MVDR, with 68.0% on Dev and 66.5% on Eval. The VAD scores remain degraded with 6.80% on the evaluation set.

B. TCN results

Table III shows each front-end performance on VAD and OSD by considering the TCN architecture. The close-talk model offers a 74.0% F1-score on the evaluation set. Considering a TCN instead of a BLSTM improves OSD by an absolute +4.8% on this subset. VAD SER also improves from 6.66% to 5.42% on the development set. The SDM system degrades OSD (64.9% Eval F1-score) and VAD (6.81% Eval SER) because of distant recording conditions.

MVDR offers remarkable OSD performance with 72.3% and 69.6% F1-score on development and evaluation sets respectively. This model also shows the best distant VAD performance (6.32% Eval SER). The SACC model offers similar OSD performance with 71.8% and 68.5% F1-score on the two subsets but highlights a lack of robustness on VAD with 6.73% on the evaluation set for the SER.

Like the BLSTM, SACC+A shows limited OSD capacities with 64.6% F1-score on the evaluation set. However, the VAD improves concerning the SDM and the original SACC (6.58% Eval SER). IcSACC gets OSD performance close to MVDR and SACC on the Dev set (71.5%). This model shows limited generalization capacities, with a 67.6% F1-score on the evaluation set. This degradation is mainly due to a drop in the recall. The EcSACC model shows similar OSD performance as SACC on both OSD (68.4% Eval F1-sore) and VAD (6.79% SER Eval).

The original SACC and its complex extensions reach competitive performance with the MVDR beamformer. The use of self-attentive methods remains justified since they do not require additional computations such as covariance matrix estimation.

While these two models do not significantly improve OSD compared to MVDR or SACC, they achieve similar performance, mainly on OSD. EcSACC and IcSACC improve speaker diarization under distant speech conditions and show strong explainability features, as shown in the next sections.

C. Performance comparison

This section compares the VAD and OSD performance of the three best TCN-based systems -SACC+STFT, IcSACC, and EcSACC-with the best model from Cornell et al. [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF]. It consists of a VAD+OSD TCN model fed with log-scale Mel spectrogram and Cosine and Sine encoded Interaural Phase Difference (CSIPD) features. The systems are compared in terms of Average Precision (AP) metric, as used in [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF]. Table II presents the results obtained on the evaluation set from the AMI corpus. It shows that our systems improve the VAD performance, as IcSACC and SACC+ST F T reach 99.3% AP where [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF] reaches 98.7%. Our systems also improve the OSD performance. Specifically, SACC+ST F T shows the best performance with 71.4% whereas the system from [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF] reaches 60.4%.

Note that the AMI protocol has been ambiguous in the context of speaker diarization, mostly due to the several versions proposed in the original paper [START_REF] Mccowan | The AMI meeting corpus[END_REF]. The work from Landini et al. [START_REF] Landini | Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks[END_REF] proposes a new protocol for speaker diarization which meets consensus in the community. As stated in section V, we used this protocol in this work. However, Cornell et al. [START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF] use a different protocol. Thus, the results are given for information only, and are not directly comparable with ours.

A. Protocol

The impact of each VAD+OSD model is evaluated on the final back-end task, i.e. speaker diarization. A first diarization is obtained using the VBx system [START_REF] Landini | Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: Theory, implementation and analysis on standard tasks[END_REF]. This system uses a ResNet101 x-vector extractor followed by a VB-HMM clustering algorithm. The VAD segments, predicted by our systems, are used as an initial segmentation. X-vector clustering is initialized with Hierarchical Agglomerative Clustering (HAC) before performing VB clustering.

A second diarization is then obtained by assigning overlapping speech segments. These segments are predicted by our OSD system. The assignation is performed using the heuristic approach from [START_REF] Otterson | Efficient use of overlap information in speaker diarization[END_REF]. VBx speaker diarization and overlap assignment are performed using the online code available 1 .

The VAD and OSD segmentations are obtained with the TCN-based systems. The diarization is evaluated with the Diarization Error Rate (DER) and the Jacquard Error Rate (JER) following the NIST evaluation protocols [START_REF] Park | A review of speaker diarization: Recent advances with deep learning[END_REF]. The DER is computed both with a 25ms collar (δ c = 25) and without (δ c = 0).

B. Impact of the VAD+OSD on speaker diarization

Speaker diarization results are presented in table IV. The Oracle performance is obtained using the reference VAD and OSD labels. It represents the best score that can be expected with our diarization system, i.e., with the same segmentation as the annotation. For each system, the speaker diarization results are presented with VAD only (VAD) and overlapping speech assignment (VAD w/ OSD). Unless otherwise specified, we present DER performance in the δ c = 0 evaluation setup, and the error between scores is relative.

The Oracle results show that the minimum DER that can be expected is 22.47% with VAD only and 14.32% by considering OSD. Assigning the overlapping speech segments improves the diarization by +36.3%, demonstrating that OSD is required for robust speaker diarization.

Among VAD+OSD systems, the SDM delivers the worst diarization performance, with 27.45% DER with VAD only and 25.01% with overlap assignments. The performance gain due to OSD is reduced compared to Oracle with +8.9% only. The OSD errors have a strong impact on the speaker diarization performance.

Channel-combination front-ends have been shown to improve both VAD and OSD and improve speaker diarization. SACC+A offers the highest DER among these systems with 23.97%. MVDR shows a 26.80% and a 23.71% DER with and without overlap assignment. It reaches one of the best JER (30.87%), highlighting a better intra-speaker segmentation. SACC+ST F T achieves the lower DER with no collar in both scenarios (VAD: 26.06%, VAD w/ OSD: 23.09%).

Among the complex extensions, IcSACC achieves the best DER with a 25ms collar (15.87%). EcSACC offers the best 1 https://github.com/BUTSpeechFIT/VBx/tree/v1.1 VoxConverse2020 accessed on October 21st, 2023 diarization performance, as it reaches a similar JER as MVDR (30.90%) and a DER close to SACC+ST F T (23.30%). This model exhibits a +11.6% improvement between the VAD and the VAD w/ OSD scenarios.

The results obtained with SACC+ST F T and EcSACC also outperform both models from [START_REF] Raj | Gpu-accelerated guided source separation for meeting transcription[END_REF]. In this paper, the authors report a DER of 25.05% with a VBx-based system and 23.69% with a spectral clustering-based method. For instance, the EcSACC system improves diarization of +7.0% and +1.7% respectively.

The experiments conducted on speaker diarization show that the performance gain in VAD+OSD leads to a gain in terms of DER and JER. Thus, using channel combination approaches for VAD+OSD is also beneficial for the final task. Complex models such as EcSACC and IcSACC achieve among the best diarization performance.

While showing competitive performance with MVDR and the original SACC, complex models offer new perspectives in terms of explainability. The analysis of this system is presented in the next section.

VIII. COMPLEX WEIGHTS ANALYSIS FOR EXPLAINABILITY

In Section III, we introduced two extensions of the Self-Attention Channel Combinator to compute complex weights from the STFT. This section proposes to apply a standard spatial filter analysis tool -beampattern -to show the explainability capacities of such systems.

A. Beampattern as an analysis tool

Similarly to beamforming, the complex combination weights allow the system to steer in a given angular direction. Each complex SACC approach can be considered as a spatial filter. The spatial response of such a filter can be visualized using the beampattern [START_REF] Benesty | Design of Circular Differential Microphone Arrays[END_REF]. This representation informs on the system response to a plane wave impinging in a θ angular direction, given an array geometry. Note that this representation can only be computed on complex combination weights. Visualizing the magnitude of the beampattern for a set of combination weights indicates the spatial direction in which the self-attention module is focused. Considering a UCA and a set of learned combination-weights w t , the narrow-band beampattern can be computed as follows:

B t [w t , ω, θ] = C c=1 w c,t e j ω cos(θ-ψc) , ( 7
)
where t is the frame index, c the microphone index, ψ c the angle of the c-th microphone in the array frame of reference, and ω = 2πrf /v s with f being the frequency of the impinging plane wave, r the radius of UCA and v s the speed of sound. The broadband beampattern can then be computed for a given set of frequencies

f = {f 0 , f 1 , • • • f sup }, where f sup < C.vs
4πr is the maximum frequency to avoid spatial aliasing [START_REF] Benesty | Design of Circular Differential Microphone Arrays[END_REF]. The broadband beampattern is computed as follows: The EcSACC and IcSACC combination weights are timevariant. The beampattern also varies as a function of time. The time average beampattern can be computed to visualize the average steering directions of the system on a given utterance:

B t [w t , θ] = B t [w t , ω0 , θ], • • • , B t [w t , ωsup , θ] . (8)
B[w t , ω, θ] = 1 T T -1 t=0 B t [w t , ω, θ], (9)
with T being the total number of time frames. The following sections demonstrate the visualization capacities provided by the beampattern for the combination-weights analysis.

B. Time averaged beampattern vs. acoustic energy map

This section evaluates the steering directions of both Ec-SACC and IcSACC on the AMI development set. Since the speaker locations are unknown, the time-averaged beampattern-Eq. ( 4)-is compared to the SRP-PHAT [START_REF] Benesty | Microphone Array Signal Processing[END_REF] energy map. A position corresponding to a maximum of energy is thought to represent an active acoustic source. The SRP energy map is compared to the time-averaged beampattern of each complex SACC combination weight. For visualization purposes, the time-averaged beampatterns are normalized between 0 and 1. Figure 4 (a)-(d) presents the time-averaged beampatterns of the IcSACC model on 1-second audio segments of the IS1008a session from the AMI development set. It shows that the main lobe of the beampattern is mostly steered towards the maximum of the energy map. The main lobe also exhibits significant shape variations. In some examples -figures 4(a) and 4(c) -the steering directions are correlated with the source position. When two sources are active (fig. 4(c)), the model seems to steer towards the directions of both sources. This model however fails to steer towards the source at 315 • as shown in figure 4(d).

Figure 4 (e)-(h) presents the time-averaged beampatterns computed on the EcSACC combination weights with the same segments. The beampatterns exhibit a similar structure as IcSACC with a match between the energy maxima and the main lobe. In both single--figures 4(a) to (c)-and dualsources -figure 4(c)-scenarios, the beampattern correlates with the energy maxima. On the last segment, the EcSACC model better selects the 315 • source direction as shown in figure 4(h).

Finally, this section shows that the beampattern is an efficient tool to better interpret the combination weights learned by IcSACC and EcSACC. This representation highlights the steering directions used by the self-attention modules with a physical interpretation. The beampattern exhibits a dependence between the combination weights and the source location. Based on the AMI data, the model appears to mostly steer toward the acoustic energy maxima. However, we lack an established protocol to evaluate the steering direction of such systems on a larger scale. This will be part of a future study.

IX. INVARIANCE TO THE NUMBER OF MICROPHONES

The previous sections have shown that multichannel frontend algorithms can be beneficial for distant VAD and OSD. However, these systems are trained with a fixed array configuration available in the training data. Under real meeting conditions, the number of available microphones may vary from one session to another. This section proposes a training strategy to enforce the system to generate a unique feature map regardless of the number of microphones provided as input. The invariant training is evaluated on the AMI meeting corpus. This dataset features audio signals captured by two types of arrays located in different positions in the meeting room and allows an evaluation in case of a mismatched array.

A. Channel-number invariant training

When a VAD+OSD system is trained with a given microphone array configuration, the performance might be degraded in case of an array mismatch in the testing conditions. The mismatch can be the result of a change in the microphone array geometry, or a set of microphones stopping to work. To train a robust system regardless of the number of microphones, the sequence of feature vectors can be enforced to be invariant to the number of channels. Therefore, we propose an invariant training loss to learn a unique representation of the input signal regardless of the number of active microphones. The VAD+OSD system is then trained to perform classification and representation invariance in a multi-task formulation jointly. As detailed in section II, VAD+OSD is formulated as a frame classification task and is optimized using a cross-entropy (CE) training objective L CE . An additional training objective, referred to as invariant training loss L inv , is proposed to learn a unique feature map. The channel-number invariant training is illustrated in figure 5 and detailed below.

Let g : x C , Θ → X C be a multi-channel neural feature extractor which maps the raw input signal x C to a feature map X C . C indicates the total number of microphones available in the training data. The channel-number invariant training determines a set of parameters Θ to extract a similar feature vector regardless of the number of active channels. The training procedure is defined as follows and illustrated in figure 5. The input audio segment x C is duplicated P times. Some channels are then randomly masked in the input and the duplicated segments. The number of kept active channels C p is randomly sampled following a uniform distribution C p ∼ U {2,C} . The duplicated masked-segments are collected in a set D = {x p Cp } P p=1 . For each segment in D, the feature map X p Cp can be computed by feeding the segment x p Cp to the function g. We define a channel-number invariant loss, L inv which forces the masked feature sequence X p Cp to be as close as the reference feature sequence X C . This loss minimizes the distance between the reference feature map X C and each masked representation X p Cp given the following expression:

L inv = 1 P P p=1 ∥X C -X p Cp ∥ F ∥X C ∥ F • ∥X p Cp ∥ F , (10)
with ∥ • ∥ F being the Frobenius norm. The training objective defined in equation ( 10) aims at generating a unique representation regardless of the number of active channels. The segmentation model is then trained using the following dual loss:

L = λL CE + (1 -λ)L inv , (11)
where λ ∈ [0, 1] is a trade-off parameter between the two training objectives.

B. Training and evaluation

The use of invariant training is investigated with our bestperforming VAD+OSD system which is the TCN architecture. Four SACC algorithms are investigated as multi-channel frontends: SACC+ST F T , SACC+A, IcSACC, and EcSACC.

The invariant system is trained with audio material from Array 1 of the AMI meeting corpus. The invariant loss L inv is computed by duplicating P = 2 times the current training segment. The trade-off parameter between the two loss functions is set to λ = 0.7 since it produces the best OSD performance.

To evaluate the robustness of invariant models, a first evaluation is conducted on signals recorded by the Array 1 by removing a fixed number of channels. The system is also evaluated using a new array configuration, which the system did not observe during training. This evaluation is conducted on the data recorded by the Array 2 of the AMI corpus. It consists of a 4-microphone uniform circular or linear array depending on the session. This array is placed on one side of the table, a little further from the participants than the Array 1. Only OSD performance is presented since it is more representative of the impact of the invariant training. The performance on both arrays is presented in Table V.

C. Evaluation on the AMI Array 1

This subsection evaluates the impact of the invariant training on VAD+OSD systems. The evaluation considers C = 8, C = 4, and C = 2 active channels in the AMI Array 1 evaluation data. The first part of table V shows OSD performance for each configuration.

In the C = 8 scenario, each kind of system offers the same performance whether the invariant training is considered or not. For example, SACC+ST F T shows a 68.5% F1-score with the original model and 68.4% with the invariant training. Only the EcSACC system degrades by an absolute -1.3% in the invariant training context.

In the C = 4 scenario, the performance of the original models degrades with the channel masking. The SACC+ST F T system degrades by -8.5% between the 8-and the 4microphone configurations. The IcSACC and EcSACC models are strongly degraded. These systems are not able to detect To better understand the loss of information of the original system with missing microphones, Figure 6 displays the prediction of the EcSACC system on a segment in the C = 8 and the C = 4 scenarios by considering invariant training or not. It shows that the dynamic of the prediction is altered by the loss of microphone pairs and becomes close to zero. This illustrates the F1-score to be 0.0%. Meanwhile, invariant training maintains good prediction in both scenarios.

D. Evaluation on the AMI Array 2

In the following experiment, each system is evaluated on AMI array 2 data to assess the OSD performance on actual mismatch array configuration. The OSD performance is presented in the second part of table V. In an ideal invariant system, the OSD performance remains consistent from one array to another. The OSD results show that the original models poorly perform under array mismatch. For example, the SACC+ST F T model reaches a 53.2% F1-score while it gets 68.5% on Array 1 with all the channels. The complex models are unable to detect overlapping speech. The SACC+A 32 system is also degraded (58.9%) considering the Array 1 performance (64.6%).

Adding the invariant training drastically improves the robustness of most systems. The SACC+ST F T features a +10.2% absolute improvement considering the original model with a 63.4% F1-score. Both IcSACC and IcSACC can detect overlapping speech with 59.6% and 56.5% F1-score respectively. Only the SACC+A 32 system shows similar performance with 57.6%. The experiments on Array 1 data show that this model highlights some robustness to channel masking. This is confirmed under mismatched array conditions.

In this section, we have proposed a new training procedure to learn a similar feature sequence regardless of the number of input microphones. The feature sequence invariance is learned at the same time as classification in the multi-task learning framework. This section shows that SACC-based VAD+OSD performance tends to be degraded under array configuration mismatch. To tackle this issue, a channel-number invariant loss L inv is proposed. AMI Array 1 experiments confirm the suitability of the invariant training framework for OSD, mostly with IcSACC and EcSACC. Under invariant training conditions, the systems can keep the OSD performance close to the C = 8 scenario, i.e. when all the channels are active. On Array 2 data, the performance is slightly degraded, but still improved concerning the original models.

X. CONCLUSIONS AND PERSPECTIVES

This paper presents several multi-channel front-ends for joint distant Voice Activity and Overlapped Speech Detection (VAD+OSD) for speaker diarization. These algorithms are all based on the same principle: weighting and combining the channels coming from a microphone array. Three combinationweight estimation procedures were proposed. Those are all inspired by the Self-Attention Channel Combinator (SACC), which estimates combination weights from the multi-channel short-time Fourier transform (STFT) magnitude.

In the first instance, the STFT is replaced with a learnable filter bank based on analytical filters. The other two methods exploit the magnitude and phase of the STFT in explicit (EcSACC) and implicit (IcSACC) ways. Each approach is investigated as a front-end of both BLSTM-and TCN-based VAD+OSD systems. The self-attention-based models achieve similar performance as the MVDR without requiring the costly estimation of the covariance matrices. Among the SACC extensions, the learnable filter bank exhibits mitigated OSD results while complex extensions drastically improve the performance in the distant speech scenario e.g. EcSACC achieves 68.4% F1-score on the AMI evaluation set with the TCN model. On the VAD task, the segmentation error rate (SER) can be improved with channel-combination algorithms.

The impact of VAD and OSD is evaluated the final backend task, i.e. speaker diarization. Complex SACC extensions offer the best diarization performance with 23.30% DER and 30.90% JER for EcSACC. The assignment of overlap segments highlights the need for robust OSD since one can expect up to +36.3% relative DER improvement, as shown by Oracle's performance.

The complex SACC extensions achieve competitive VAD+OSD and speaker diarization performance. These approaches are also designed to be more explainable. The analysis of the combination weights with the beampattern exhibits a correlation between the maxima of acoustic energy and the steering directions. The EcSACC seems better at steering towards the source. Finally, the performance of the proposed front-ends highly relies on the microphone array configuration used during training. Thus, a mismatch between the training and the testing array setup may lead to severe performance degradation. To minimize this degradation, a new loss function is proposed and added to the training process. This training objective consists of learning a unique feature sequence regardless of the number of available microphones. Experiments conducted on a TCN-based VAD+OSD system with four frontends demonstrate that the performance remains steady regardless of the number of active microphones (SACC+ST F T : +10.2% F1-score improvement on the mismatched data from AMI array 2). In the case of EcSACC and IcSACC, this loss prevents the model from failing the detection.

In future work, new training objectives to learn a channelinvariant representation will also be investigated. The evaluation of invariant models in the cross-corpus scenario will also be considered. Since the complex combination weights encode spatial information about the active source, they could bring more information to the OSD system. Complex combination weights will thus be investigated as additional spatial features.

Fig. 1 :

 1 Fig. 1: Simplified sequence-to-sequence model architecture for joint VAD+OSD. The feature extraction is different between the single channel C = 1 and the multi-channel C > 1 scenario. The sequence modeling step remains unchanged regardless the number of channels.

Fig. 2 :

 2 Fig. 2: Channel combination procedure for multi-channel feature extraction. The time-frequency representation and the weight calculation algorithm depend on the algorithm considered (e.g. SACC).

Fig. 3 :

 3 Fig. 3: Weight calculation procedures, with (b) to (d) being the proposed methods. (a) STFT-based SACC [42], (b) SACC based on the analytic filter bank, (c) EcSACC, and (d) IcSACC. For complex systems (c) and (d), magnitude and phase (∥ • ∥-ϕ) can be replaced by real and imaginary parts (R-I).

B

  . Training and evaluation procedure VAD+OSD systems are trained on 2000 batches of 64 segments per epoch. Training segment duration is set to 2s. These segments are randomly sampled in the AMI training set.

Fig. 4 :

 4 Fig. 4: Time-averaged beampatterns (white line) at frequency f = 750Hz computed on the IcSACC ((a)-(d)) and EcSACC ((e)-(h)) combination weights on 1-second segments from the AMI development set. The heatmap represents the SRP-PHAT acoustic map computed on a 2m radius circular plane. The beampatterns are normalized between 0 and 1.

Fig. 5 :

 5 Fig.5: Flowchart of the channel-number invariant procedure. Only one duplicated segment is represented here (P =1).

Fig. 6 :

 6 Fig. 6: OSD class prediction with the EcSACC model with (Top) C = 8 microphones and (Bottom) C = 4 microphones. (• • • ) target OSD label, (-) invariant system prediction, (---) original system prediction.

TABLE I :

 I VAD and OSD performance of the 3-class BLSTM segmentation model. The results are obtained on the AMI development and evaluation sets. Bold values indicate the best systems, i.e. within the confidence interval of the best system estimated at the file level. Pr: precision, Re: recall.

				OSD					VAD
	BLSTM	Pr. %↑	Re. %↑	F1 %↑	FA %↓	Miss %↓	SER %↓
		Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval
	Close talk	64.8	72.8	76.4	66.0	70.1	69.2	5.35	3.85	1.32	1.79	6.66	5.64
	SDM	59.5	63.0	69.3	61.9	64.0	62.5	4.13	3.70	2.54	3.39	6.68	7.06
	MVDR	64.0	70.7	73.7	63.5	68.5	66.9	4.04	3.91	1.87	2.64	5.91	6.55
	SACC+ST F T	69.2	71.9	71.1	63.4	70.2	67.4	4.52	4.13	1.53	2.27	6.05	6.40
	SACC+A	66.6	68.2	67.6	61.2	67.1	64.5	5.22	4.65	1.58	2.06	6.80	6.71
	IcSACC Mag-ϕ	61.4	67.9	72.9	64.2	66.7	66.0	4.44	3.68	1.99	3.04	6.43	6.73
	EcSACC Mag-ϕ 64.8	71.6	71.6	62.1	68.0	66.5	4.39	3.58	1.96	3.23	6.35	6.80
	set (66.7%) but improves on the evaluation data with 66.0%.

TABLE II :

 II VAD and OSD Average Precision (AP %↑ ) score on the AMI Eval subset of the three best SACC-based models compared with the best system from[START_REF] Cornell | Overlapped speech detection and speaker counting using distant microphone arrays[END_REF]. * This score is obtained with a different protocol for AMI partition. The comparison has to be taken with care.

	Task	IcSACC EcSACC SACC+STFT	Cornell et al. [19]
	VAD	99.3	99.2	99.3	98.7 *
	OSD	70.6	70.9	71.4	60.4

* VII. IMPACT OF THE SEGMENTATION ON SPEAKER DIARIZATION

TABLE III :

 III VAD and OSD performance of the 3-class TCN segmentation model. The results are obtained on the AMI development and evaluation sets. Bold values indicate the best systems, i.e. within the confidence interval of the best system estimated at the file level. Pr: precision, Re: recall.

				OSD					VAD
	TCN	Pr. %↑	Re. %↑	F1 %↑	FA %↓	Miss %↓	SER %↓
		Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval	Dev	Eval
	Close talk	69.9	76.9	78.8	71.3	73.8	74.0	3.23	2.58	2.18	2.82	5.42	5.41
	SDM	70.6	72.7	67.6	58.7	69.0	64.9	6.08	5.27	1.14	1.54	7.22	6.81
	MVDR	72.5	77.0	72.3	63.6	72.3	69.6	3.69	3.21	2.25	3.12	5.94	6.32
	SACC+ST F T	72.2	74.8	71.5	63.2	71.8	68.5	2.89	2.43	3.04	4.31	5.94	6.73
	SACC+A	71.0	73.2	67.5	57.8	69.2	64.6	3.98	3.64	2.16	2.93	6.14	6.58
	IcSACC Mag-ϕ	71.6	75.6	71.5	61.1	71.5	67.6	4.10	3.36	2.13	3.41	6.24	6.77
	EcSACC Mag-ϕ 73.0	76.9	70.3	61.7	71.6	68.4	3.50	2.85	2.60	3.94	6.10	6.79

TABLE IV :

 IV Diarization performance of the VBx system with each distant segmentation model. Results obtained on the AMI evaluation set. δ c : duration of the collar applied during the evaluation in milliseconds.

	Segmentation	System	DER %↓ δc = 25 δc = 0	JER %↓
	Raj et al. [61]	VBx w/ OSD Spectral w/ OSD	--	25.05 23.69	--
	Oracle	VAD VAD w/ OSD	15.63 10.12	22.47 14.32	30.52 25.78
	SDM	VAD VAD w/ OSD	19.38 17.67	27.45 25.01	33.82 32.20
	MVDR	VAD VAD w/ OSD	18.19 16.00	26.80 23.71	32.77 30.87
	SACC+ST F T	VAD VAD w/ OSD	18.21 16.26	26.06 23.09	32.83 31.13
	SACC+A	VAD VAD w/ OSD	17.92 16.29	26.51 23.97	33.02 31.60
	EcSACC	VAD VAD w/ OSD	18.52 16.40	26.35 23.30	32.54 30.90
	IcSACC	VAD VAD w/ OSD	17.85 15.87	26.58 23.73	33.07 31.40

TABLE V :

 V OSD F1-score (%) on the Array 1 and Array 2 data of AMI evaluation subset. The Array 1 results are computed for a varying number of active channels C. Four SACC-based approaches are compared with TCN sequence modeling. Only the SACC=A 32 performance remains stable. Considering invariant training limits the degradation of the SACC+ST F T model (65.9%). SACC+A 32 retains its performance with the invariant training (65.3%). The IcSACC and EcSACC show the best performance with 67.8% and 67.1% respectively.The observations in the C = 2 scenario are similar to C = 4. The original SACC+ST F T is degraded by absolute -16.8%. This degradation is reduced with the invariant training (64.2%). SACC+A 32 remains robust to channel dropout (65.3%) while retaining performance with invariant training (65.6%). The invariant training enables IcSACC and EcSACC to detect overlapping speech. IcSACC offers the best performance with 67.2%.

	Array	C	Model	ST F T	A 32	IcSACC EcSACC
		8	Original /w L inv	68.5 68.4	64.6 65.2	67.6 67.2	68.4 67.1
	1	4	Original /w L inv	60.0 65.9	65.2 65.3	0.0 67.8	0.0 67.1
		2	Original /w L inv	51.7 64.2	65.3 65.6	0.0 67.2	0.0 66.7
	2	4	Original /w L inv	53.2 63.4	58.9 57.6	0.41 59.6	0.0 56.5
	overlapping speech anymore.


Céline Demouge
email: célinedemouge@enac.fr

Andréas Guitart

Gaspard Dannet

Daniel Delahaye

† C Demouge

An adapted Fast Marching Tree for contrails mitigation: A short-and long-range flight study

This paper addresses the problem of finding a trajectory for an aircraft minimizing its total environmental impact. In particular, CO 2 and contrails are considered. The Fast Marching Tree method is known to be efficient to find trajectories avoiding obstacles. This method is adapted to the case of soft obstacles, here areas favorable to contrails. This study is done in 2D and for the cruise phase of airliners. Two case studies on short-and long-haul flights are presented. The given results show trajectories minimizing the overall impact according to the given metric by trading off between CO 2 and contrails. Moreover, they are obtained with a low computation time allowing one to consider extensions. Finally, an improvement concerning the sampling is proposed, reducing the computation time and cost.

INTRODUCTION

Air transportation is responsible for 1.59% of radiative forcing due to human emission of CO 2 , and 5 % of total radiative forcing due to human activity [START_REF] Lee | The contribution of global aviation to anthropogenic climate forcing for 2000 to 2018[END_REF]. Fuel optimization is thereby a critical issue for environmental and economic reasons. This is a subject that has been studied extensively in the literature because of the need of efficient models to optimize trajectories. However, the environmental impact is not limited to CO 2 . Among these other effects, greenhouse gases such as NO x can be mentioned, but also other phenomena like contrails. They form behind aircraft when the air is humid and cold enough. They can persist and transform into cirrus clouds, leading to radiative forcing. This phenomenon is increasingly studied in the literature. Many uncertainties remain on their prediction and impact but it is established that the overall effect of this phenomenon is warming.

Contrail areas should be avoided as much as possible. However, if the CO 2 cost of avoidance is too high, it is not worthwhile to avoid contrails. Moreover, unlike thunderstorm areas, safety is not an issue when crossing contrail areas: they are not "hard" obstacles and are therefore considered as "soft" obstacles. In a free-flight context [START_REF] Schultz | Free flight concept[END_REF], this boils down to finding a trajectory that minimizes CO 2 while avoiding such areas as much as possible.

This paper addresses the problem of trajectory optimization in presence of contrails by using the Fast Marching Tree (FMT*) algorithm. This work follows a study done on wind-optimal trajectory generation using FMT* [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF], demonstrating the efficient adaptability of FMT* for wind and great circle distance considerations. In addition, previous works have shown the performance of FMT* in environments constrained by hard obstacles. We therefore propose an adaptation of this method to the case of contrails, considered as soft obstacles.

This paper is organized as follows: Section 2 presents previous works on contrails avoidance trajectory generation. Section 3 proposes a mathematical model, and a resolution method. Finally, Section 4 shows the first results of the proposed algorithm on two study cases (Berlin-Madrid and Paris-Los Angeles) and presents an improvement of the algorithm.

PREVIOUS RELATED WORKS

This section presents previous works related to contrails avoidance methods and samplingbased path planning algorithms.

Contrail avoidance methods

Contrails form at the rear of aircraft in humid and cold areas, typically at cruising altitudes. Under certain conditions, detailed in the sequel, they persist and turn into clouds called cirrus (named hereafter Contrails Induced Cirrus -CIC). They can have either a parasol effect or a greenhouse effect. The later prevails and therefore, globally, these CIC have a warming effect on the climate [START_REF] Lee | The contribution of global aviation to anthropogenic climate forcing for 2000 to 2018[END_REF].

Making a big detour to avoid contrails would increase fuel consumption and thereby CO 2 emissions too much, and would therefore not be environmentally efficient. Thus, contrails should be considered as soft obstacles. This subsection first presents the different metrics related to contrails that can be found in the literature. Then, different methods used in the literature are presented.

Contrail metrics

The goal is to minimize the total environmental impact of a trajectory, then metrics are needed to put CO 2 and non-CO 2 effects on the same scale.

The Global Warming Potential (GWP) is used as a conversion factor to balance the impact of a greenhouse gas and that of CO 2 . It is based on the comparison of a pulse impact of a gas to 1 kg of CO 2 and integrated over a time horizon H. GWP is therefore a function of H. For contrails and CIC, it is defined on a per CO 2 kg emission basis. More details about GWP can be found in [START_REF] Fuglestvedt | Transport impacts on atmosphere and climate: Metrics[END_REF], and Table 1 gives some examples over three time horizons. Other metrics, based on more complex models than those mentioned before, have been developed [START_REF] Matthes | A concept for multi-criteria environmental assessment of aircraft trajectories[END_REF]. They are more realistic, but by using a more complex model, the trajectory generation can be more complicated and may require more computation time.

Methods used in the literature

Several methods have been used for contrail mitigation trajectory design.

Since the temperature is a crucial point in contrail formation, flying at lower altitudes can be a very efficient mean for mitigating contrails. In [START_REF] Matthes | Mitigation of non-CO 2 aviation's climate impact by changing cruise altitudes[END_REF], authors show that reducing by 2000 ft the cruise altitude, radiative forcing due to CO 2 effects increases by 1%, but non-CO 2 effects decrease by 18%.

In [START_REF] Ng | Cross-polar aircraft trajectory optimization and the potential climate impact[END_REF], optimal control is used to compute wind-optimal trajectories for cross-polar flights while avoiding regions favorable to persistent contrail formation. The results show gradual reduction of flight time in contrail areas with fuel consumption increase. In [START_REF] Campbell | Fuel-optimal trajectory generation for persistent contrail mitigation[END_REF], a Mixed Integer Linear Programming (MILP) approach is used, resulting in some cases to 48% reduction of persistent contrails, with only 0.5% supplementary fuel consumption. Metaheuristics such as genetic algorithm are also used [START_REF] Yin | Impact on flight trajectory characteristics when avoiding the formation of persistent contrails for transatlantic flights[END_REF]. Finally, deterministic graph methods are used, for instance in [START_REF] Rosenow | Multicriteria-optimized trajectories impacting today's air traffic density, efficiency, and environmental compatibility[END_REF], Rosenow et al. use an A* algorithm coupled with a simulation framework. These deterministic methods have really good results but are based on some established grid, which, in the free-flight context, can be a problem. Indeed, a dense graph leads to a solution near to the optimal in the free space but with a high computation time. At the opposite, a less dense graph leads to larger sub-optimality but with small computation time.

Sampling-based methods do not appear in the literature review, but are effective in the case of optimal-wind trajectory computations and obstacle avoidance. This paper therefore proposes to adapt them to the case of contrails.

Sampling based methods

First introduced by Lavalle et al. [START_REF] Lavalle | Rapidly-exploring random trees: A new tool for path planning[END_REF], these methods were then improved in [START_REF] Karaman | Incremental sampling-based algorithms[END_REF] and [START_REF]Sampling-based algorithms for optimal motion planning[END_REF] to become very fast and asymptotically optimal. The three main methods are Probabilistic RoadMaps (PRM), Rapidly-exploring Random Tree (RRT), and Fast Marching Tree (FMT*) [START_REF] Janson | Fast Marching Tree: A fast marching sampling-based method for optimal motion planning in many dimensions[END_REF]. Due to their efficiency, these methods are widely used in robotics. Recently, they have been adapted to aircraft trajectories design [START_REF] Guitart | An accelerated dual fast marching tree applied to emergency geometric trajectory generation[END_REF][START_REF] Sáez | A fast and flexible emergency trajectory generator enhancing emergency geomet-ric planning with aircraft dynamics[END_REF]. The principle of these methods is to generate a graph by random sampling. The sampling can be initially generated as in FMT* and PRM or generated during the graph growth as in RRT. In both cases, these algorithms create a graph that avoids obstacles, and then finds a shortest path between two points. This paper proposes to use the FMT* algorithm to generate optimal paths in presence of contrails. Indeed, according to the literature, this algorithm is the fastest sampling-based path-planning algorithm and seems particularly adaptable to aircraft trajectory design. The FMT* algorithm is described in Subsection 3.3.

PROBLEM MODELLING AND RESOLUTION METHOD

This section presents the problem model, contrail areas computation, and the resolution approach.

Problem model

The goal is to find an aircraft trajectory γ minimizing its total environmental impact taking into account both CO 2 and non-CO 2 effects (only contrails here). To do so, the fuel consumption is to be minimized and the passage in persistent contrails areas is penalized with a conversion factor to render it comparable to CO 2 . The problem is written as follows:

minimize γ ∈ Γ t f t 0 C CO 2 f +C r r(x, y)dt,
where f is the fuel flow of the aircraft, C CO 2 is the conversion factor from the fuel quantity to the CO 2 quantity emitted (for weighting standard fuel, C CO 2 = 3.16 kg (CO 2 ) / kg (fuel) [START_REF] Graver | CO 2 emissions from commercial aviation[END_REF]), C r is the factor used to balance the impact of CO 2 and that of contrails, typically here the GWP is used, and r(x, y) is a penalty function to determine whether the aircraft is in a persistent-contrail area. It is equal to 1 when in contrail areas and 0 elsewhere. A constant fuel flow is assumed in the sequel. Indeed, the computation is done for the aircraft cruise phase with a constant altitude. Then, the aim is to minimize

t f t 0 1 + g H r(x, y)dt
, where g H is the GWP subject to the time horizon H chosen.

Where are contrails?

Based on temperature, pressure and relative humidity data, contrails and persistent contrails areas are computed thanks to the following process, as usually done in the literature (see for instance [START_REF] Soler | Flight trajectory design in the presence of contrails: Application of a multiphase mixed-integer optimal control approach[END_REF]). First, at a given point, the Schmidt-Appleman criterion is applied: a contrail is formed if the relative humidity of the air in liquid water, RH w is above a threshold r min , where

r min = G(T -T c ) + e liq sat (T c ) e liq sat (T ) , (1)
e liq sat (T ) is the saturation vapor pressure over water, and T c is the estimated threshold temperature (in Celsius degrees) for contrail formation at liquid saturation. The later is computed as follows:

T c = -46.46 + 9.43 log(G -0.053) + 0.72 log 2 (G -0.053), [START_REF] Schultz | Free flight concept[END_REF] where

G = EI H 2 O C p P εQ(1-η) , EI H 2 O = 1.
25 is the water vapor emission index, C p = 1004 J.kg -1 .K -1 is the heat capacity of the air, P is the ambient pressure (in Pascals), ε = 0.6222 is the ratio of the molecular masses of water and dry air, Q = 43 • 10 6 J.kg -1 is the specific heat of combustion, and η = 0.3 is the average propulsion efficiency of a commercial aircraft (using the same values as in [START_REF] Soler | Flight trajectory design in the presence of contrails: Application of a multiphase mixed-integer optimal control approach[END_REF]).

Then, if the point is in an ice-supersaturated area, it is considered in a persistentcontrail favorable area. In [START_REF] Soler | Flight trajectory design in the presence of contrails: Application of a multiphase mixed-integer optimal control approach[END_REF], these areas are determined thanks to the following criterion: RH i > 1, where the relative humidity over the ice, noted RH i is computed as follows:

RH i =
, ( 3
)
where T is the ambient temperature in Celsius degrees.

Resolution algorithm

The resolution algorithm is an adaptation of the Fast Marching Tree algorithm to great circle distances, wind and soft obstacle considerations. The sequel describes the original algorithm before focusing on the adaptation done in this study.

The environment is sampled with N points (nodes). At each iteration, nodes are split into three sets:

1. V unvisited : nodes never visited;

2. V open : nodes already visited but for which the cost value is not yet definitively computed;

3. V closed : nodes visited with a definitive cost value.

One iteration follows this process:

1. A lowest-cost node z is chosen in V open .
2. For each neighbors node x of z which are in V unvisited , an edge between x and an open neighbors of x is added to make the connection locally optimal without considering any constraint.

3. If this connection does not violate constraints (e.g. obstacles crossing), the edge is added to the tree and x is put in V open .

4. Once all unvisited neighbors of z have been visited, z is put in V closed .

All the nodes at a distance r N of a node are considered as its neighbors. It is the neighborhood radius and is a function of the number N of samplings. The radius used in [START_REF] Janson | Fast Marching Tree: A fast marching sampling-based method for optimal motion planning in many dimensions[END_REF] is:

r N = 2(1 + η) 1 d 1/d µ(χ f ree ) ζ d 1/d log(N) N 1/d , (4)
where η > 0, d is the space dimension, χ f ree is the free space, µ(χ f ree ) is its Lebesgue measure, and ζ d is the volume of the d-dimensional unitary ball. First, based on the preliminary study [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF], the problem where no contrails are taken into account is solved. In [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF], the cost to be minimized between two nodes (Step 3) is the flight time, implying great circle distances and ground speeds. The first one is computed in a computationally non-expensive way as in [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF]. The ground speed is computed taking into account spatially-dependent static wind.

In this study, the cost (used at Step 3) is further adapted to take into account contrails: the cost of edge (i, j) between node i and node j is:

c i, j = t i, j (1 + λ i, j g H ), (5)
where t i, j is the flight time between i and j (great circle distance divided by the ground speed), λ i, j ∈ [0, 1] is the proportion of the edge that is in a persistent contrail area, and g H is the chosen GWP value, depending on the time horizon H. The cost function differs from the one used in [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF], which only took flight time into account.

CASE STUDIES AND METHOD IMPROVEMENT

This section presents two study cases of our algorithm before focusing on an improvement on the sampling. All the presented results have been developed in Java programming on a computer with an Intel Core i5-10210U, 1.60 Hz, with 8 Go RAM and a Debian Linux OS. The weather data for wind consideration and data for contrail areas computation (see Subsection 3.2) have been extracted from ERA5 reanalysis data [START_REF] Hersbach | Copernicus Climate Change Service (C3S) Climate Data Store (CDS)[END_REF]. CO 2 emissions are estimated from OpenAP [START_REF] Sun | Openap: An open-source aircraft performance model for air transportation studies and simulations[END_REF], considering an initial mass of 80% of the Maximum Take Off Mass of the aircraft. Unless otherwise indicated, the results are obtained with a uniform sampling of 10,000 points on a rectangle.

Berlin-Madrid case study 4.1.1 Instance setup

A typical short-range aircraft, the Airbus 320, is considered for the results presented in the sequel, and the airspeed considered is V a = 450 kts [START_REF]SKYbrary aviation safety[END_REF].

The first study case is a flight from Berlin to Madrid at the Flight Level 300 on January 5 th 2023, at 00:00 UTC. Figure 1 and Figure 2 show the wind and the persistent contrail areas encountered.

The computational experiments have been done with a GWP computed with two different time horizons (20 and 100 years). This study case shows the efficiency of the algorithm, since the mean computation time is 6.5 seconds. It is then a fast algorithm allowing to avoid contrails while trading off CO 2 considerations which could be used at tactical level. Moreover, the impact of the chosen metric, in particular the time horizon H chosen for the GWP computation, is shown. In the case of a short time horizon, contrails have a strong impact (see Table 1) and the areas are thereby avoided, even at the expense of a big increase in CO 2 . This kind of solution does not seem operationally acceptable. On the contrary, with a

Long haul study case

A long-range flight is considered: Paris to Los Angeles with a typical long-range aircraft, the Boeing 777-300ER and the airspeed V a = 490 kts [START_REF][END_REF]. The flight level chosen is FL 340 (250 hPa).

Figure 4 shows the wind encountered for the transatlantic area presented in this subsection and Figure 5 shows the considered persistent-contrail areas. Table 3 shows the quantitative results obtained with a mean computation time of 6.8 seconds. These results shows that, even though transatlantic flights are constrained by wind, contrails can be avoided at the expense of a small increase in CO 2 emissions. This is obtained within a small computation, allowing the user to perform several tests if ensemble forecasts [START_REF] Leutbecher | Ensemble forecasting[END_REF] are considered for instance or to update the trajectory all along the flight, based on the updates of the weather forecasts.

Sampling improvement

Some areas, when sampling a rectangle, are never visited by the algorithm. Indeed in the case of the transatlantic flights, some areas of the sample rectangle are not visited or will not be with certainty in the final solution, as shown in Figure 6. By avoiding sampling these areas, with a same total number of sampling points, the area of interest could be more finely sampled. Similarly, the number of points can be reduced by the same factor as the area is reduced, since the sampling is uniform, reducing thereby computation time and memory-space use. Moreover, this allows one to constrain the trajectory in a specific part of the space, avoiding areas that are too far for instance.

To reproduce a desirable search space for trajectories between Paris and Los Angeles, it is possible to process historical flight data (shown on Figure 7) and determine a tube that the computed trajectory should not leave. A safe margin is taken with re-spect to the flown extremes, arbitrary fixed to 10% here. The construction principle is described on Figure 8 and such a sampling tube is shown on Figure 9 1. 10,000 samples, as before but within the tube; 2. 7,350 samples in a reduced sampling area, i.e., using the same distribution as before with a sampled area reduced by a factor 1.36. A reduced computing time with roughly the same performance is expected.

The setup of the previous studies is used. Figure 10 and Table 4 show the results with 10,000 samples (obtained in 6.7 seconds on average) and with 7,350 samples (in 5.7 seconds on average). Costs and trajectories are likely the same with a lowest computation time. To conclude, this improvement reduce the number of samples or increase the concentration of the sampling. If the number of samples can be reduced, then the algorithm is faster and uses less memory. Thus, it would allow the user to implement it on board or to repeat more easily computations to obtain updated solutions throughout the flight which is also useful considering uncertainties. By increasing the concentration of the sampling, better results are expected with a same number of samples.

CONCLUSION

This paper presents a new method for contrail mitigation in a free-flight context. To do this, an adaptation of a sampling-based method, the Fast Marching Tree (FMT*), is implemented. FMT* is known to be efficient for obstacle avoidance [START_REF] Janson | Fast Marching Tree: A fast marching sampling-based method for optimal motion planning in many dimensions[END_REF], and has been used in a first study for wind optimal trajectory computation [START_REF] Demouge | Fast Marching Tree applied to geodesic trajectories in presence of wind: A day of flights in Europe study[END_REF]. It is adapted here to address soft obstacles, such as contrail areas.

Based on experiments on short and long-range flights, it is shown that trajectories avoiding contrails balancing CO 2 according to a given metric within small computational time. For instance, on a long range, with a time horizon of 100 years for GWP computation, the CO 2 emissions increases by less than 5% when the time spent in the contrails decreases by around 90%. Moreover, an improvement is proposed, allowing the user to constrain the trajectory to a desirable space and reducing computation time. These encouraging results pave the way to possible extensions. In particular, in the dynamic case, the trajectory could be updated continuously, as weather information is received. The proposed sampling strategy in a tube could avoid the algorithm to modify too much the trajectory from one step to another, while taking advantage of the new information. The margin considered can also be adapted to the flight envelope, improving sampling area to meet flying constraints. In addition, uncertainties could be considered using ensemble forecasts. Finally, an extension to three dimensions could be envisaged by imposing operational constraints.

Figure 1 :

 1 Figure 1: Wind setup, at FL300 on January 5 th 2023, at 00:00 UTC.

Figure 2 :

 2 Figure 2: Contrails setup, at FL 300 on January 5 th 2023, at 00:00 UTC.

Figure 3 and

 3 Figure 3 and Table 2 show the results obtained by our algorithm. Loss in flight time or CO 2 emissions and gain in flight time in contrail areas are computed in comparison with the case where no contrail are considered.

Figure 3 :

 3 Figure 3: Results on Berlin-Madrid, at FL300 on January 5 th 2023, at 00:00 UTC, with different time horizons for GWP.

Figure 4 :

 4 Figure 4: Wind setup, at FL340 on January 01 th 2022, at 11:00 UTC.

Figure 5 :

 5 Figure 5: Contrails setup, at FL340 on January 01 th 2022, at 11:00 UTC.

Figure 6 :

 6 Figure 6: Some areas are sampled but are not useful in the algorithm.

Figure 7 :

 7 Figure 7: Past flights from Paris to Los Angeles in September 2019, from Eurocontrol R&D data [24] (in black: the great circle trajectory).

Figure 8 :

 8 Figure 8: Tube construction principle.

Figure 9 :

 9 Figure 9: Example with 1000 samples in a tube between Paris and Los Angeles and bounded by extremal latitudes. Then, two experiments are conducted:

Figure 10 :

 10 Figure 10: Results on Paris-Los Angeles, at FL340 on January 01 th 2022, at 11:00 UTC, with tube sampling.

Table 1 :

 1 GWP for contrails with different time horizon.

		H = 20 years H = 100 years H = 500 years
	GWP contrail (H) 0.74	0.21	0.064
	GWP CIC (H)	2.2	0.63	0.19

Table 2 :

 2 Results on Berlin-Madrid, at FL300 on January 5 th 2023, at 00:00 UTC, with different time horizons for GWP.

			Flight time in	CO 2 emissions in	Time in contrail
			minutes (loss)	kg (loss)	areas in minutes
					(gain)
	No contrail con-	134	13,595	55
	siderations
	GWP	with	141 (+5%)	14,294 (+5.1%)	27 (-51%)
	H=100 years
	GWP	with	165 (+23%)	16,674 (+22.6%)	0 (-100%)
	H=20 years
	longer time horizon, the computed solution is more balanced between total CO 2 and
	time spent in contrails.

Table 3 :

 3 Results on study from Paris to Los Angeles, at FL340 on January 01 th 2022, at 11:00 UTC.

			Flight time in	CO 2 emissions	Time in contrail
			minutes (loss)	in kg (loss)	area in minutes
					(gain)
	No contrail con-	597	256,649	95
	siderations
	GWP	with	622 (+4.2%)	266,735 (+3.9%) 11 (-88.4%)
	H=100 years

Table 4 :

 4 Results on study from Paris to Los Angeles with tube sampling. Comparison are computed regarding cases with the same sampling.

		Flight time in min-	Time in contrail areas
		utes (loss)	in minutes (gain)
	No contrail considerations -	597	95
	No tube -10,000 samples
	GWP with H=100 years -No	622 (+4.2%)	11 (-88.4 %)
	tube -10,000 samples
	No contrail considerations -	596	89
	Tube -10,000 samples
	GWP with H=100 years -	623 (+4.5%)	9 (-89.9 %)
	Tube -10,000 samples
	No contrail considerations -	596	89
	Tube -7,350 samples
	GWP with H=100 years -	623 (+4.5%)	8 (-91 %)
	Tube -7,350 samples

ACKNOWLEDGMENT

The authors thank DGAC (French civil aviation authority) for prompting and funding this work, and more specifically its DTA and DSNA services.