domain: info scco

text:
Marvin Lavechin
email: marvinlavechin@gmail.com

Yaya Sy

Hadrien Titeux

María Andrea

Cruz Blandón

Okko Räsänen

Hervé Bredin

Emmanuel Dupoux

Alejandrina Cristia

BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models

Keywords: spoken language modeling, language acquisition, self-supervised learning, child language

come

Introduction and related work

Machine learning for Natural Language Processing (NLP) has led to models that develop linguistic competence from exposure to written or spoken language. On text, Language Models (LMs) now achieve impressive performance on a wide variety of natural language understanding tasks [START_REF] Brown | Language models are few-shot learners[END_REF]. More recently, speech-based LMs have also shown impressive linguistic competence on lexical or grammatical acceptability judgment tasks [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF][START_REF] Dunbar | The zero resource speech challenge 2021: Spoken language modelling[END_REF], or spoken language generation [START_REF] Lakhotia | On generative spoken language modeling from raw audio[END_REF][START_REF] Kharitonov | Text-free prosody-aware generative spoken language modeling[END_REF]. Since these models develop linguistic competence without the need for human labels, they promise to advance our understanding of how infants learn language [START_REF] Dupoux | Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner[END_REF][START_REF] Lavechin | Reverse engineering language acquisition with childcentered long-form recordings[END_REF][START_REF] Warstadt | What artificial neural networks can tell us about human language acquisition[END_REF]. However, if we want to maximize the impact of the evidence obtained from LMs, it is essential to ensure that our simulations closely emulate real-life situations -as advocated for syntactic acquisition in text-based LMs in [START_REF] Warstadt | What artificial neural networks can tell us about human language acquisition[END_REF][START_REF] Huebner | Babyberta: Learning more grammar with small-scale child-directed language[END_REF].

How can we do so? First, we should match the quantity of data available to young infants. Although large differences exist across cultures [START_REF] Cristia | Childdirected speech is infrequent in a forager-farmer population: A time allocation study[END_REF] and socioeconomic contexts [START_REF] Dailey | Language input to infants of different socioeconomic statuses: A quantitative meta-analysis[END_REF], current estimates of yearly speech input vary between 300 and 1, 000 hours for American English-learning children [START_REF] Dupoux | Cognitive science in the era of artificial intelligence: A roadmap for reverse-engineering the infant language-learner[END_REF][START_REF] Hart | Meaningful differences in the everyday experience of young American children[END_REF]. This means that by age 3, American English-learning children would have been exposed to approximately 3,000 hours of speech -for those who received the most speech input. Yet, by then, infants know many words and already engage in simple conversations [START_REF] Hoff | Blackwell handbook of language development[END_REF]. Second, we should match the quality of data available to young infants. Contrary to LMs, infants do not learn language by scraping the entire web or through exposure to a large quantity of audiobooks. Instead, infants' input is speech -not text -, and it contains a relatively small vocabulary arranged in simple and short sentences, sometimes overlapping across speakers and laced with various background noises [START_REF] Lavechin | Reverse engineering language acquisition with childcentered long-form recordings[END_REF][START_REF] Lavechin | Statistical learning models of early phonetic acquisition struggle with child-centered audio data[END_REF].

Evaluating LMs trained on quantitatively and qualitatively plausible corpora requires the creation of adapted benchmarks, but none exists for speech-based LMs -see [START_REF] Huebner | Babyberta: Learning more grammar with small-scale child-directed language[END_REF] or the BabyLM challenge [START_REF] Warstadt | Call for papers -The BabyLM challenge: Sampleefficient pretraining on a developmentally plausible corpus[END_REF] for text-based LMs. Current benchmarks using zero-shot probing tasks, although inspired by human psycholinguistics (e.g., spot-the-word or grammatical acceptability judgment tasks), have been designed for models trained on audiobooks [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF]. As a result, these benchmarks use a large vocabulary specific to books (including words like 'rhapsodize', 'zirconium', or 'tercentenary') and probe syntactically complex sentences that are vanishingly rare even in spontaneous adultadult conversation.

Here, we propose BabySLM, the first language-acquisitionfriendly benchmark to probe speech-based LMs at the lexical and syntactic levels, both of which are compatible with the vocabulary typical of children's language experiences. Our benchmark relies on zero-shot behavioral probing of LMs [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF] and considers a spot-the-word task at the lexical level and a grammatical acceptability judgment task at the syntactic level. To show the utility of our benchmark, we first use it to evaluate text-based and speech-based LMs trained on developmentally plausible training sets. The text-based LM is a long short-term memory (LSTM) trained on phonemes or words. The speechbased LM is the low-budget baseline used in the ZeroSpeech 2021 challenge on unsupervised representation learning of spoken language [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF]. Both systems are trained on Providence [START_REF] Börschinger | A joint model of word segmentation and phonological variation for English wordfinal/t/-deletion[END_REF], a dataset of spontaneous parent-child interactions. The comparison between text-based and speech-based LMs shows an important gap that future work should address. Next, BabySLM enables us to compare the performance of speech-based LMs when trained on 1, 000 hours of speech extracted from 1) audiobooks, a source of training data commonly used [START_REF] Kahn | Libri-light: A benchmark for ASR with limited or no supervision[END_REF][START_REF] Mohamed | Self-supervised speech representation learning: A review[END_REF]; or 2) child-centered long-form recordings acquired via child-worn microphones as people go about their everyday activities [START_REF] Bergelson | What do North American babies hear? A large-scale cross-corpus analysis[END_REF]. Our results reveal that speech-based LMs are overly sensitive to the differences between clean speech and in-the-wild speech. General principle. In the lexical task, the system is presented with minimal pairs of an existing word and a pseudo-word that 1). The system gets a score of 1 if it returns a higher probability for the former, and 0 otherwise. Contrary to [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF], we generate multiple pseudo-words per word. Scores are first averaged across pseudo-words to yield per-word accuracy, which are then averaged across all words to yield a measure of lexical accuracy. Task generation. We first listed all words in the American English CHIld Language Data Exchange System (CHILDES) database [START_REF] Macwhinney | The child language data exchange system[END_REF]. This database contains human-annotated transcripts of various child-centered situations (play sessions, storytelling, etc.), making it a valuable source of vocabulary in real children's input. After excluding items not found in either the Celex [START_REF] Baayen | Linguistic Data Consortium[END_REF] or CMU dictionary [START_REF] Weide | The Carnegie Mellon pronouncing dictionary[END_REF] (e.g., mispronounced, incorrectly annotated or made-up words: 'insectasaurus', 'hiphippopotamus'), we obtained 28, 000 word types. Pseudo-words were produced using the Wuggy pipeline [START_REF] Keuleers | Wuggy: A multilingual pseudoword generator[END_REF], which generates, for a given word, a list of candidate pseudo-words matched for syllabic and phonotactic structure. We applied the same postprocessing steps used in [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF]. Contrary to [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF], to ensure that there is no bias from phone-based unigrams or bigrams, we balanced the count of pseudo-words that had higher (or lower) phonemes unigram and bigram probabilities compared to those computed for the actual word. If a given word had only pseudo-words with higher (or lower) unigram or bigram possibilities, it was discarded from the evaluation set. The resulting > 90, 000 minimal pairs across 18, 000 words were each synthesized using Google Text-To-Speech (TTS) system using 10 voices (5 males, 5 females).

Methods

Syntactic evaluation: grammatical acceptability

General principle. In the syntactic task, the system is presented with minimal pairs of grammatical and ungrammatical sentences across six syntactic phenomena [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF][START_REF] Huebner | Babyberta: Learning more grammar with small-scale child-directed language[END_REF] (examples in Table 2), giving the system a score of 1 when it assigns a higher probability to the former, and 0 otherwise. We average scores within each syntactic phenomenon, then across phenomena to obtain our measure of syntactic accuracy. Task generation. We generated templates for each of the six syntactic phenomena explored. For instance, for the nounverb agreement phenomenon, we used templates such as "The <noun1> <3 rd person verb> <noun2>" versus "The <noun1> <1 st person verb> <noun2>". Contrary to [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF], we restricted this benchmark to simple syntactic phenomena and short sen- tences which better reflect the type of input children are exposed to. We filled the templates using high-frequency words from CHILDES [START_REF] Macwhinney | The child language data exchange system[END_REF]. For instance, selected animate nouns include words like 'mom', 'girl', or 'cat'; selected adjectives include words like 'good', 'little', or 'big'; and selected verbs include words like 'see', 'know', or 'need'. The resulting 10, 800 minimal pairs were each synthesized using Google TTS system using the same 10 voices (5 males, 5 females).

Development and test split

For both our lexical and syntactic evaluation sets, we randomly selected one male and one female voice for the development set and the 8 remaining ones for the test. We randomly selected 20 % of the lexical and syntactic minimal pairs for the development set and the remaining 80 % for the test.

Training sets

We built a first training set by extracting human-annotated speech utterances from Providence [START_REF] Börschinger | A joint model of word segmentation and phonological variation for English wordfinal/t/-deletion[END_REF], a publicly available corpus containing transcribed recordings of six American children during spontaneous interactions with their parents. Available utterance-level timestamps were refined with a pretrained voice activity detection (VAD) system [START_REF] Lavechin | An open-source voice type classifier for child-centered daylong recordings[END_REF]. We converted human orthographic transcripts into phonetic transcripts using [START_REF] Bernard | Phonemizer: Text to phones transcription for multiple languages in python[END_REF]. This procedure resulted in 128 hours of highly naturalistic infant-parent interactions in audio, orthographic, and phonetic form, allowing us to compare LMs trained on speech, phonemes, or words. We built a second training set by extracting 1, 024 hours of adult speech utterances -using the same VAD system [START_REF] Lavechin | An open-source voice type classifier for child-centered daylong recordings[END_REF] from SEEDLingS [START_REF] Bergelson | What do North American babies hear? A large-scale cross-corpus analysis[END_REF], a corpus of child-centered long-form recordings collected in 61 American English families. This training set enables us to train speech-based LMs in maximally plausible conditions, i.e., directly on what infants hear.

Models

STELA (speech-based). STELA is a speech-based LM originally proposed in [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF][START_REF] Lavechin | Statistical learning bootstraps early language acquisition[END_REF]. It comprises an acoustic model that learns discrete representations of the audio and a language Table 3: The BabySLM benchmark. Lexical and syntactic accuracies obtained by different language models trained on developmentally plausible corpora of speech, phonemes, or words. Numbers are computed on the test set, and performances on the development set are reported using small font size. The starred cumulated duration and number of words are estimates based on the 1.2 M of words present in the 128 hours of speech from Providence. Data plausibility indicates the extent to which the training set is close to the real sensory signal available to infants. The language model consists of LSTM layers. We used the same architecture and hyper-parameters as the low-budget baseline proposed in [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF]. Contrary to [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF] who trained CPC by sampling the positive and negative examples from the same speaker, we applied a second constraint: negative examples were drawn from temporally close speech sequences to reduce mismatch between the positive and negative examples in terms of their local environment as this was found to be helpful when training on long-forms [START_REF] Lavechin | Statistical learning models of early phonetic acquisition struggle with child-centered audio data[END_REF].

Cumulated

LSTM (text-based). We include LSTM LMs trained on words -using byte-pair encoding -or on phonemes, using the same architecture and hyper-parameters than [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF].

BabyBERTa (text-based). BabyBERTa [START_REF] Huebner | Babyberta: Learning more grammar with small-scale child-directed language[END_REF] is a transformerbased LM trained on a 5 M word corpus of American English child-directed input built from the CHILDES database [START_REF] Macwhinney | The child language data exchange system[END_REF].

Results and discussion

The BabySLM benchmark

Results obtained on our BabySLM benchmark are reported in Table 3. Rows are sorted according to the plausibility of the training data. Child-centered long-form recordings (SEEDLingS) have the highest plausibility score as these recordings faithfully capture children's everyday language experiences. In particular, long-forms collect audio data over a whole day -or several -and therefore sample the full range of language experiences across all possible contexts: the child may be in or out of the house, the speech may be directed to the child or others, etc. The audio extracted from in-home recordings of spontaneous infant-parent interactions (Providence) is slightly less plausible as it fails to capture the full range of language experiences: fewer speakers than in a real-life setting, most of the speech is directed to the child, etc. Finally, words and phonemes extracted from AO-CHILDES or Providence have the lowest plausibility score since infants do not learn language from orthographic or phonetic transcriptions but from the continuous signal that is speech. Results indicate no evidence of lexical and syntactic knowledge for STELA trained on 1, 024 hours of speech from SEEDLingS. This contrasts, in appearance, with what has been found in the ZeroSpeech challenge [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF], but this is due to the large variability of speech found in long-forms as we will see in Section 3.3. Results are no different for STELA trained on 128 hours of speech extracted from Providence whose lexical and syntactic accuracies remain close to chance level. However, we hypothesize that the lexical accuracy obtained by STELA might increase with more audio data from semi-controlled recordings of infant-parent interactions as these contain cleaner speech than what is typically found in long-forms. Contrary to speechbased LMs, text-based LMs perform largely above chance level. As expected, the LSTM model trained on words reaches higher syntactic accuracy than the LSTM trained on phonemes. The highest syntactic accuracy is obtained by BabyBERTa, which is a transformer-based LM and has been trained on a larger quantity of data than our LSTM LMs.

Performances on BabySLM show a clear gap between textbased and speech-based LMs. Another important finding is that, as of now, spoken language modeling from children's real language experiences seems out of reach, as evidenced by the chance-level lexical and syntactic accuracies obtained by STELA trained on SEEDLingS. We dedicate the remaining sections to illustrating these two challenges: bridging the gap between text and speech and between clean speech and in-the-wild speech.

Language modeling: from text to speech

Figure 1 shows lexical and syntactic accuracies obtained by text-based (words or phonemes) or speech-based LMs as a function of quantity of data. The LSTM trained on phones requires at least 16 hours of speech, equivalent to 150, 000 words, to start performing above chance level. Once lexical knowledge has emerged, the model follows a logarithmic trend (note the log-scale x-axis), initially improving rapidly and then slowing down. In other words, we need to double the amount of data to obtain the same gain in lexical accuracy. The same patterns hold for the syntactic accuracy obtained by the LSTM model trained on words 1 . For STELA, the lexical accuracy remains close to chance level, although the curve seems to increase between 32 and 128 hours of speech, and there is no evidence for syntactic knowledge.

All in all, the lexical and syntactic accuracy slopes show very different patterns when training from raw speech or phonemes or words. This is despite receiving the same data in different forms. Admittedly, the speech-based LM faces a more challenging task as it must learn its own discrete units, while text-based LMs must not. Future work might investigate how these slopes change with more data, particularly for the speech-based LM for which 128 hours seems insufficient.

Language modeling: from clean to in-the-wild speech

So far in the paper, we have little evidence that lexical or syntactic knowledge can emerge in speech-based LMs. To address this concern, we ran one more experiment, this time training STELA under more controlled recording conditions: on up to 1, 024 hours of speech extracted from audiobooks -commonly used to train speech-based LMs [START_REF] Kahn | Libri-light: A benchmark for ASR with limited or no supervision[END_REF]. Figure 2 compares this experiment against the performance obtained by STELA when trained on child-centered long-forms (SEEDLingS, Table 3).

Results are unequivocal: we observe a strong improvement on the lexical task for the model trained on audiobooks, while the same model trained on long-forms remains at chance level. On the syntactic task (not shown above), STELA trained on 1, 024 hours of audiobooks obtains an accuracy of 52.8 % compared to 50.3 % on long-forms. This is in line with the results in [START_REF] Nguyen | The zero resource speech benchmark 2021: Metrics and baselines for unsupervised spoken language modeling[END_REF] showing that more powerful architectures are necessary to learn at the syntactic level.

Why do we observe chance-level performance when training on long-forms? First, the speech signal found in long-forms is much more challenging than the one found in audiobooks: the speech might be distorted as it is being spoken far from the child; it might overlap with various background noises; and it is often produced in short turns that might be under-articulatedsee [START_REF] Lavechin | Statistical learning models of early phonetic acquisition struggle with child-centered audio data[END_REF] for a comparative analysis. Another essential factor to consider is the domain mismatch between the training and test sets. While the training set contains far-field under-articulated speech as well as close-field storytelling, the test set consists of well-articulated synthesized stimulus to which STELA fails to generalize. However, infants show no difficulties generalizing from uncontrolled real-life conditions to more controlled ones (in-laboratory conditions). We advocate here that generalization is part of the language acquisition problem, and LMs should be evaluated accordingly.

We hypothesize that the discrete units learned by STELA might be too dependent on the various non-linguistic factors found in long-forms, as suggested in [START_REF] Lavechin | Statistical learning models of early phonetic acquisition struggle with child-centered audio data[END_REF]. This dependency could prevent the LSTM LM from learning long-term dependencies necessary to solve the lexical or syntactic tasks.

Conclusion

Benchmarks are instrumental in allowing cumulative science across research teams. In this paper, we have described how BabySLM has been carefully designed to be adapted to the kinds of words and sentences children hear. We have shown how it can be used to evaluate LMs trained on developmentally plausible text or speech corpus. By doing so, we revealed two outstanding challenges that the community must solve to build more plausible cognitive models of language acquisition. First, we need to reduce the gap between text-based and speech-based LMs, as the latter performed close to chance level on BabySLM. Second, we need to reduce the gap between LMs trained on clean and in-the-wild speech, as evidenced by the striking difference we obtained on the lexical task when training on clean audiobooks versus ecological long-forms.

Future work might consist in evaluating speech-based LMs grounded in the visual modality [START_REF] Nikolaus | Learning English with Peppa Pig[END_REF], or linking performances obtained on BabySLM with behavioral measures in infantse.g., age of acquisition as in [START_REF] Portelance | Predicting age of acquisition in early word learning using recurrent neural networks[END_REF]. A crucial limitation of our benchmark is that it focuses on English, which already accounts for a whopping 54 % of language acquisition studies [START_REF] Kidd | How diverse is child language acquisition research?[END_REF]. We hope that this paper, together with shared scripts2 , will facilitate the creation of similar benchmarks in other languages.

2. 1 .

 1 Metrics 2.1.1. Lexical evaluation: the spot-the-word task

Figure 1 :

 1 Figure 1: Language modeling from text to speech. Top panel shows the lexical accuracy obtained by language models trained on audio (STELA) or phonemes (LSTM). Bottom panel shows the syntactic accuracy obtained by language models trained on audio (STELA) or byte-pair-encoded (BPE) words (LSTM). All models are trained on the Providence corpora in audio, phonetic, or orthographic form. Numbers are computed on the test set. Error bars represent standard errors computed across mutually exclusive training sets.

Figure 2 :

 2 Figure 2: Language modeling from clean to in-the-wild speech. Lexical accuracy obtained by STELA trained on audiobooks (Libri-light, in blue) or child-centered long-forms (SEEDLingS, in orange) as a function of speech quantity. Numbers are computed on the test set. Error bars represent standard errors computed across mutually exclusive training sets.

Table 1 :

 1 Lexical task. Minimal pairs of real and pseudo-words. Phonetic (Phon.) transcriptions are given in International Phonetic Alphabet (IPA) standard. Orthographic (Orth.) transcriptions of pseudo-words are proposed for ease of reading.

		Pseudo-words		Pseudo-words
	Word	Phon.	Orth.	Word	Phon.	Orth.
		l @ l oU	lello		T E N k s	thaynks
	hello h @ l oU	p @ l oU pello s @ ô oU sero d @ l oU dello	thanks T ae N k s	T O N k s T I s k s T ae m p s thamps thoanks thisks
		s @ l oU sello		T ae n t s	thants
		k U t i:	kootie		dZ ae m p jamp
	cookie k U k i:	k U n i ô U d i: ô U t i:	koonie roodie rootie	jump dZ 2 m p	dZ 2 l k dZ 2 s k dZ 2 f t	julk jusk juft
		b U n i:	boonie		dZ 2 b s	jubs
	is phonologically plausible but does not actually exist [2, 20]
	(examples in Table

Table 2 :

 2 Syntactic task. Minimal pairs of grammatical (✓) and ungrammatical (✗) sentences from each of the six syntactic phenomena included in our benchmark. N is the number of 1, 000 minimal pairs within each category.

	Phenomenon	N	Sentence example
	Adjective-noun order	1.6	✓ The good mom. ✗ The mom good.
	Noun-verb order	1	✓ The dragon says. ✗ The says dragon.
	Anaphor-gender agreement	2	✓ The dad cuts himself. ✗ The dad cuts herself.
	Anaphor-number agreement	1	✓ The boys told themselves. ✗ The boys told himself.
	Determiner-noun agreement	3.6	✓ Each good sister. ✗ Many good sister.
	Noun-verb agreement	1.6	✓ The prince needs the princess.

✗ The prince need the princess.

Note, however, that the syntactic accuracy obtained by the LSTM model trained on words decreases to 45 % (below chance level) between 0 and 8 hours (= 75, 000 words). This effect was found to be driven by co-occurrence statistics in the noun-verb order task. The same pattern was found with a 3-gram model, with a slight decrease between 0 and 8 hours and an increase between 8 and 128 hours.

https://github.com/MarvinLvn/BabySLM

We thank HPC resources of GENCI-IDRIS (2022-AD011012554); ANR-19-P3IA-0001; J. S. McDonnell Foundation; ERC (ExELang, 101001095). We thank Robin Algayres and Hugo Laurençon for their help designing the syntactic tasks.


Askar Kondybayev

Gaukhar Konuspayeva
email: konuspayevags@hotmail.fr

Caroline Strub
email: caroline.strub@umontpellier.fr

Gerard Loiseau
email: loiseau@cirad.fr

Christian Mestres
email: christian.mestres@cirad.fr

Joel Grabulos
email: joel.grabulos@cirad.fr

Marie Manzano
email: mariemanzano@icloud.com

Shynar Akhmetsadykova
email: shynar.akhmetsadykova@gmail.com

Nawel Achir
email: nawel.achir@supagro.fr

Growth and Metabolism of Lacticaseibacillus casei and Lactobacillus kefiri Isolated from Qymyz, a Traditional Fermented Central Asian Beverage

Keywords: lactic acid bacteria, qymyz (koumiss), mathematical modeling, mare milk

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Fermentation is an ancient process of conservation of animal and plant raw materials in which microorganisms transform perishable foodstuffs into more stable fermented foods. Fermented milk products play an essential role by enhancing not only the stability but also the nutritional and sensorial properties of end-products [START_REF] Rizzoli | Effects of fermented milk products on bone[END_REF]. In addition, the presence of living microorganisms in the final fermented product may have a positive effect on digestion comfort or nutrient assimilation [START_REF] Beena Divya | Probiotic fermented foods for health benefits[END_REF]. Therefore, such products are increasingly demanded by consumers, as they are considered nonhazardous and multifunctional foods. In Central Asia, the growing number of horses and the higher availability of mare milk encourage the improvement of the quality of byproducts obtained from horses for their distribution to the wider population. Qymyz (also known as koumiss or kumis) is a lactic and alcoholic fermented mare milk consumed in Central Asian countries, the production of which includes several steps, such as inoculation, agitation, and incubation. In addition, since it is a traditional drink that has been consumed for centuries for its nutritional and medicinal properties, the production of qymyz helps to safeguard the draft horse breeds of the region.

Initially, and due to a lack of study, the fermentation of mare milk was the result of uncontrolled processes that did not guarantee the quality of the product, leading to sanitary or sensory defects [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF]. Indeed, the composition of the microbiota of the initial mare milk is variable and can differ depending on the region and season of qymyz production. The studies available on qymyz revealed that the main microorganisms responsible for the fermentation were lactic acid bacteria (LAB) and yeasts [START_REF] Ishii | Study on Production and Properties of Kumiss of Herders in Mongolian Dry Steppe[END_REF]. The authors mainly focused on the identification of microorganisms [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF][START_REF] Wu | Bacterial composition and function during fermentation of Mongolia koumiss[END_REF][START_REF] Yao | A Perspective Study of Koumiss Microbiome by Metagenomics Analysis Based on Single-Cell Amplification Technique[END_REF]. Rakhmanova et al. [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF] found 52 lactic acid bacteria and 20 yeasts. Wu et al. [START_REF] Wu | Bacterial composition and function during fermentation of Mongolia koumiss[END_REF] stated that LAB played a major role in the sanitary quality of qymyz by decreasing the pH. Yeasts in qymyz produce ethanol and could contribute to production of its typical aroma [START_REF] Dzialo | Physiology, ecology and industrial applications of aroma formation in yeast[END_REF][START_REF] Mu | Detection and identification of wild yeast in Koumiss[END_REF]. Lastly, Yao et al. [START_REF] Yao | A Perspective Study of Koumiss Microbiome by Metagenomics Analysis Based on Single-Cell Amplification Technique[END_REF] showed that the main LAB of qymyz belonged to the Lactobacillus genus.

Works on qymyz microorganisms' metabolism, particularly on their acid and alcohol production characteristics, which are important for the control of qymyz sensory quality, are scarce. To our knowledge, only Rakhmanova et al. [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF] have studied the effect of fermentation conditions (time and temperature) in cow milk by qymyz strains on the ethanol content and sensory properties. They used an empirical experimental design approach that was useful to provide insights into the effect of fermentation conditions, but that could also be used as a reliable modeling tool to monitor the fermentation conditions. Predictive microbiology is a powerful approach that involves robust mathematical modeling of bacterial growth and metabolite production, and that has been approved as a useful tool for the prevention of food spoilage in Europe and in several non-European countries [START_REF] Leporq | The "Sym'Previus" software, a tool to support decisions to the foodstuff safety[END_REF][START_REF] Sumner | The use of predictive microbiology by the Australian meat industry[END_REF]. This approach is also being increasingly used for beneficial microorganisms such as LAB to represent the evolution of functional properties such as acidification power and other functional metabolites, as a function of fermentation conditions [START_REF] Leroy | A novel area of predictive modelling: Describing the functionality of beneficial microorganisms in foods[END_REF]. Furthermore, studies on the control of products co-fermented with a number of different microorganisms are still at an early stage, due to the complex interactions between species that influence their growth dynamic and metabolite productions [START_REF] Kouamé | Development of a model for the alcoholic fermentation of cocoa beans by a Saccharomyces cerevisiae strain[END_REF][START_REF] Mestres | Modeling mixed fermentation of gowé using selected Lactobacillus plantarum and Pichia kluyveri strains[END_REF]. The results of these works are controlled fermentation products that have higher quality and are safer for consumers. A modeling approach can be very useful in identifying the optimal conditions of the microbial growth rate. Models play an essential role in designing and developing strategies to optimize and control biological processes to ensure their economic viability [START_REF] Infantes | Kinetic model and study of the influence of pH, temperature and undissociated acids on acidogenic fermentation[END_REF]. In multiple culture fermentations, the main challenge is to establish and control the growth conditions so that the desired final ratio of populations of each culture and metabolites is achieved. The main monitored parameters in this regard are temperature, pH, and the inoculation ratio [START_REF] Béal | Comparison of growth, acidification and productivity of pure and mixed cultures of Streptococcus salivarius subsp. thermophilus 404 and Lactobacillus delbrueckii subsp. bulgaricus 398[END_REF]. However, to achieve this goal, important experimental work is needed on the different strains of the product to identify the key fermentation control parameters and characterize their growth as a function of them.

The temperature effect on LAB in qymyz preparation is particularly relevant. Indeed, the usual season of qymyz preparation is summer. However, some farms can produce qymyz from late spring to late fall, by controlling foaling. Consequently, the continental climate of Eurasian countries leads to different temperature conditions, from ~5 • C to ~35 • C, in which qymyz is prepared. Additionally, pasture composition also depends on the seasons, which directly impacts the substrate contents in mare milk available for LAB. The temperature and substrates directly affect the growing rates of LAB and the production rate of their main metabolite, lactic acid, and minor constituents such as ethanol, which affects the quality of qymyz. This results in another puzzle of qymyz preparation according to the preference of consumers, as the main qymyz consumers are practicing Muslims, and a low ethanol level may be preferred. According to Kazakhstani standards, qymyz can be classified as weak, medium, and strong, according to the level of ethanol and lactic acid [START_REF]Committee for Standardization, Metrology and Certification of the Ministry of Energy, Industry and Trade of the Republic of Kazakhstan. ST RK 1004-98[END_REF][START_REF] Kondybayev | Fermented mare milk product (Qymyz, Koumiss)[END_REF].

In this work, we propose the study of two strains of LAB isolated from qymyz. In addition to the description of the growth kinetic of each strain, complete modeling was carried out to capitalize on experimental data and to represent their behavior according to various conditions, with the objective of obtaining better control over qymyz quality.

Materials and Methods

Bacteria Identification

Ten cultures of bacteria isolated from local qymyz in the Almaty region of Kazakhstan were kindly provided by KazNU, Kazakhstan. Identification was performed at the RSE National Center for Biotechnology (Nur-Sultan, Kazakhstan) by sequencing the 16S RNA gene, with the subsequent determination of nucleotide identity. The sequences were deposited in the international GenBank database [START_REF] Clarridge | Impact of 16S rRNA gene sequence analysis for identification of bacteria on clinical microbiology and infectious diseases[END_REF]. Bacteria were stored in MRS broth with 30% glycerol at -80 • C. API 50 CHL (Biomerieux, Marcy-l'Étoile, France) was used to analyze the properties of each strain.

Growth Medium Composition

All experiments were performed using De Man Rogosa and Sharpe (MRS) broth medium (Biomerieux, Marcy-l'Étoile, France). For experiments that required a change in the carbon source (lactose, galactose, or a different amount of glucose), the MRS composition was kept the same using the following reagents:

Bacto peptone-ThermoFisher, Waltham, MA, USA; Glucose, galactose, yeast extract, magnesium sulphate, potassium phosphate dibasic, and ammonium citrate dibasic-Sigma-Aldrich, St. Louis, MO, USA;

Lactose, sodium acetate anhydrous, manganese sulphate 1-hydrate, and sodium di-Hydrogen phosphate-Panreac, Barcelona, Spain.

Growth Experiments 2.3.1. Effect of Temperature

Temperature effect experiments were conducted in Minifor Lambda fermenters (Czech Republic), with a working volume of 800 mL and using an IR radiator to control temperature. Bacteria were grown in temperatures ranging from 5 • C to 50 • C [START_REF] Wu | Bacterial composition and function during fermentation of Mongolia koumiss[END_REF][START_REF] Sumner | The use of predictive microbiology by the Australian meat industry[END_REF][START_REF] Béal | Comparison of growth, acidification and productivity of pure and mixed cultures of Streptococcus salivarius subsp. thermophilus 404 and Lactobacillus delbrueckii subsp. bulgaricus 398[END_REF][START_REF] Munanga | Modeling Lactic Fermentation of Gowé Using Lactobacillus Starter Culture[END_REF][START_REF] Lorenzo | Main Groups of Microorganisms of Relevance for Food Safety and Stability[END_REF][START_REF] Baubekova | Volatile organic compounds profile of Lactobacillus casei and Streptococcus thermophilus in fermented mare milk of Kazakhstan[END_REF][START_REF] Møller | Modelling the biphasic growth of non-starter lactic acid bacteria on starter-lysate as a substrate[END_REF][START_REF] Rajoka | Characterization and anti-tumor activity of exopolysaccharide produced by Lactobacillus kefiri isolated from Chinese kefir grains[END_REF]40,42,45,47, and 50 • C). Bacterial precultures grown overnight were prepared at 35 • C, and the initial population for both bacteria was set at approximately 10 6 CFU•mL -1 . The pH was set to 6.8 and automatically adjusted using sterile 2 M sodium hydroxide for all temperatures. Agitation was set at 240 rpm. MRS broth was used as a medium for the growth of bacteria. Bacterial growth was monitored using an inline near-infrared turbidity sensor (Optek FC20-ASD19-N, Elscolab, Arcueil, France). Fermentations were carried out until reaching stationary phase.

Effect of Carbohydrate Source and pH

These experiments were conducted using Bioscreen C MBR (Labsystems, Helsinki, Finland), which is an automated system that analyzes high-throughput microbial growth curves in aqueous media.

Maximal growth rates at 30 • C at different pH, ranging from 3.5 to 10 (L. casei was grown in pH 4, 5, 6, 6.8, 8, 9, and 10, while L. kefiri in pH 3.5, 4.5, 5.5, 6.5, and 7.5), and with different limiting substrates (glucose, galactose, and lactose), the concentration of which ranged from 0.6 g•L -1 to 60 g•L -1 (0.6, 6, 15, 30, and 60 g•L -1 ), were calculated using the technique described by Augustin et al. [START_REF] Augustin | Estimation of temperature dependent growth rate and lag time of Listeria monocytogenes by optical density measurements[END_REF].

Plate-Counting Methods for Calibration Curve

The ten-fold serial dilution of homogenized samples (0.1 mL) was prepared in sterile 9 g•L -1 NaCl water and plated on the surface of De Man Rogosa and Sharpe agar plates. The plates were incubated at 37 • C for 48 h. The colony-forming unit (CFU) counting was plotted against the turbidity values given by the fermenter sensor and the optical density at 600 nm (OD600) to obtain the proportionality coefficient.

Chemical Analyses

The carbohydrate contents and the production of lactic acid and ethanol were measured by HPLC with separation on an Aminex HPX-87H column (Biorad, Hemel Hempstead, UK). Detection was performed using a refractive index detector for the ethanol and carbohydrates, while an ultraviolet (UV) (210 nm) detector was used for lactic acid detection. Aliquots of 20 µL were injected and elution was performed at 30 • C with 5 mM sulfuric acid at a flow rate of 0.6 mL•min -1 [START_REF] Munanga | Modeling Lactic Fermentation of Gowé Using Lactobacillus Starter Culture[END_REF]. Samples were filtered through a 0.45 µm pore size filter before injection.

Mathematical Modeling 2.5.1. Microorganism Growth

The logistic growth model [START_REF] Delhalle | Les modèles de croissance en microbiologie prévisionnelle pour la maitrise de la sécurité des aliments (synthèse bibliographique)[END_REF] with modification to incorporate the second phase was chosen to describe microbial growth according to Equation (1):

ln[N(t)] =    ln N(0) if t ≤ λ n n ∑ 1 ln N max(n) -ln 1 + N max(n) N max(n-1) -1 e [-µ n (t-λ n )] if t > λ n ( 1
)
where n is the number of phases of bacterial growth; Nt and N max (CFU•mL -1 ) are the values of the microbial population at time t and at the end of the phase, respectively; µ max is the maximal growth rate (h -1 ); and λ is the lag time (h).

The gamma concept model [START_REF] Zwietering | A decision support system for prediction of microbial spoilage in foods[END_REF] was used as a secondary model to describe the impact of temperature and pH on the maximum growth rate (µ max ) according to Equation ( 2)

µ max = µ opt × γ(T) × γ(pH) (2)
with γ values ranging between 0 and 1.

The effect of the pH on µ max was expressed using the cardinal temperature and pH model (CTPM) proposed by [START_REF] Rosso | Convenient Model to Describe the Combined Effects of Temperature and pH on Microbial Growth[END_REF], according to Equation (3):

CM n (X) = X ≤ X min ; 0 (X -X min ) n (X -X max ) X opt -X min n-1 X opt -X min X -X opt -(X -X max ) (n -1)X opt + X min -nX X ≥ X max ; 0 (3)
where X corresponds to environmental factors such as pH and temperature, and the n values are 1 for pH and 2 for temperature.

Limiting Substrate

The Monod equation [START_REF] Monod | The growth of bacterial cultures[END_REF] was chosen to relate the growth rates to different limiting substrates:

µ = µ max S Ks + S + µ max 0 ( 4
)
where µ is the specific growth rate, µ max is the maximum specific growth rate, µ max 0 is the maximum specific growth rate in the absence of the limiting substrate, S is the concentration of the limiting substrate for growth, and Ks is the "half-velocity constant"-the value of S when µ/µ max = 0.5.

Metabolite Production Modeling

The kinetics of metabolite production were set to be directly proportional to bacterial growth, as shown in Equation ( 5), as a simplification of the Luedeking and Piret equation [START_REF] Munanga | Modeling Lactic Fermentation of Gowé Using Lactobacillus Starter Culture[END_REF].

d(metabolite) dt = Y(metabolite) N × µN (5)
where Y (metabolite)/N is the yield for lactic acid and ethanol production over N (the population), and µ is the growth rate.

Model Fitting and Estimation of Model Parameters

Model fitting and the estimation of model parameters were conducted by the least squares minimization technique, using the Solver add-in routine of Microsoft Excel. Standard deviations were calculated using the SolverAid macro of Microsoft Excel. Standard errors of regression lines were calculated using the Data Analysis add-in of Microsoft Excel.

Results and Discussion

Bacterial Identification and General Metabolism Features

The identification of the 10 bacteria isolated from qymyz showed that the majority belonged to the Lactobacillus genus, which is consistent with the findings of Yao et al. [START_REF] Yao | A Perspective Study of Koumiss Microbiome by Metagenomics Analysis Based on Single-Cell Amplification Technique[END_REF]. These strains consisted of six Lacticaseibacillus casei strains and one Lactobacillus kefiri strain. In addition, two bacteria were identified as Enterococcus faecium, which can be found in the gastrointestinal tract of humans and animals, and may also be pathogenic [START_REF] Willems | Global spread of vancomycinresistant Enterococcus faecium from distinct nosocomial genetic complex[END_REF]. One bacterium was identified as Aneurinibacillus migulanus, which was first discovered in soil and has been found to produce an antibiotic against pathogenic Staphylococcus strains [START_REF] Berditsch | The ability of Aneurinibacillus migulanus (Bacillus brevis) to produce the antibiotic gramicidin S is correlated with phenotype variation[END_REF].

For further studies, L. casei strain B7 and L. kefiri strain B2 were chosen (Supplemental File). Figures S1 andS2 show locations of strains on phylogenetic tree, while Tables S1 andS2 provide their 16S RNA sequences. L. casei can be found in ripening Cheddar cheese and in Sicilian green olives [START_REF] Lorenzo | Main Groups of Microorganisms of Relevance for Food Safety and Stability[END_REF], while L. kefiri is mainly found in beer, kefir drinks, and kefir grains [START_REF] Ibrahim | Lactic Acid Bacteria: Lactobacillus spp.: Other Species[END_REF]. Both LAB can be found in qymyz [START_REF] Alexandraki | Chapter 6-The Microbiota of Non-cow Milk and Products[END_REF].

More specifically, L. casei has already been studied in fermented mare milk for its acidity and volatile organic compound production with a predominance of 2-methyl-1propanol and 2-undecanol [START_REF] Baubekova | Volatile organic compounds profile of Lactobacillus casei and Streptococcus thermophilus in fermented mare milk of Kazakhstan[END_REF]. L. kefiri is also called Lentilactobacillus kefiri. The genus Lentilactobacillus is described as comprising species with a slow (lentus) growth rate [START_REF] Zheng | A taxonomic note on the genus Lactobacillus: Description of 23 novel genera, emended description of the genus Lactobacillus Beijerinck 1901, and union of Lactobacillaceae and Leuconostocaceae[END_REF]. The studies on this strain are scarce, and no literature is available about their metabolism. However, this strain is interesting for its ability to survive in gastrointestinal environments and to reduce cholesterol [START_REF] Kim | Dual function of Lactobacillus kefiri DH5 in preventing high-fat-diet-induced obesity: Direct reduction of cholesterol and upregulation of PPAR-α in adipose tissue[END_REF].

According to the API 50 CHL test, L. casei was able to consume a wide range of substrates, which included adonitol, galactose, glucose, fructose, mannose, sorbose, rhamnose, mannitol, sorbitol, N-acetyl glucosamine, arbutin, esculin, salicin, cellobiose, maltose, lactose, sucrose, trehalose, inulin, melezitose, B-gentiobiose, D-turanose, D-tagatose, and gluconate. On the other hand, L. kefiri was able to consume only L-arabinose, ribose, galactose, glucose, fructose, maltose, lactose, melibiose, and gluconate, which is consistent with the description of Kandler and Kunath [START_REF] Kandler | Lactobacillus kefir sp.nov., a component of the microflora of Kefir[END_REF].

To compare their growth kinetics, the first set of experiments was carried out on a glucose medium in controlled pH conditions.

Growth Characteristic of L. casei and L. kefiri 3.2.1. Microorganism Growth Modeling

The primary models fitted the experimental data for both L. casei and L. kefiri (Figure 1) well. L. casei exhibited monophasic growth, while L. kefiri experienced biphasic growth at 37 • C. However, L. kefiri growth could be assumed to be monophasic at temperatures below 15 • C. The specific growth rates were 0.68 h -1 for L. casei and 0.20 h -1 during the first phase and 0.35 h -1 during the second phase for L. kefiri at 37 • C.

Microorganism Growth Modeling

The primary models fitted the experimental data for both L. casei and L. kefiri (Figure 1) well. L. casei exhibited monophasic growth, while L. kefiri experienced biphasic growth at 37 °C. However, L. kefiri growth could be assumed to be monophasic at temperatures below 15 °C. The specific growth rates were 0.68 h -1 for L. casei and 0.20 h -1 during the first phase and 0.35 h -1 during the second phase for L. kefiri at 37 °C.

Figure 1.

Growth of L. kefiri and L. casei in glucose MRS broth at 37 °C, pH 6.8. Experimental data (symbols) and predicted data (continuous lines).

Modeling Effect of pH and Temperature for L. casei

The effect of pH and temperature on the growth rate is presented in Figure 2 for L. casei. The global trends of these effects on μmax are in accordance with the usual bell shape. The results show that the optimal pH and temperature for L. casei equaled 6.69 ± 0.20 and 38.63 ± 0.32 °C, respectively, with a μopt of 0.66 ± 0.01 h -1 . The growth rate of L. casei was similar to reports in another study, where two strains of L. casei had a growth rate of 0.7 h -1 [START_REF] Alvarez | Kinetic analysis and mathematical modeling of growth and lactic acid production of Lactobacillus casei var. rhamnosus in milk whey[END_REF].

Modeling Effect of pH and Temperature for L. casei

The effect of pH and temperature on the growth rate is presented in Figure 2 for L. casei. The global trends of these effects on µmax are in accordance with the usual bell shape. The results show that the optimal pH and temperature for L. casei equaled 6.69 ± 0.20 and 38.63 ± 0.32 • C, respectively, with a µ opt of 0.66 ± 0.01 h -1 . The growth rate of L. casei was similar to reports in another study, where two strains of L. casei had a growth rate of 0.7 h -1 [START_REF] Alvarez | Kinetic analysis and mathematical modeling of growth and lactic acid production of Lactobacillus casei var. rhamnosus in milk whey[END_REF]. L. kefiri had a slower growth rate, and its growth kinetics consisted of two phases (Figure 1). The biphasic growth of LAB could be explained by the ability to consume the end products of the metabolism of the main substrates that can be produced from the strain itself or by other microorganisms [START_REF] Møller | Modelling the biphasic growth of non-starter lactic acid bacteria on starter-lysate as a substrate[END_REF]. However, in our case, L. kefiri was grown in L. kefiri had a slower growth rate, and its growth kinetics consisted of two phases (Figure 1). The biphasic growth of LAB could be explained by the ability to consume the end products of the metabolism of the main substrates that can be produced from the strain itself or by other microorganisms [START_REF] Møller | Modelling the biphasic growth of non-starter lactic acid bacteria on starter-lysate as a substrate[END_REF]. However, in our case, L. kefiri was grown in monoculture and in a medium containing one substrate, which indicates that biphasic growth had other causes.

In particular, the strains of the genus Lentilactobacillus, to which L. kefiri belongs, were reported to be able to consume lactate [START_REF] Zheng | A taxonomic note on the genus Lactobacillus: Description of 23 novel genera, emended description of the genus Lactobacillus Beijerinck 1901, and union of Lactobacillaceae and Leuconostocaceae[END_REF]. The parameters of each phase are given in Table 1. The second phases of the temperatures below 15 • C were not estimated as the differences were negligible. Even though each fermentation was started using ~10 6 bacteria, only 10 7 bacteria were detected by the sensor. No growth was observed at 45 • C. From the table, we can see that the highest growth rate of the second growth phase occurs at 40 • C, while the highest growth rate for the first phase occurs at 30 • C. The change in phase cannot be totally explained by population since the second phase could begin from 3•× 10 8 to 7•× 10 8 CFU•mL -1 . However, it can be noticed that the second phase began at lower populations in extreme conditions, below 20 • C or higher than 37 • C. For the same conditions, the final population after phase one was lower than that obtained from 27-35 • C. The ratio of µ2/µ1 is also presented in Table 1. The µ of the second phase increased, while the µ of the first phase decreased as a function of the fermentation temperature.

The high growth rate of the second phase at 40 • C was short, as after reaching ~2 × 10 9 CFU•mL -1 a coagulation of the media was observed, which hindered further detection of population change by the fermenter sensor. The microscopic analysis of the coagulated media revealed a shortening of rods, which, according to [START_REF] Kandler | Lactobacillus kefir sp.nov., a component of the microflora of Kefir[END_REF], is characteristic of L. kefiri, as it has the ability to form short rods (3 nm) or long filaments (15 nm). Morphological change could affect the metabolism of L. kefiri and explain its biphasic growth [START_REF] Shen | Morphological plasticity of bacteria-Open questions[END_REF]. Rajoka et al. [START_REF] Rajoka | Characterization and anti-tumor activity of exopolysaccharide produced by Lactobacillus kefiri isolated from Chinese kefir grains[END_REF] found that L. kefiri is able to attach to the grain surface in Brazilian kefir and is important for the production of the kefiran polymer present in the kefir grain's structure, which could explain the coagulation of the media.

Because of its ability to modulate its metabolism depending on the temperature, the growth pattern of L. kefiri can be unpredictable at high temperatures, as shown by the fact that patterns that had no first or second phases were observed in some trials. The parameters of 37 • C fermentation presented in Table 1 are taken from a period of fermentation that had clear first and second phases. Taking into consideration the specifics of morphology and growth in the second phase, pH and temperature effects were modeled only for the first phase of L. kefiri growth. In addition, during the fermentation of qymyz, it is highly probable that only the first phase may be expressed due to its global lower fermentation rate (Figure 1). The effect of pH and temperature on the first phase of L. kefiri can be seen in Figure 3. eters of 37 °C fermentation presented in Table 1 are taken from a period of fermentation that had clear first and second phases. Taking into consideration the specifics of morphology and growth in the second phase, pH and temperature effects were modeled only for the first phase of L. kefiri growth. In addition, during the fermentation of qymyz, it is highly probable that only the first phase may be expressed due to its global lower fermentation rate (Figure 1). The effect of pH and temperature on the first phase of L. kefiri can be seen in Figure 3. The results show that the optimal pH and temperature for L. kefiri equaled 5.93 ± 0.08 and 33.15 ± 0.53 °C, respectively, with a μopt of 0.29 ± 0.01 h -1 . No data on the growth rate of L. kefiri were found in the literature. However, all the results on LAB are consistent with the works of Rakhmanova et al. [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF], who found that the optimal fermentation parameters for the maximum score of the sensory evaluation are a fermentation temperature of 36 °C, and a fermentation time of 16 h, which is consistent with Figure 1. The results show that the optimal pH and temperature for L. kefiri equaled 5.93 ± 0.08 and 33.15 ± 0.53 • C, respectively, with a µ opt of 0.29 ± 0.01 h -1 . No data on the growth rate of L. kefiri were found in the literature. However, all the results on LAB are consistent with the works of Rakhmanova et al. [START_REF] Rakhmanova | Isolation and identification of microorganisms in Kazakhstan koumiss and their application in preparing cow-milk koumiss[END_REF], who found that the optimal fermentation parameters for the maximum score of the sensory evaluation are a fermentation temperature of 36 • C, and a fermentation time of 16 h, which is consistent with Figure 1.

Metabolite Production

Lactic acid and ethanol were the main metabolites produced by L. casei and L. kefiri. Their production, monitored by HPLC, is plotted against population in Figure 4. Lactose was chosen as a carbon source because it is the main carbohydrate in milk. Our results show that L. casei and L. kefiri have opposite fermentation routes. L. casei has a homofermentative pathway, while L. kefiri has a heterofermentative pathway.

Fermentation 2022, 8, 367 9 of 12

Metabolite Production

Lactic acid and ethanol were the main metabolites produced by L. casei and L. kefiri. Their production, monitored by HPLC, is plotted against population in Figure 4. Lactose was chosen as a carbon source because it is the main carbohydrate in milk. Our results show that L. casei and L. kefiri have opposite fermentation routes. L. casei has a homofermentative pathway, while L. kefiri has a heterofermentative pathway. The lactic acid production rate was 7.36 ו10 -9 mg•CFU -1 for L. casei. For L. kefiri, the lactic production rate was 7.64•× 10 -9 mg•CFU -1 , while the ethanol production rate was 2.90•× 10 -9 mg•CFU -1 . The lactic acid production rate was 7.36 ו10 -9 mg•CFU -1 for L. casei. For L. kefiri, the lactic production rate was 7.64•× 10 -9 mg•CFU -1 , while the ethanol production rate was 2.90•× 10 -9 mg•CFU -1 .

Resulting Parameters of Models

The model parameters of Lacticaseibacillus casei and Lactobacillus kefiri are given in Table 2. The Monod model values were hard to obtain, as there was growth in the media that did not contain any milk carbohydrates. As can be seen from the table, L. casei has the highest growth rate and Ks values in glucose. L. kefiri, on the other hand, has similar growth rates in all media, but its Ks value is the highest for lactose. These results show that L. casei, after breaking down lactose, first consumes galactose and then glucose, while for L. kefiri there is not much preference between glucose and galactose.

Conclusions

Qymyz is a mare milk fermented with LAB and yeasts. LAB enable yeast metabolism as the main yeast in qymyz does not metabolize the lactose but glucose and galactose, produced thanks to the LAB beta galactosidases [START_REF] Montanari | Saccharomyces unisporus as the principal alcoholic fermentation microorganism of traditional koumiss[END_REF]. Towards the later stages of fermentation, when the acidity of qymyz becomes very high, only the most acid-tolerant bacteria and (most probably) yeasts can survive.

Our results showed that L. casei had, in general, a faster growth rate than L. kefiri. Temperature-wise, L. casei prefers higher temperatures of around 38 • C, while L. kefiri prefers different temperatures during its two phases, with 33 • C being the optimal temperature for the first phase. The rate of the second phase was negligible below 20 • C but increased significantly with temperature, surpassing that of the first phase to 37 • C, but only after ~35+ hours. Therefore, during qymyz production, it is more probable that L. kefiri only expresses its first growth phase. The optimal pH of L. kefiri was slightly lower than L. casei. They also had opposite fermentation pathways: L. casei had a homofermentative pathway, while L. kefiri was heterofermentative.

These results are helpful in identifying which conditions should be used when preparing qymyz. The differences between these species will help in the preparation of starters for different purposes. L. casei could be used to reduce the alcohol content of qymyz, while the slow growth rate of L. kefiri could be beneficial in slowing down the yeast fermentation and thus enabling a more balanced qymyz in terms of lactic acid and ethanol. However, after a long storage period, the final product could reach a higher ethanol content as both L. kefiri and yeast will increase the ethanol content. L. casei is a homofermentative bacteria, so only yeast will be responsible for an ethanol increase. The results show that it is more optimal for L. kefiri to be grown with lactose-fermenting yeasts as its growth rate will delay the growth of lactose nonfermenting yeasts. L. casei can compete with lactose-fermenting yeasts for lactose, so its growth with lactose non-fermenting yeasts could be optimal. Speaking of optimal conditions, it has to be noted that industrial production often requires suboptimal temperatures to reduce production costs. In which case, optimal balance should be found between fermentation temperature and fermentation duration to achieve the desired qymyz quality.

To conclude, by studying the growth and metabolism kinetics of two different LAB of qymyz, this paper provides insights into qymyz fermentation monitoring. Strategies to modulate the final qymyz quality rely not only on the choice of lactic acid bacteria, but also on yeast strains. Modeling both yeast and bacteria will facilitate understanding of the interaction between these microorganisms. For example, growth according to the model in a co-culture implies no interaction between microorganisms. In contrast, a decrease or increase in growth rate or metabolite production indicates inhibition or stimulation between microorganisms. As a result, this work is the first step in qymyz fermentation modeling. Additionally, the second phase of L. kefiri growth still requires further study. The effect of medium coagulation on qymyz quality in the long term is unknown.

Figure 1 .

 1 Figure 1. Growth of L. kefiri and L. casei in glucose MRS broth at 37 • C, pH 6.8. Experimental data (symbols) and predicted data (continuous lines).

FermentationFigure 2 .

 2 Figure 2. Effect of environmental factors on growth rate of L. casei in glucose MRS broth. (a) pH at 30 °C and (b) temperature. Experimental data (symbols) and predicted data (dashed lines). 3.2.3. Modeling Effect of pH and Temperature for L. kefiri

Figure 2 .

 2 Figure 2. Effect of environmental factors on growth rate of L. casei in glucose MRS broth. (a) pH at 30 • C and (b) temperature. Experimental data (symbols) and predicted data (dashed lines).

3. 2 . 3 .

 23 Modeling Effect of pH and Temperature for L. kefiri

Figure 3 .

 3 Figure 3. Effect of environmental factors on first-phase growth rate of L. kefiri in glucose MRS broth. (a) pH at 30 °C and (b) temperature. Experimental data (symbols) and predicted data (dashed lines).

Figure 3 .

 3 Figure 3. Effect of environmental factors on first-phase growth rate of L. kefiri in glucose MRS broth. (a) pH at 30 • C and (b) temperature. Experimental data (symbols) and predicted data (dashed lines).

Figure 4 .

 4 Figure 4. Change in products during fermentation of (a) L. casei and (b) L. kefiri in lactose MRS broth at different temperatures. L. casei-25 °C; 37 °C (shaded symbols). L. kefiri-30 °C; 38 °C (shaded symbols).

Figure 4 .

 4 Figure 4. Change in products during fermentation of (a) L. casei and (b) L. kefiri in lactose MRS broth at different temperatures. L. casei-25 • C; 37 • C (shaded symbols). L. kefiri-30 • C; 38 • C (shaded symbols).

Table 1 .

 1 Parameters of two phases of L. kefiri growth at different temperatures.

			Phase 1			Phase 2
	Temperature ( • C)	λ (h)	µ max (h -1 )	N max (CFU•mL -1 )	λ h	µ max (h -1 )	N max (CFU•mL -1 )	µ2/µ1
	20	26.30 ± 0.38	0.08 ± 0.01	3.83•× 10 8	55.17 ± 0.01	0.02 ± 0.00	9•× 10 8	0.28
	27	10.55 ± 0.09	0.20 ± 0.01	6.10•× 10 8	44.33 ± 0.01	0.03 ± 0.00	1.32•× 10 9	0.15
	30	1.91 ± 0.08	0.25 ± 0.01	7.60•× 10 8	28.66 ± 0.01	0.07 ± 0.01	1.32•× 10 9	0.28
	35	3.69 ± 0.09	0.23 ± 0.01	7.16•× 10 8	35.83 ± 0.01	0.07 ± 0.01	1.60•× 10 9	0.30
	37	11.57 ± 0.11	0.20 ± 0.01	4.10•× 10 8	46.09 ± 0.10	0.35 ± 0.01	2.67•× 10 9	1.75
	40	0.00 ± 0.00	0.16 ± 0.01	3.28•× 10 8	42.49 ± 0.47	0.64 ± 0.20	1.19•× 10 9	4.00

Values are expressed as mean ± S.D. λ-lag phase duration; µ max (h -1 )-growth rate.

Table 2 .

 2 Model parameters according to growth model, Monod equation, and Luedeking-Piret equation.

	Parameters	Lacticaseibacillus casei (Value ± SD/SE)	1st Phase of Lactobacillus kefiri (Value ± SD/SE)
	pH min	3.84 ± 0.12	3.35 ± 0.12
	pH max	10.33 ± 0.18	7.64 ± 0.05
	pH opt	6.69 ± 0.20	5.93 ± 0.08
	T min ( • C)	9.66 ± 1.18	8.10 ± 1.27
	T max ( • C)	48.48 ± 0.13	44.94 ± 0.31
	T opt ( • C)	38.63 ± 0.32	33.15 ± 0.53
	µ opt (h -1 ) Glucose µ opt (h -1 ) Glucose Ks (g L -1 )	0.66 ± 0.01 0.49 ± 0.01 0.43 ± 0.07	0.29 ± 0.01 0.26 ± 0.01 0.21 ± 0.03
	Galactose µ opt (h -1 ) Galactose Ks (g L -1 )	0.47 ± 0.01 0.19 ± 0.05	0.27 ± 0.01 0.18 ± 0.05
	Lactose µ opt (h -1 ) Lactose Ks (g L -1 )	0.43 ± 0.01 0.12 ± 0.04	0.24 ± 0.01 0.72 ± 0.57
	Lactic Acid Production * (10 -9 mg•CFU -1 )	7.36 ± 0.43	7.64 ± 0.54
	Ethanol Production* (10 -9 mg•CFU -1 )	0	2.90 ± 0.15

pH and temperature parameters were obtained in MRS broth containing glucose as carbon source. * Values are expressed as mean ± S.E.

Funding: This research received no external funding.

Informed Consent Statement: Not applicable.

Data Availability Statement: Not applicable.

Acknowledgments:

The work was supported by the Abaï-Verne joint program of the Ministry of Education and Science of the Republic of Kazakhstan and the French Ministry for Europe and Foreign Affairs. We also thank A. Baubekova from KazNU, Almaty, for providing the bacterial strains.

Supplementary Materials:

The following supporting information can be downloaded at: https:// www.mdpi.com/article/10.3390/fermentation8080367/s1, Figure S1: B2 (2_antigen) location on phylogenetic tree; Figure S2: B7 (7_antigen) location on phylogenetic tree; Table S1: B2-16S RNA sequence and BLAST identification results; and Table S2:

Conflicts of Interest:

The authors declare no conflict of interest.


Marvin Lavechin

Marianne Métais

Hadrien Titeux

Alodie Boissonnet

Jade Copet

Morgane Rivière

Elika Bergelson

Alejandrina Cristia

Emmanuel Dupoux

Hervé Bredin

Marvin Lavechin ⋆

Marianne Métais ⋆

Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimation

Keywords: voice activity detection, speech-to-noise ratio, speech clarity, acoustic environment, reverberation

come   L'archive ouverte pluridisciplinaire

INTRODUCTION

Robustness to degraded acoustic environments is a critical factor limiting the impact and adoption of speech technologies. Numerous sources of variations in the audio can degrade or hide the signal of interest and impact the performance of automatic speech processing systems. Be it automatic speech recognition (ASR) [START_REF] Giri | Improving speech recognition in reverberation using a room-aware deep neural network and multi-task learning[END_REF][START_REF] Kinoshita | A summary of the reverb challenge: state-of-the-art and remaining challenges in reverberant speech processing research[END_REF][START_REF] Gamper | Predicting word error rate for reverberant speech[END_REF], speaker identification/diarization [START_REF] Zhao | Robust speaker identification in noisy and reverberant conditions[END_REF][START_REF] Ryant | The third dihard diarization challenge[END_REF], or speaker localization [START_REF] Chakrabarty | Broadband doa estimation using convolutional neural networks trained with noise signals[END_REF], most systems exhibit a loss of performance when applied in noisy or reverberant conditions.

While speech processing systems are being improved to handle degraded acoustic environments [START_REF] Ko | A study on data augmentation of reverberant speech for robust speech recognition[END_REF][START_REF] Donahue | Exploring speech enhancement with generative adversarial networks for robust speech recognition[END_REF][START_REF] Narayanan | Learning mask scalars for improved robust automatic speech recognition[END_REF], little work ⋆ M. Lavechin and M. Métais equally contributed to this work. This work was granted access to the HPC resources of GENCI-IDRIS under the allocation 2022-AD011012554. It also benefited from the support of ANR-16-DATA-0004 ACLEW, ANR-17-EURE-0017, ANR-19-P3IA-0001; the J. S. Mc-Donnell Foundation; and ERC ExELang grant no 101001095.

has been devoted to automatically predict the properties of the acoustic environment. A proposed approach involves using synthetic audio generated by applying an audio transformation of interest (e.g., reverberation). A neural network is then trained to extract the 'strength' of this audio transformation. This approach is most commonly used to develop systems that predict room acoustic measures like speech clarity (C 50 ), reverberation time (T 60 ) or direct-to-reverberant ratio (DRR) [START_REF] Eaton | The ace challenge-corpus description and performance evaluation[END_REF][START_REF] Peso Parada | A single-channel non-intrusive c50 estimator correlated with speech recognition performance[END_REF][START_REF] Xiong | Exploring auditory-inspired acoustic features for room acoustic parameter estimation from monaural speech[END_REF][START_REF] Nicholas | Impulse response data augmentation and deep neural networks for blind room acoustic parameter estimation[END_REF][START_REF] Gamper | Blind c50 estimation from singlechannel speech using a convolutional neural network[END_REF]. In practice, these values can be estimated directly from the room impulse response (RIR, the recording of a high-energy and bursty sound, such as a pistol shot or a balloon popping). However, in most cases, RIRs are not available, and we need to estimate the values of interest from the observed single channel audio recording. A similar approach has been adopted in [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] to automatically estimate the frame-level speech-to-noise ratio (SNR). The authors evaluate the performance of their system on synthetic data, but not on real data. In practice, real SNRs are not available making it impossible to compare the estimated values to the real ones. Thus, it remains unclear if such a system can generalize to real data.

Given the high interplay between noise and reverberation (the SNR may be influenced by how noise and speech sources reverberate, and it is harder to obtain reliable estimates of reverberation parameters in low SNR conditions [START_REF] Heinrich Löllmann | Comparative study of single-channel algorithms for blind reverberation time estimation[END_REF][START_REF] Eaton | Estimation of room acoustic parameters: The ace challenge[END_REF]), can we design a system that tackles both tasks simultaneously? This is one of the questions we address in this work. Our approach is closest to [START_REF] Looney | Joint estimation of acoustic parameters from single-microphone speech observations[END_REF] who proposes to train a neural network for jointly estimating room acoustic parameters and the utterance-level SNR. However, the authors use a restrained set of noise segments which cast doubts on the ability of their model to generalize to unseen noises. More importantly, they do not evaluate their system with respect to the SNR, and they do not address the question of whether the proposed multitask regime is beneficial for the estimation performance.

We propose Brouhaha, a model jointly trained on the speech/non-speech classification task and the SNR and C 50 regression tasks. Our model is trained on 1, 250 hours of synthetic audio generated from clean speech segments con- With probability p RIR = 0.9, the clean speech segment (marked as S) contaminated with silence (marked as NS) s 1 is convolved with a randomly drawn impulse response RIR s . n 1 → n 2 : With probability p RIR , the randomly drawn noise segment n 1 is convolved with a randomly drawn impulse response RIR n . s 2 + n 2 → s 3 : The reverberated speech segment s 2 and the reverberated noise segment n 2 are added together to obtain a Speech-to-Noise Ratio (SNR) randomly drawn between 0 and 30 dB. C 50 is computed as the ratio of early (0 to 50 ms) and late (50 ms to the end of the response) energies of the room impulse response RIR s . As noises can have a wide dynamic range and the utterance-level SNR captures only global information about the noise level, we recompute SNRs using a 2-second long sliding window shifted every 10 ms over s 2 and n 2 . To simulate different locations for the speech and noise sources, we used two randomly RIRs (as opposed to only one). This design choice aimed at forcing the model to rely solely on the reverberation characteristics of speech, disregarding the reverberation of noise, in order to predict the C 50 measure. Labels obtained via this pipeline include: speech/non-speech (frame-level), C 50 measure of RIR s (utterance-level), and SNR (frame-level).

taminated with silence, noise and reverberation. We first demonstrate that the proposed multi-task regime is beneficial and compare the performance of Brouhaha against state-ofthe-art systems. We then apply Brouhaha on real data (under naturally noisy and reverberant conditions) to: 1) analyze the error patterns of a speaker diarization system (pyannote.audio [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF]); and 2) assess the reliability of an ASR system (Whisper [START_REF] Radford | Robust speech recognition via large-scale weak supervision[END_REF]). In addition to showing how Brouhaha can be used, these experiments constitute evidence that our system is applicable to real data.

Beyond the scientific interest of exploring the effectiveness of the proposed multi-task training regime and assessing the applicability of the method on real data after training on synthetic ones, we believe our work has a strong practical interest. Unlike previous work [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF][START_REF] Looney | Joint estimation of acoustic parameters from single-microphone speech observations[END_REF], Brouhaha can be applied to any audio regardless of whether it contains speech, non-speech or both. By using our system, there is no requirement to implement a preliminary voice activity detection system prior to obtaining SNR and C50 values. We believe such advancement, in addition to a simple user interface (one python command!), significantly aids empowering researchers who may not possess expertise in speech processing or machine learning to make the most out of speech technology.

AUDIO CONTAMINATION PIPELINE

We start from: 1) a set of clean speech segments that will be contaminated; 2) a set of noise segments used to simulate noisy conditions; and 3) a set of RIRs to simulate reverberation. The clean speech segments are contaminated following the steps presented in Figure 1, which we will not repeat here.

MULTI-TASK TRAINING

We tackled the voice activity detection problem as a classification problem where, for each 16-ms frame, the expected output is 1 if there is speech, 0 otherwise. C 50 and SNR estimations were tackled as regression problems where, for each 16-ms frame, the expected output is the actual C 50 or SNR in dB. We tackled the C 50 estimation at the frame level during training -despite the label being at the utterance level -to allow the model to return smoother transitions when a change in C 50 is detected at inference time.

At training time, short fixed length sub-sequences are drawn randomly from the training set and gradient-descent is used to minimize the multi-task loss function L = L VAD + L C50 +L SNR , where L VAD is the binary cross-entropy loss, and L C50 and L SNR are mean squared error (MSE) losses. Before training, L C50 and L SNR are normalized by their maximum value (computed over 10 batches) to ensure all three losses lie between 0 and 1. We computed L SNR only over speech frames as the SNR is not defined on non-speech frames.

EXPERIMENTS

Datasets

Our audio contamination pipeline requires three types of audio data: 1) clean speech segments; 2) noise segments; and 3) RIRs. A pretrained VAD model [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF] was applied to find non-speech segments in 1000 hours of clean read-speech, retrieved from the LibriSpeech [START_REF] Panayotov | Librispeech: an asr corpus based on public domain audio books[END_REF]. Predicted non-speech segments were extended with silence to obtain a ratio of approximately 30 % of non-speech. We used noise segments from AudioSet [START_REF] Jort F Gemmeke | Audio set: An ontology and human-labeled dataset for audio events[END_REF] and discarded human vocalizations. We also downsampled music segments from 38 % to 5 %, leading to a total of 1500 hours of noise segments. Finally, 385 impulse responses were obtained from EchoThief [START_REF] Warren | Echothief impulse response library[END_REF] and the MIT Acoustical Reverberation Scene [START_REF] Traer | Statistics of natural reverberation enable perceptual separation of sound and space[END_REF] datasets. We used the same train/dev/test split originally proposed in LibriSpeech. Noise segments and impulse responses were randomly split into 80 %, 10 % and 10 % for the training, development and test set, respectively. All files used in this paper consist of 16-kHz single-channel recordings.

Evaluation metrics

We evaluated Brouhaha performance on the VAD task using the F-score between precision and recall, such as implemented in pyannote.metrics [START_REF] Bredin | pyannote.metrics: a toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems[END_REF]. SNR and C 50 predictions were evaluated using the mean absolute error (MAE) at the frame level. Since SNR is not defined on non-speech frames, the SNR was only evaluated across speech frames.

Architecture, optimization and training procedure

The model consists of SincNet (using the configuration in [START_REF] Ravanelli | Speaker recognition from raw waveform with sincnet[END_REF]), followed by a stack of bidirectional long short-term memory (LSTM) and feed-forward layers. Finally, we have three parallel layers: one classification layer (with softmax activation) that returns the predicted probability of speech, and two regression layers that return the predicted SNR and C 50 (with sigmoid activation parametrized between -15 and 80 dB for the SNR, and -10 and 60 dB for the C 50 ).

We trained 144 different architectures across different sets of hyperparameters, varying the duration of the input sequences: 4, 6, 8, or 10 seconds; the batch size: 32, 64, or 128 sequences; the size of the hidden LSTM layers: 128 or 256; the number of LSTM layers: 2 or 3; and the dropout proportion: 0, 30 or 50 %. The best architecture was trained with 6-s segments, a batch size of 64 sequences, 3 LSTM layers of size 256, and a dropout proportion of 50 %. The best architecture was selected on the validation metric: an average of the VAD F-score, SNR and C 50 MAEs, with the latter two normalized by the maximum error to balance the contribution of each term.

RESULTS

The effect of multi-task training

✓ 93.5 -- -- ✓ -- 4.3 -- ✓ -- -- 4.2
Table 1 shows performance obtained by models trained to solve either one, two or three of the proposed tasks (VAD, SNR, C 50 ). All models shared the same set of hyperparameters, only the dimension of the output layer differed.

Results indicate that the multi-task training regime is beneficial: the model trained simultaneously on the three tasks obtained better performance than models trained on two tasks which themselves obtained better performance than models trained on a single task. The largest performance gain is observed for the C 50 estimation, with a decrease of 0.7 dB in terms of MAE between the single-task and the three-tasks training regime. These results seem to show that sharing weights during training helps better solve the proposed three tasks. Not only does using a single model provide a performance gain, but it is also more convenient and computationally efficient.

Voice activity detection

Table 2 shows voice activity detection performance obtained by Brouhaha and a state-of-the-art system (pyannote.audio [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF]). We consider two evaluation sets: 1) our test set made of unseen synthetic audio data (referred as 'synthetic' in the table); and 2) BabyTrain [START_REF] Lavechin | An opensource voice type classifier for child-centered daylong recordings[END_REF], a corpus of highly naturalistic child-centered recordings (referred as 'real' in the table). Specifically, BabyTrain recordings are acquired via child-worn microphones as they go about their everyday activities and are widely used in language acquisition research [START_REF] Lavechin | Reverse engineering language acquisition with childcentered long-form recordings[END_REF]. Child-centered recordings are notoriously challenging for speech processing systems as they contain spontaneous Table 2: Voice activity detection F-score obtained by Brouhaha and pyannote.audio pretrained system [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF]. Numbers are reported on synthetic data (our test set) and on real data (BabyTrain [START_REF] Lavechin | An opensource voice type classifier for child-centered daylong recordings[END_REF]).

Data type System

VAD F-score (%) synthetic Brouhaha (ours) 93.7 pyannote.audio [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF] 89.0 real Brouhaha (ours) 77.2 pyannote.audio [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF] 80.8

and overlapping speech, and a wide variety of noisy and reverberant conditions. Results show a strong advantage for Brouhaha over pyannote.audio on unseen synthetic data (4.7 % absolute difference in terms of F-score) . This indicates that, on highly noisy and reverberant synthetic audio, our system is competitive on the VAD task. Admittedly, Brouhaha has an advantage over pyannote.audio as the latter has not been trained on synthetically noisy and reverberant audio. Turning to a performance comparison on real data, numbers reveal that pyannote.audio outperforms Brouhaha by a 3.6 % absolute difference in terms of F-score. This result suggests that training a VAD system on LibriSpeech [START_REF] Panayotov | Librispeech: an asr corpus based on public domain audio books[END_REF] contaminated with reverberation and additive noise might not be optimal, and this is despite the precautions taken in simulating challenging noisy and reverberant conditions. Nonetheless, LibriSpeech is currently the only source of clean speech available in sufficiently large quantities to run our audio contamination pipeline and obtain SNR and C 50 labels.

Speech-to-noise ratio estimation

Table 3: Mean absolute error on the SNR estimation task computed on unseen synthetic data (our test set). All predicted and gold SNRs are brought back to the [-15, 30] dB range as done in [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF]. For a given speech utterance, the heuristic estimates the noise (resp. speech) power as the mean power of non-speech (resp. speech) frames within a 6-s window centered around each annotated speech frame (defaulting to the average SNR when no non-speech frames were found within the 6-s window).

System SNR MAE (dB)

Brouhaha (ours) 2.3 Heuristic 8.4 Li et al. [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] 12.5

Table 3 shows MAE performance on the SNR estimation task computed on our test set made of unseen synthetic audio data for: 1) Brouhaha; 2) a heuristic using the oracle VAD that es-timates the noise (resp. speech) as the mean power of neighboring non-speech (resp. speech) frames; and 3) the system proposed in [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] (a 4-layer LSTM trained from mel frequency cepstral coefficients). Results indicate that Brouhaha is better at estimating the frame-level SNR than our heuristic, with an absolute difference of 6.1 dB in terms of MAE (note that both systems use a 6-s window as input, and that our heuristic requires oracle VAD boundaries). Surprisingly, our heuristic performs better than the system proposed in [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] with a 4.1 dB absolute difference in terms of MAE. This indicates that [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] struggles generalizing to unseen noise or to reverberant environments. Unfortunately, we could not compare systems on the test used in [START_REF] Li | Frame-level signal-to-noise ratio estimation using deep learning[END_REF] as the latter has not been publicly released.

C 50 estimation

Fig. 2: C 50 estimation. Real C 50 against C 50 predicted by Brouhaha on 1000 utterances from the BUT Speech@FIT Reverb dataset [START_REF] Szöke | Building and evaluation of a real room impulse response dataset[END_REF].

We ran Brouhaha on the BUT Speech@FIT Reverb dataset [START_REF] Szöke | Building and evaluation of a real room impulse response dataset[END_REF]. This dataset consists of LibriSpeech test-clean utterances retransmitted by a loudspeaker in 5 different rooms. For each room, the speaker was placed on 5 positions on average and retransmitted utterances were recorded with 31 microphones. RIRs were measured multiple times for each speaker position. Here, we compare the real C 50 (averaged over between 1 and 9 duplicated RIR measures) to the C 50 predicted by Brouhaha on 1000 randomly drawn utterances.

Figure 2 shows a strong correlation between the real and the predicted C 50 , with a R 2 of .85 and a mean average error of 1.1 dB. We would have liked to compare the performance of our system on the C 50 estimation task with other systems, but we could not find any open-source pre-trained C 50 estimators despite extensive research in this area [START_REF] Peso Parada | A single-channel non-intrusive c50 estimator correlated with speech recognition performance[END_REF][START_REF] Xiong | Exploring auditory-inspired acoustic features for room acoustic parameter estimation from monaural speech[END_REF][START_REF] Gamper | Blind c50 estimation from singlechannel speech using a convolutional neural network[END_REF].

Investigating speaker diarization errors

We ran a pretrained pyannote.audio speaker diarization pipeline [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF] on the VoxConverse dataset [START_REF] Joon | Spot the Conversation: Speaker Diarisation in the Wild[END_REF] and evaluated its performance at Brouhaha frame resolution (16 ms). Each frame can either be classified as: 1) missed detection (when the speaker diarization pipeline incorrectly classifies a speech frame as non-speech): 2) false alarm (the other way around); 3) speaker confusion (when a speech frame is assigned to the wrong speaker); or 4) correct. Figure 3 focuses on speaker confusion (but the same pattern holds for missed detections) and shows the distribution of predicted SNR (left) and C 50 (right) depending on whether the speech frame was assigned to the correct speaker. There is a clear trend as far as SNR is concerned: pyannote.audio is much more likely to confuse speakers in low (predicted) SNR regions. Similarly, the accuracy degrades significantly as we get closer to the lowest predicted C 50 values.

Exploring the errors made by a pretrained system can provide valuable insights for developing effective strategies. In our case, one might devise strategies to address the issue of high speaker confusion in low SNR conditions: increasing the weight of low-SNR sequences in the training loss, or running speech enhancement algorithms on low SNR areas for instance.

Fig. 3: Investigating speaker diarization errors. Distribution of SNR (left) and C 50 (right) predicted by Brouhaha as a function of whether a pretrained speaker diarization system [START_REF] Bredin | Pyannote.audio: neural building blocks for speaker diarization[END_REF] assigns a speech frame to a wrong (red) or to the right speaker (blue).

Assessing the reliability of an ASR system

We ran Whisper large ASR system [START_REF] Radford | Robust speech recognition via large-scale weak supervision[END_REF] on highly naturalistic speech utterances from the American English Bergelson corpus [START_REF] Bergelson | What do North American babies hear? A largescale cross-corpus analysis[END_REF][START_REF] Bergelson | SEEDLingS HomeBank corpus[END_REF] (child-centered recordings, similar to the ones used in Section 5.2). We evaluate the performance of Whisper using the percentage hits (i.e., percentage of words correctly transcribed). We include a total of 804 utterances at least 5words long (as short sequences most often led to a score 0 % or 100 %).

Figure 4 shows the average percentage of hits obtained by Whisper for utterances binned according to their predicted SNR (top panel) or C 50 (bottom panel) decile. On average, Whisper correctly transcribes 83 % of the words on utterances whose SNR belongs in the [START_REF] Xiong | Exploring auditory-inspired acoustic features for room acoustic parameter estimation from monaural speech[END_REF][START_REF] Traer | Statistics of natural reverberation enable perceptual separation of sound and space[END_REF] dB (last SNR decile, top panel). This number decreases as the SNR decreases until Whisper successfully transcribes only 38 % of the words on utterances whose SNR is in the [-9, -4] dB range (first SNR decile). Although utterances whose predicted C 50 is high tend to be better transcribed by Whisper, the trend with respect to the C 50 is less clear (bottom panel). In conclusion, this experiment shows that Brouhaha can be used to investigate the robustness of ASR systems to noisy conditions.

CONCLUSION AND FUTURE WORK

We proposed Brouhaha, a model jointly trained on the voice activity detection, SNR, and C 50 estimation tasks. After evaluating the performance of our system and demonstrating that the multi-task training regime is beneficial, we illustrated two use cases showing how our model can be used on real data. Beyond investigating errors made by speech processing systems or assessing their reliability in noisy and reverberant conditions, we foresee other potential downstream tasks, e.g., SNR-or C 50 -based microphone selection [START_REF] Wolf | Towards microphone selection based on room impulse response energyrelated measures[END_REF] or SNR-aware speech enhancement [START_REF] Fu | SNR-aware convolutional neural network modeling for speech enhancement[END_REF]. Future work could explore these downstream tasks, the use of spontaneous clean speech to improve VAD performance, or the estimation of other room acoustic parameters, such as T 60 or DRR. Both a pre-trained model and our audio contamination pipeline are shared with the community1 .

Fig. 1 :

 1 Fig.1: Audio contamination pipeline. s 1 → s 2 : With probability p RIR = 0.9, the clean speech segment (marked as S) contaminated with silence (marked as NS) s 1 is convolved with a randomly drawn impulse response RIR s . n 1 → n 2 : With probability p RIR , the randomly drawn noise segment n 1 is convolved with a randomly drawn impulse response RIR n . s 2 + n 2 → s 3 : The reverberated speech segment s 2 and the reverberated noise segment n 2 are added together to obtain a Speech-to-Noise Ratio (SNR) randomly drawn between 0 and 30 dB. C 50 is computed as the ratio of early (0 to 50 ms) and late (50 ms to the end of the response) energies of the room impulse response RIR s . As noises can have a wide dynamic range and the utterance-level SNR captures only global information about the noise level, we recompute SNRs using a 2-second long sliding window shifted every 10 ms over s 2 and n 2 . To simulate different locations for the speech and noise sources, we used two randomly RIRs (as opposed to only one). This design choice aimed at forcing the model to rely solely on the reverberation characteristics of speech, disregarding the reverberation of noise, in order to predict the C 50 measure. Labels obtained via this pipeline include: speech/non-speech (frame-level), C 50 measure of RIR s (utterance-level), and SNR (frame-level).

Fig. 4 :

 4 Fig. 4: Assessing the reliability of an ASR system. Percentage of hits obtained by Whisper large as a function of predicted SNR decile (top panel) and predicted C 50 decile (bottom panel). Bars represent the percentage of hits averaged across utterances. Thin black lines represent standard errors.

Table 1 :

 1 Performance on unseen synthetic data (our test set) in terms of F-score (VAD) and mean absolute errors (SNR and C 50 ). A checkmark below a given training task indicates that the associated loss is activated during training.

	Training tasks:	VAD	SNR	C 50
	VAD SNR C 50 F-score (%)MAE (dB) MAE (dB)
	✓	✓	✓	93.7	4.1	3.5
	✓	✓		93.7	4.2	--
	✓		✓	93.6	--	3.8
		✓	✓	--	4.3	3.7

https://github.com/marianne-m/brouhaha-vad