Djegdjiga Amazouz
email: djegdjiga.amazouz@univ-paris3.fr

Martine Adda-Decker

Lori Lamel
email: lamel@limsi.fr

Addressing Code-Switching in French/Algerian Arabic Speech

Keywords: Code-switching, Language Identification, Algerian Arabic, French

This study focuses on code-switching (CS) in French/Algerian Arabic bilingual communities and investigates how speech technologies, such as automatic data partitioning, language identification and automatic speech recognition (ASR) can serve to analyze and classify this type of bilingual speech. A preliminary study carried out using a corpus of Maghrebian broadcast data revealed a relatively high presence of CS Algerian Arabic as compared to the neighboring countries Morocco and Tunisia. Therefore this study focuses on code switching produced by bilingual Algerian speakers who can be considered native speakers of both Algerian Arabic and French. A specific corpus of four hours of speech from 8 bilingual French Algerian speakers was collected. This corpus contains read speech and conversational speech in both languages and includes stretches of code-switching. We provide a linguistic description of the code-switching stretches in terms of intra-sentential and intersentential switches, the speech duration in each language. We report on some initial studies to locate French, Arabic and the code-switched stretches, using ASR system word posteriors for this pair of languages.

Introduction

Code-switching (CS) is the process of switching from one language to another in the same conversational sequence [START_REF] Myers-Scotton | Duelling languages: Grammatical structure in codeswitching[END_REF][START_REF] Scotton | Multiple voices: An introduction to bilingualism[END_REF][START_REF] Isurin | Multidisciplinary approaches to code switching[END_REF]. CS is a spoken phenomenon of bilingual and multilingual communities [START_REF] Auer | Language and Space: An International Handbook of Linguistic Variation. Theories and Methods[END_REF][START_REF]Code-switching in conversation: Language, interaction and identity[END_REF]. Given the high number of bilingual speakers worldwide, the phenomenon of CS is extremely frequent although it has not yet been frequently studied by the automatic speech processing community.

Bilingual speakers may spontaneously introduce words and phrases from one language (the embedded language) placed within the basic language (the matrix language). CS may take different forms within the speech, linguists distinguish between inter-sentential and intra-sentential CS [START_REF] Kebeya | Inter-and intra-sentential switching: are they really comparable?[END_REF]. We notice that there are different terms used to refer to bilingual speech phenomena that we consider CS such as mixed structure [START_REF] Canut | Comment les langues se mélangent: codeswitching en francophonie[END_REF], mixed code [START_REF]Code-switching in conversation: Language, interaction and identity[END_REF], transcodic marks, loan words. All these phenomena, that are included in this present CS study, are typically studied in interactional sociolinguistics [START_REF] Gumperz | Discourse strategies[END_REF][START_REF]Code-switching in conversation: Language, interaction and identity[END_REF], language learning/acquisition [START_REF] Ezeizabarrena | Les phénomènes de codeswitching dans les conversations adulte-enfant (s) en basqueespagnol: une approche syntaxique[END_REF][START_REF] Moore | Code-switching and learning in the classroom[END_REF] as well as general linguistics. More recently, CS speech has also raised interest in computational linguistics and automatic speech processing research [START_REF] Vu | A first speech recognition system for mandarin-english code-switch conversational speech[END_REF][START_REF] Yılmaz | Investigating bilingual deep neural networks for automatic recognition of code-switching frisian speech[END_REF][START_REF] Solorio | Learning to predict code-switching points[END_REF]. A lot of the studied CS data come from media and web podcast corpora. These types of corpora are relatively easy to obtain and may contain large amounts of data. However, for acousticphonetic studies, media data are often inadequate as the acoustic signal quality may be low (noise, weak sound quality and audibility) and they may present some difficulties to process (low quantity of CS, high speech rate, overlapping speech). One of the aims of this study is to collect and annotate a French/Arabic CS speech corpus of high acoustic quality and with a relatively high density of CS. To this purpose, we selected speakers who are used to language switch in their daily lives. The data were not collected in a field study, the speakers were invited to come to our lab. However, we designed a special protocol to collect CS data in different production rates and styles. The potential usages of the data are multiple: classification of foreign accent, work on L2 pronunciation learning, phonetic and prosodic comparisons of monolingual and bilingual speech.

In this paper we will first present related work, before introducing our corpus collection and the resulting data. Too, we address the challenge of how to obtain a CS corpus in laboratory environment by creating a sociolinguistic context favoring speech in two languages. As the corpus annotation is still underway, we provide some partial statistics on the content and some first experiments at detecting CS segments. Figure 1 illustrates an example of intra-sentential French/Arabic code switching. The sentence starts with a word in French and is then completed in Arabic. This figure provides annotations at the word level, the language segmentation and tags, and the English translation (bottom line).

Related Work

From LID and ASR perspectives, CS has received attention in several studies using a variety of data from speech web data to carefully recorded speech. In Mandarin-Taiwanese CS, switches of language where a high CS was observed, Lyu and Lyu [START_REF] Lyu | Language identification on codeswitching utterances using multiple cues[END_REF] used three sets of data for their LID system: a monolingual training data set for Mandarin, another monolingual training set for Taiwanese, and finally a code-switching test data set (Test-CS). The Test-CS data consist of 4.8 hours of speech and it corresponds to 4600 utterances. Their LID system proposes to evaluate the CS and contribute to error reduction for LID and ASR. In the same line, Lyu et al., [START_REF] Lyu | Mandarin-english code-switching speech corpus in south-east asia: Seame[END_REF] designed the SEAME (Mandarin-English code-switching speech corpus in South-East Asia) corpus with 93 hours recorded from media interviews of Singaporean and Malaysian speakers who mix Mandarin and English in their speech. In their corpus, intrasentential CS represented 30 hours of spontaneous Mandarin-English with Mandarin being the dominant language. In CS radio broadcast data, Modipa et al., [START_REF] Modipa | Implications of sepedi/english code switching for asr systems[END_REF] tested the performance of ASR system in Sepedi spoken mixed with English as dominant language, detection of the language change in the utterance. They analysed the mechanism of CS in this pair of languages by quantifying the frequency of CS. An acoustic analysis was also done using this corpus in order to develop an ASR system. They produced and used a dictionary for the two languages in a single bilingual ASR system.

Research questions

In this paper, we pose and attempt address the following questions:

i. Is CS a common practice in Maghrebian broadcast speech and in which country is CS practice more frequent?

ii. How can we collect conversational speech including code-switching? Are there methods to incite CS?

iii. Can code-switching be detected using LID systems [START_REF] Laurent | Investigating techniques for low resource conversational speech recognition[END_REF]? what is the minimum time span to be successful?

iv. Can we detect code-switching using ASR systems?

v. How can speech technologies help in quantifying different types of CS?

The first question is addressed using a large collection of broadcast and podcast speech data from the Maghreb countries referred to as the MCSM corpus. The results of this preliminary study on Arabic/French code-switching guided our choices to collect a CS corpus of read and spontaneous speech focused mainly on Algerian Arabic (question ii) . The last three questions deal with evaluating the potential help of automatic speech processing tools to detect code-switching and/or in providing support in annotating such corpora for further linguistic studies.

The French-Algerian FACST corpus

In this section we present the bilingual community and describe the corpus and the different steps of the data collection. We also provide some basic statistics of the collected CS data.

Code-switching in FR/AA bilingual community

Algerian Arabic (AA) is one of the Arabic dialects used in informal situations and in daily life contrary to Modern Standard Arabic (MSA) which is mainly used in formal communication as in eduction, press news, media and political speeches. AA is the mother tongue of about 40 millions of Algerians. It is an oral language and has few written resources. In Algeria, French is the major second language inherited from the bygone colonial era and still learned during education. It is also used in daily life in contact with AA. French is the language of several scientific faculties and it is used by part of Algerian press and media. Hence, AA has been in contact with FR for historical and educational reasons over many decades [START_REF] Caubet | Métissages linguistiques ici (en france) et là-bas (au maghreb)[END_REF]. As a consequence, Algerian people tend to speak fluently both French and AA, and CS phenomena frequently appear in their daily communication. One can notice that, in the Algerian community, a lot of languages and dialects coexist: Arabic and its dialects, Berber and its variants, French in the larger cities of the country and Spanish in the North West of Algeria. In this study, we focus on spoken Algérois [START_REF] Saadane | A conventional orthography for algerian arabic[END_REF], the AA dialect of the Algiers area. This region shows a high degree of contact between AA and FR, where more CS practice can be expected than in other regions of Algeria. The situation of CS in France, has a tendency to be dominated by the French language. In their professional or educational contexts, Algerians tend to prefer French as basic communication language, while occasionally switching to AA.

Corpus

Prior to collecting our French/Arabic CS speech corpus in France, we made a preliminary study on the MCSM database (Maghrebian Code-switching in Media) [START_REF] Amazouz | Arabic-french code-switching across maghreb arabic dialects : a quantitative analysis[END_REF], which consists of a collection of Maghrebian broadcasts from Algeria, Morocco and Tunisia. In this study, we compared the quantity of CS occurring in dialectal Arabic with a focus on code switches to French. The corpus consists of 53 hours of television media containing entertainment and TV talk shows in Arabic dialects: Algerian (14 hours), Moroccan (15 hours), Tunisian (24 hours). This study aimed at answering the following question: is Arabic-French CS frequent in Maghreb dialects? In which country is the CS effect the strongest? Table 1 Although the number of switches is relatively high, the segments tend to be of very short duration and typically include either discourse markers and adverbs (alors 'then', donc 'so', justement 'precisely', d'accord 'OK',) or technical vocabulary such as fashion products. This latter switches can be considered as loan word switches. Given these results on the media data, we decided to focus our CS corpus collection on the French/Algerian pair of languages as the chance to find codeswitching speakers seems to be higher.

FACST

The FACST corpus (French Arabic Code-switching Triggered) consists of records of CS conversations with bilingual adult speakers who tend to code-switch in there daily lives. At present, FACST includes 8 bilingual speakers aged from 20 to 35. They all have lived a part of their lives in Algeria and another in France. They all have studied at university and use both languages daily and often with CS. We recorded the speakers in a soundproof room at LPP (Labortoire de Phonétique et Phonologie) of Sorbonne-Nouvelle University in Paris. The corpus is intended to serve multiple research purposes in CS including language identification studies, language boundary de-tection, as well as phonetic and prosodic issues in CS speech and other linguistics studies. The aim of recording these conversations at LPP was on one hand to get speech with a high quantity of CS in the records, on the other hand, recordings in a soundproof booth ensure a high acoustic quality of the signal.

Data collection

We selected the participants using a sociolinguistic online questionnaire (ECSP 2016) "Experience of Code-switching practice" with questions about their linguistic autobiography, the environment in which languages are practised, language acquisition/learning, CS habits. . .

Recording sessions started with a preliminary unrecorded conversation with the speaker to get her/him in a relaxed setting practicing both languages in the same interaction. First, we asked the speakers to perform oral readings of two texts with three different speech rates (slow, normal, fast), one in AA and and other one in FR. We asked the speakers to read a text excerpt of "Le Petit Prince" for French, and an excerpt from the movie scenario "Bab El-Oued city" in Algerian Arabic. The stimuli of the controlled read speech are summarized in Table 2. The goal of this experiment was, firstly to obtain a controlled monolingual speech corpus in AA and FR for bilingual speakers before proceeding to the bilingual speech. Secondly, we would like to highlight potential pronunciation differences of consonants and vowels in each language separately. Thirdly, using the three speech rate task, we intend to study realisation differences in consonants and vowels.

The second step consisted of recording dual conversations between the linguist (who is a bilingual speaker of both languages) and the speaker. The CS conversations are triggered by questions about life and study in both countries. So, the role of the linguist is to ask the questions and let the speaker answer freely and hopefully make use of CS. The recordings lasted from 15 to 25 minutes for each speaker. Figure 2 illustrates the methodology used to entice natural CS conversations. The linguist asked questions in one or the other of the targeted languages (FR or AA) also including CS to stimulate spontaneous speech with FR/AA code-switching.

The main role of the linguist here was to create a sociolinguistic context in which conversational CS occurs naturally in guided dialogs. However, we want to point out that this collection experiment, carried out in a soundproof room, is clearly not a sociolinguistic survey in the field -it is an attempt to trigger similar naturalistic CS phenomena in a lab setting.

Annotation

We used the Transcriber [START_REF] Barras | Transcriber: development and use of a tool for assisting speech corpora production[END_REF] program to manually segment the audio utterances into sentences, breath groups, speech turns, languages used. We also annotated the speakers and the language of the segments as follows: The transcription of the speech has been done after the segmentation and language annotation. The AA speech was transcribed using a transliteration orthography inspired by Buckwalter Arabic transliteration [START_REF] Buckwalter | Arabic transliteration[END_REF]. We chose this transcription convention in order to facilitate the use of the manual transcripts for phonetic analyses while keeping the possibility to convert the transliterated characters to Arabic charterers in future studies.

For the time being, the speech data from 5 subjects has been manually transcribed. Their production in terms of words and duration by language are shown in Table 3. Figure 3 shows a histogram of length of CS speech segments for FR/AA.

Speech Technologies for Code-Switching

In the following, we report on some first experiments to detect CS using automatic speech processing tools. First, the speech files were automatically segmented into acoustically homogeneous segments, which ideally correspond to speaker turns and/or to a given language or stable acoustic conditions (broad band/telephone band...). These segments were then automatically transcribed using different ASR systems [START_REF] Gauvain | Conversational telephone speech recognition[END_REF][START_REF] Laurent | Investigating techniques for low resource conversational speech recognition[END_REF] in parallel: a French system, a multi-dialect Arabic system (predominantly Lebanese) and an Algerian Arabic (dialect) system. The systems were trained on several hundreds of hours of speech from a large number of speakers. The Algerian models are the result of adapting the multi-dialect Arabic system with about 300 hours of data from Algerian speakers. Our expectation is that the French system will produce the highest scores on French speech and vice versa, that AA speech will be best decoded by one of the two Arabic systems. Table 4 gives the average word posteriors (confidence scores) for each speaker as a function of the manually annotated language and the ASR system used to transcribe the data. Overall, and for most speakers, a higher confidence is obtained when the data is processed with the matching system.

Conclusions

In this paper, we presented our ongoing studies on French/Algerian Arabic code switching in spoken language. We tried to answer several different research questions. First, we used a large set of media data from the Maghrebian countries to measure the Arabic/French code switching reality in entertainment shows. This study revealed that the Algerian shows featured a much higher CS rate as compared to Tunisian and Moroccan broadcast shows. We described a speech collection protocol to record both a monolingual speech and a CS speech corpus testing different ways to elicit or trigger natural codeswitching in spontaneous speech. The purpose of the corpus is to facilitate acoustic-phonetic studies of segments in bilingual and CS situations. At present, the corpus contains 8 hours of speech which are still in process of being manually segmented, annotated and transcribed. The four hours of transcribed data which have been used in this work show a high density of code-switching with mainly short intra-sentential CS, although some segments are of longer duration and cover stretches of CS beyond sentential boundaries. These first results confirm that our CS triggering protocol was successful and that using it we were able to obtain spontaneous code switched speech with both inter-and intra-sentential CS. Concerning the question as to whether CS can be automatically detected, we presented first results using three different ASR systems in parallel (French, multi-dialectal Arabic, Algerian Arabic). These results show that short duration CS segments poses serious challenges to automatic language identification in CS speech, although parallel ASR systems may produce word posteriors which are good indicators of language change. We will pursue this line of investigation in our future work. Concerning our last question on the use of speech technologies to study CS, our present assessment is that automatic speech transcription is of great help to achieve a high quality transcription and temporal alignments into words and phones which opens new perspectives for large scale CS studies on acoustic-phonetic and prosodic levels.

Acknowledgements

Figure 1 :

 1 Figure 1: An example of intra-sentential FR (French) / AA (Algerian Arabic) codeswitching.

Figure 2 :

 2 Figure 2: Examples of questions asked by the linguist to trigger spontaneous FR/AA CS in the subject's responses. Type of CS produced that follows the questions in the speech.

Figure 3 :

 3 Figure 3: Histogram of the CS segment lengths for FR/AA.

  Figure 4 shows the French system's confidence scores on an excerpt of speech including CS. The x-axis corresponds to the word numbers. Words on the left (70-79) and right (85-89) are in French and the middle words are in Arabic. The purple curve shows the French ASR word posteriors, which as can be expected are higher for French than for Arabic. The green curve is a smoothed version of the posteriors, as the raw values are quite brittle. The bottom line specifies the true language (0.1 is French, 0.2 is Arabic).

Figure 4 :

 4 Figure 4: ASR word posteriors of the French transcription systems (raw scores and smoothed). The X-axis corresponds to an excerpt of speech: words numbered from 70-79 and 85-89 are in FR, words numbered from 80-84 correspond to an AA codeswitch. The lowest curve (blue) denotes FR (0.1) or AA (0.2).

  This research is supported in one hand by the French National Agency for Research as part of the SALSA project (Speech and Language technologies for Security Applications) project under grant ANR-14-CE28-0021 and in the other by the French Investissements d'Avenir -Labex EFL program (ANR-10-LABX-0083). The study was supported by LPP-CNRS Paris-III Sorbonne Nouvelle University and LIMSI-CNRS Paris-Orsay University. We thank the participants who contributed to the FACST corpus.

Table 1 :

 1 displays the figures describing the MCSM corpus in terms of total duration and number of CS segments and duration. Results show that Maghrebian bilingual communities all use Arabic-French CS in media, but the quantity of CS varies with the country. It can be seen that Algerian/French CS is much more frequent (148 CS segments per hour) than Moroccan/French CS (62 CS segments per hour) and Tunisian/French (21 CS segments per hour). CS segment quantities and number of CS speakers of each country in MCSM

	Country show	Algeria Morocco Tunisia
	Nb. of CS segments	2081	938	509
	Total show duration 14h	15h	24h
	Nb. CS/hour	148.6	62.5	21.2

Table 2 :

 2 Controlled read speech in FR and AA. Number of words and average reading time in seconds (slow-medium-fast)

	Language Nb. of words Average reading time
			for each rate (s)
	FR	185	92 -60 -55
	AA	102	50 -37 -30

Table 3 :

 3 Some statistics about the FACST spontaneous speech. Number of words and duration of CS speech in each language.

	Speakers
	and gender FR words Duration AA words Duration
	S 5 M	1619	531.11s	348	118.20s
	S 3 F	1283	480.88s	369	138.97s
	S 7 M	1157	425.66s	243	90.93s
	S 8 F	829	391.72s	277	145.56s
	S 2 M	277	118.16s	285	116.67s

Table 4 :

 4 Average word level posterior scores by speaker with the French (left) and Algerian (right) ASR systems, for words of the CS segments manually annotated as French and Arabic.

	CS language →	Fre	Ara	Fre	Ara
	Speaker	French ASR Arabic ASR
	Speaker 5	0.74	0.72 0.54	0.56
	Speaker 3	0.79	0.57 0.58	0.65
	Speaker 7	0.74	0.78 0.51	0.54
	Speaker 8	0.78	0.61 0.56	0.58
	Speaker 2	0.62	0.59 0.59	0.56
	Overall	0.76	0.64 0.56	0.59


Djegdjiga Amazouz
email: djegdjiga.amazouz@univ-paris3.fr

Martine Adda-Decker

Lori Lamel
email: lamel@limsi.fr

The French-Algerian Code-Switching Triggered audio corpus (FACST)

Keywords: Code-switching, bilingual speakers, oral speech data, French, Arabic

The French Algerian Code-Switching Triggered corpus (FACST) was created in order to support a variety of studies in phonetics, prosody and natural language processing. The first aim of the FACST corpus is to collect a spontaneous Code-switching speech (CS) corpus. In order to obtain a large quantity of spontaneous CS utterances in natural conversations experiments were carried out on how to elicit CS. Applying a triggering protocol by means of code-switched questions was found to be effective in eliciting CS in the responses. To ensure good audio quality, all recordings were made in a soundproof room or in a very calm room. This paper describes FACST corpus, along with the principal steps to build a CS speech corpus in French-Algerian languages and data collection steps. We also explain the selection criteria for the CS speakers and the recording protocols used. We present the methods used for data segmentation and annotation, and propose a conventional transcription of this type of speech in each language with the aim of being well-suited for both computational linguistic and acoustic-phonetic studies. We provide an a quantitative description of the FACST corpus along with results of linguistic studies, and discuss some of the challenges we faced in collecting CS data.

Introduction

When multilingual speakers communicate using two shared languages in a conversation, they may switch between these languages in a same sentence or utterance. Switching may become a common practice in social groups with strong language contact. This simultaneous use of two language codes can take several forms at the lexical, syntactic and communication levels, and is commonly referred to as code-switching (CS) [START_REF] Muysken | Bilingual speech: A typology of codemixing[END_REF][START_REF] Grosjean | A psycholinguistic approach to codeswitching: The recognition of guest words by bilinguals. One speaker, two languages: Cross-disciplinary perspectives on code-switching[END_REF]. CS may present complex structures in particular at morphological, syntactical and phonetic levels. Hence, CS has interested several research fields such as sociolinguistics and interactional linguistics [START_REF] Gumperz | Discourse strategies[END_REF][START_REF] Auer | Language and Space: An International Handbook of Linguistic Variation[END_REF][START_REF] Bullock | Phonetic reflexes of code-switching, chapter 10[END_REF][START_REF] Gardner-Chloros | Code-Switching[END_REF][START_REF] Sebba | On the notions of congruence and convergence in code-switching[END_REF][START_REF] Tabouret-Keller | L'enquête sociolinguistique à grande échelle: Un exemple: Sociolinguistic survey of multilingual communities. part i, british honduras survey[END_REF][START_REF] Ziamari | Le code switching au Maroc: l'arabe marocain au contact du français[END_REF][START_REF] Piccinini | Cross-language activation and the phonetics of code-switching[END_REF][START_REF] Gullberg | Research techniques for the study of code-switching, chapter 2[END_REF] with numerous studies describing CS phenomena, explaining the process of its realization and analyzing its productions. Computational linguistics enable the study of large textual and spoken corpora [START_REF] Çetinoglu | Challenges of computational processing of code-switching[END_REF][START_REF] Schultz | Multilingual speech processing[END_REF]. Consequently, experiments have been carried out on language identification (LID) for both oral and written CS as well as automatic identification of CS languages pairs [START_REF] Lyu | Mandarin-english code-switching speech corpus in south-east asia: Seame[END_REF][START_REF] Lyu | Language identification on code-switching utterances using multiple cues[END_REF][START_REF] Modipa | Implications of sepedi/english code switching for asr systems[END_REF]. Furthermore, quantifying CS in large corpora at the word and oral segment levels gives a precise idea about the use of CS in bilingual communities [START_REF] Amazouz | Arabic-french code-switching across maghreb arabic dialects : a quantitative analysis[END_REF]. It also allows the determination of which language is dominant in the CS speech and in the daily CS practices. Applying language processing tools can help to compare CS in different language pairs and to determine how frequent CS is for pairs of languages. In this paper, we focus on eliciting spontaneous CS and how to collect CS data. And, we question about methods used to build this hybrid oral speech. So, this work describes the design, recording, and annotation of the French Algerian Code-switching Triggered corpus (FACST) being created to support a variety of linguistic studies. Using a methodology designed to elicit CS speech, of 20 bilingual Algerian/French speakers have been recorded in a quiet room. The paper is organized as follow: presentation of FACST corpus, selection of CS speakers, the recording protocols and the stimuli of CS, and CS data processing.

Algerian Arabic dialect, French and CS

The Algerian Arabic dialect (AA) is an oral dialect of North African Arabic dialects group spoken in Algeria and it is the mother tongue of 80% of Algerians. AA is different from Modern Standard Arabic (MSA) at several levels: lexicon, phonetics, phonology, syntax and morphology [START_REF] Saadane | A conventional orthography for algerian arabic[END_REF][START_REF] Souag | Explorations in the syntactic cartography of algerian arabic[END_REF]. MSA is mainly a written language while AA has few written resources. But AA written form becomes more and more widespread especially in social medias. Commonly, AA is written with Arabic characters Table 5 of Appendix. This script is written from right to left. Too, another form of AA script transliterated with Latin characters called "Arabizi transliteration" used heavily on the internet and SMS [START_REF] Cotterell | An algerian arabic-french codeswitched corpus[END_REF][START_REF] Al-Badrashiny | Automatic transliteration of romanized dialectal arabic[END_REF][START_REF] Bies | Transliteration of arabizi into arabic orthography: Developing a parallel annotated arabizi-arabic script sms/chat corpus[END_REF]. Thus, AA can be written with two forms. French language (FR) is the first foreign language spoken in the Algerian community and is for the most part the second language for Algerians. This bilingual community has tens of millions of bilingual speakers who live in Algeria and in France. CS is increasingly part of their daily communications.

Code-switching is a general term referring to language change within a given conversation or an utterance. Codeswitching can take many different forms, such as intersentential CS (language changes at sentence boundaries), intra-sentential CS (CS within sentences) [START_REF] Kebeya | Inter-and intra-sentential switching: are they really comparable?[END_REF], Code-mixing (insertion of L2 word within L1 utterance) [START_REF] Muysken | Bilingual speech: A typology of codemixing[END_REF], borrowing, and bilingual verbs (L1 verb inserted in L2 form) [START_REF] Muysken | Bilingual speech: A typology of codemixing[END_REF]. CS is also characterized by individual choices and individual forms as a placement of adverbs in the beginning or at the end of the sentences, the inclusion or deletion of articles/particle at the switch moment and the construction of CS sentences. In intra-sentential CS, speaker may produce in ungrammatical sentences due to the non-correlation of the two grammar codes but the sentences are semantically correct [START_REF] Tossa | Phénomènes de contact de langues dans le parler bilingue fongbe-français[END_REF]. In FACST corpus, we used this general term CS for these transcodic marks. Building an oral CS corpus requires careful attention, in particular on oral languages as Algerian Arabic dialect (AA). Indeed, comparing French language (FR) and AA, AA is an oral language with low written resources unlike French which is standardized with more written resources and more steady in grammar. CS corpus involves selection of speakers who represent CS speech of the linguistic community. It requires too an appropriate methodology to ensure that the recordings contain sufficient instances of CS.

FACST corpus

Corpus annotation is challenging, requiring guidelines for segmentation of CS and transcription of dialectal speech.

In the FACST corpus, these particularities are taken into account in the essential steps of oral corpus construction, that is, segmentation, annotation, transcription and alignment.

CS speaker selection

We selected participants using a socio-linguistic online questionnaire named Experience of Code-switching practice (ECSP). The ECSP form includes questions about linguistic autobiography of the potentially recruted speakers, their bilingualism, the environment in which French and AA languages are practiced, language acquisition/learning, CS habits. . . The participants were selected and recorded in 2016-2018 and were (young) adult speakers aged between 23 to 39 years, with an equal number of men and women. We selected speakers who stated that they used to codeswitch in their daily lives and that they use CS at least in two domains among studies, work, family, friends. All the speakers have lived a part of their life in Algeria and another part in France. They all have studied in university. At this date, we transcribed and aligned 13 speakers.

Recording protocol

The speakers were recorded in a soundproof room at LPP (Labortoire de Phonétique et Phonologie) of Sorbonne-Nouvelle University in Paris or in a very calm room. The corpus is intended to serve multiple CS research purposes including phonetic, prosodic and lexical studies as well as more automatic processing challenges such as language identification and language boundary detection. The main challenge of recording the CS conversations in a lab environment was to meet two antagonistic requirements: ensure the collection of informal speech with a high quantity of CS and at the same time guarantee a high acoustic quality of the signal.

To this aim, each recording session started with a preliminary unrecorded conversation with the speaker, to get her/him in a relaxed setting, practicing both languages in the same interaction. We started the recording with the first task of reading texts in both French and AA. This was followed by discussions designed to elicit spontaneous speech with CS production. Figure 1 shows the corpus organization in two parts: controlled speech with reading texts task and spontaneous speech triggered by questions.

Reading tasks

The speakers were asked to perform oral readings of two texts, one in AA and another one in French, at three different speech rates (slow, normal, fast). For French, the text was an excerpt from "Le Petit Prince" ('The little Prince', by A. de Saint-Exupéry). For Algerian Arabic, we used an excerpt from an Algerian movie scenario "Bab El-Oued City" (by M. Allouache) transcribed in Arabic letters. The controlled read speech recordings are summarized in Table 2. The first goal of the read-speech recordings was to obtain a controlled monolingual speech corpus in AA and FR for the bilingual speakers before proceeding to the bilingual speech. Second, the recordings can serve to highlight potential pronunciation differences of consonants and vowels in each language separately. Third, studying the productions at the three speech rates provides data to investigate rate-related differences in the realization of consonants and vowels in each language. So, the controlled speech data helps to apply acoustic analysis and serves to beacon for phonetic observations in CS spontaneous speech.

Elicited CS speech

In this step, we recorded dual conversations between the linguist (who is a bilingual speaker of both languages) and the speaker. The CS conversations are triggered by questions. The principal questions were inspired by the feedback in the ECSP questionnaire of each speaker.

Most of the subjects covered by the speakers were about describing and comparing life in both countries, studies in both countries and conversations about language and bilingualism practices. Other sub-themes were also addressed in the conversations following a free speech approach. So, the role of the linguist in this task is to ask the questions and let the speaker answer freely, making spontaneous use of CS. The recordings of this task lasted from 10 to 25 minutes for each speaker. To elicit CS, we asked four main types of questions in each speech sequence. These questions are summarized as follows:

1. AA questions about studies and work.

2. FR questions about life and studies in Algeria.

3. FR/CS: code-switched questions using French as the base language.

4. AA/CS: code-switched questions using AA as a base.

In these stimuli, the linguist tries to implicitly suggest the use of AA and French in a same conversation to obtain CS spontaneous speech.

Segmentation and annotation

First, manual segmentation of the recordings is based on language change, breath groups and speaker turns. There are two types of segments: language segments which correspond to one language, and breath group segments which correspond to a rhythmic group of an utterance or an oral sentence. The lengths of the segments are quite variable, ranging from very short segments (less than 2s) to longer ones, with an average segment length of 6s. Indeed, a switch can be limited to a very short word such as an article or a particle. The aim of this segmentation is to get boundaries for each language and label them "ALG" for AA segments and "FRA" for French segments. This type of segmentation and annotation helps thereafter when processing and manipulating the data. We used Transcriber program [START_REF] Barras | Transcriber: development and use of a tool for assisting speech corpora production[END_REF] to segment and annotate FACST data. Figure 2 shows an example of segmentation with the segment-level annotations.

CS Transcription

The segments were manually transcribed using an orthographic transcription for French and a transliterated transcription for AA inspired by Buckwalter Arabic transcription (BKW) [START_REF] Buckwalter | Arabic transliteration[END_REF] and modified according to AA specificities. The aim of the transliteration is to get a scripts with the same characters in both languages and to get a script which is written in the same direction (Arabic script is written from right to left). This transcription convention has been created in order to facilitate the use of the manual transcriptions, without special characters, for phonetic analyses while keeping the possibility to convert the transliterated characters to Arabic characters in future studies. Table 5 of Appendix illustrates the characters chosen for FACST and the corresponding symbols for each characters in Arabic letters, BKW convention and IPA symbols. For AA and FRA languages, the transcription also includes pauses, repetitions, hesitations, speech backchannels and various linguistic disfluencies. 13 of the speakers are transcribed at this date. Table 3 shows statistics about the number of segments and words counted in CS speech of the transcribed speakers.

In Arabic script, the articles and some particles are attached to the word: ( ) "the door", ( ) "for/to the door". For AA segments, we transcribed the articles and a large number of particles, placed initially at the beginning of the words, separately from the word. This separation is applied in order to count the number of words and to compare speech production at word level in CS FR-AA. So, in FACST transcription the utterance is transcribed "li al baAb" "for/to the door".We used this to readily separate the languages in intra-sentential CS. Example: "liy" in AA, a mark placed at the end of a word refers to pronoun suffixes, conjugation morphemes, and number and gender marks.Thus, due to the morphological construction of AA [START_REF] Souag | Explorations in the syntactic cartography of algerian arabic[END_REF], we did not apply separation for attached morphemes at the end of words in this corpus, examples: attached objects "jabthum" I brought them back.

With the help of these transcriptions, we counted too the highest occurrences of words in AA and FRA in order to know the words and the utterances which appears frequently in this pair of languages. Especially, getting a precise overview of AA insertion as an embedded language in FRA as a matrix language. The lists bellow present words and groups of words occurrences > 20 for AA and > 60 for FRA.

AA: waAHad, allah, bazzaAf, gaAc, waAluw, in, kaAmal, bacd, hnaAyaA, kiymaA, wiyn, kaAyan, laAzam, claY, hnaA, kunt, liy, mca, baAX, iyh, min, li, kaAn, lak, taAc, anaA, waAX, dzaAyar, kiy, alliy, laA, fiy, maA, wa, al, salaAm. FRA: c'est-à-dire, un petit peu, il y a , deux, non, dans, très, par, beaucoup, quand, voilà, enfin, fait, plus, ils, tout, même, oui, tu, pour, donc, on, qui, parce que, mais, une, ça, en, j'ai, un, des, il, à, et, le, la, les, c'est, pas, que, je ...

A lot of AA words introduce CS, they start AA segment after FRA segment or end AA segment to switch to FRA. Mostly this words are particles:"fiy" in, "li" for/to, "taAc" for, "wiyn" where, "claY" on, "alliy" . . . They connect the two languages and introduce a significant part of switches with a high number of occurrences as shown in

Data alignment

The data has been aligned using LIMSI speech recognizer in a forced alignment mode [START_REF] Gauvain | Conversational telephone speech recognition[END_REF], assigning word and phone level time codes. The alignments made use of acoustic models from different ASR systems [START_REF] Gauvain | Conversational telephone speech recognition[END_REF][START_REF] Laurent | Investigating techniques for low resource conversational speech recognition[END_REF] in parallel: a French system, and an Algerian Arabic (dialect) system. The alignment is composed by two separate parts. First alignment for French segments using pronunciation dictionary of standard french.The second alignment for ALG segments with pronunciation dictionary of AA created on the basis of MSA model and adapted on AA with specific phonemes used in this dialect as /p, v, g, Z/. Thereafter, both alignments are combined to the speech signal as shown in

Discussion and future work

In spite of a high CS production in the FR-AA bilingual community, recording speakers in laboratory requires a careful preparation of the communicative situation and of the setting of CS usage [START_REF] Gumperz | Discourse strategies[END_REF]. The role of the linguist, in this experience is, to be an interactive speaker who can adapt himself to the daily CS usage of the participant in order to incite a CS conversation. We found that frequent CS could be elicited by different questions, but the number of segments in FR and AA is not equal. FR has a higher number of segments and of produced words than Algerian for almost all speakers Table 3 shows clearly the quantity of AA and FRA distribution in CS spontaneous speech. So, we can say that in the FACST spontaneous speech corpus, FR can be considered as the dominant language and AA as dominated one. In the segmentation of spontaneous speech portions of FACST, some segments are very short (less than 1 second) because of the speed of language change. These segments generally correspond to particles and articles of both languages. One of the major difficulties of language annotation in CS FR-AA is the articles al /l/ in AA and l' /l/ in french at the switches. The two articles are pronounced identically but it is very difficult to identify the language of this word at the switch moment. E.G.: ( fiy l'école fiy-AA, l'-FR, école-FR) or ( fiy al école fiy-AA, al-AA, école-FR) CS verbs can take a base in one language but are redesigned with the other language form. An illustration is the following CS example of AA-FR: ypartaAjiy /jparta:Zi:/ "he shares"

This type of neologism is not easy to classify as French or as AA because it doesn't conventionally belong to none of the two languages. The root of the verb in bold ypartaAjiy is in French partager "to share". The prefix and suffix y -iy are in Arabic: the present form of the verb with the pronoun "he". In summary, this paper describes the steps used to construct a CS corpus containing spontaneous speech and read bilingual speech. We present the methods employed to obtain CS in a dual conversation between a linguist and a speaker. We developed a methodology to annotate the CS corpus and we proposed transcription conventions for dialectal Algerian Arabic speech. An acoustico-phonetic study will be planned to analyze CS phenomena, in particular, phonemes at the moment of switches and the question of articles mentioned above.

Appendix

FACSTFigure 1 :

 1 Figure 1: FACST data speech organization.

Figure 2 :

 2 Figure 2: Example of segments annotation by speaker code (SPx), time-codes (columns 2-3), gender, language code (FRA/ALG) and transcription.

Figure 3 :

 3 Figure 3: Example of FACST alignment on intra-sentential CS audio segment with spectrogram. Three levels of segmentation and transcription are applied. From top to bottom of tiers: phoneme segmentation and phonetic transcription, word segmentation and orthographic transcription, language segmentation and annotation.

Table 1 :

 1 Compact FACST presentation

	Label	French Algerian Code-switching Trig-
		gered (FACST) corpus
	Languages	French (FR), Algerian Arabic (AA)
	Speakers	20 speakers: 10 male, 10 female
		Ages: 23-39
	Duration	Recordings ranging from 15 to 40 min-
		utes/speaker
		Total: 7 h 30 of speech
	Content	Read speech and stimulated sponta-
		neous speech.
	Year	2016-2018

Table 2 :

 2 Read speech in French and AA.

	Number of words

Table 3 :

 3 FACST spontaneous speech: number of segments and words counted for each language and for each speaker (13 speakers)

	Spks#	FR seg FR wrd AA seg AA wrd
	SP1 M	28	112	36	108
	SP2 F	126	583	170	734
	SP3 F	219	1283	126	369
	SP4 F	113	720	67	230
	SP5 M	175	1619	93	348
	SP7 M	180	1157	98	243
	SP8 F	288	829	186	277
	SP10 M	84	793	11	45
	SP11 F	307	1746	236	948
	SP12 F	334	1960	223	598
	SP16 M	419	2075	277	1108
	SP19 F	372	2122	248	710
	SP20 F	335	1906	202	1172

table 4 .

 4 The following examples illustrate the dispositions of

	word #occurrences #occ introducing CS
	fiy	208	112
	wa	224	54
	alliy	60	21
	li	34	14
	taAc	32	25
	Table 4: AA word occurrences introducing CS > 14 occur-
	rences

Table 5 :

 5 Transliteration of AA consonants and vowels in FACST corpus with corresponding symbols in IPA, BKW, Arabic symbols and examples of words. FASCT symbols represent the phonological pronunciation of AA (Algiers region)

	Transliteration convention for AA consonants and
	vowels

Acknowledgements

This work is supported in one hand by the French National Agency for Research as part of the SALSA project (Speech and Language technologies for Security Applications) project under grant ANR-14-CE28-0021 and in the other hand by the French Investissements d'Avenir -Labex EFL program (ANR-10-LABX-0083). The study was conducted at LPP-CNRS (UMR 7018) Paris-III Sorbonne Nouvelle University and LIMSI-CNRS Paris-Orsay University. Many thanks to the 20 participants recorded who contributed to realize the FACST corpus.


We invite researchers, according to their specialization, to look for other non energetic processes that could be influenced by the irradiation with de-energized photons.

Absorber

Scattered de-energized photons

Experimental set-up for the detection of de-energized photons

G. Sardin

We propose a prototype of an innovative experimental set-up and invite those having the suitable means to consider it. After having submitted the issue to debate [1], we will approach here the experimental part by presenting a proposal whose purpose is to test the related viewpoint and try to give it an experimental answer.

Experimental set-up for the detection of de-energized photons through absorption

Schematic representation of irradiation of the phosphorescent decay process with de-energized photons: the absorber is cooled with liquid nitrogen (-196 °C, 77 K) or still better with liquid helium (-269 °C, 4 K).

Scheme of the scattered de-energized photons through the target

Liquid N or He

Laser beam

Shielding vessel

Phosphorescent substance Absorber

Photo-detector

UV laser pulse

Liquid N or He

Laser beam

Vacuum chamber

Fundamentals

It is intended to check if absorbed photons just deliver their energy, preserving hence their corporeity and merely transiting to a de-energized quantum state. The official standpoint takes for granted that when absorbed photons cease to exit. Here it is projected to experimentally check the issue.

Let us assume that through absorption, photons just loose their energy and thus preserve their structure. In doing so, photons can be seen as endlessly hopping up and down from the energetic state to the de-energized state and vice-versa, through successive emission and absorption. Free space can hence be seen as fundamentally populated of energy-less photons that constantly shift to the energized state (through emission) and back to the de-energized state (through absorption). This offers a new look at free space, which in the same way that it is fully populated of CMB photons (Cosmic Microwave Background), it would also be primarily populated of de-energized photons, i.e. their fundamental a-energetic state. In fact they would constitute a primordial field-like matrix, from which would have emerged the CMB photons, as well as massive elementary particles.

If in the emission process photons are not created but instead energy-less photons from free space are energized through the energy release from atomic or nuclear transitions or from accelerated charged particles, it consequently turns out that there would be neither creation nor annihilation of photons. So, the emission-absorption process would just correspond to the transfer of energy to pre-existent energy-less photons that hence become energized, and so, coming into view in the energetic component of the Universe.

Through irradiation with de-energized photons it is intended to affect the phosphorescent spontaneous decay by increasing the density of de-energized photons in the surrounding of the phosphorescent element. When free space is considered to be full of energy-less photons, it can be presumed that spontaneous decay could be dependent on the density of available de-energized photons. By increasing their density it can be expected that the phosphorescent decay could be influenced and therefore would proceed accordingly.

In atomic physics, the electronic bond is considered to be sustained by a force field, envisaged as being carried by the so-called "virtual" photons. Concretely, the atomic structure is regarded as being sustained by an exchange of virtual photons between the nucleus and the peripheral electrons, interchange of which is dependent each quantum state. Consequently, it is reasonable to think that altering the natural exchange of virtual photons by exposing the atom to a flux of "virtual" or equivalently "de-energized" photons generated by a laser beam impinging on an absorber, could alter the spontaneous decay of the phosphorescent transitions of electrons.

Beyond the proposed experiment, the objective is to look further for physical or chemical processes likely to be affected by exposure to a dense flow of de-energized photons. Since having no energy, these de-energized photons may only have an influence on spontaneously occurring processes, such as the decay of excited states from e.g. phosphorescence, through the alteration of the state of their spatial environment. This is an open issue which has not yet been investigated and on which there is much to be explored.

Experimental approach

The aim is to alter the environment of different set-up through the exposure to an intense irradiation of de-energized photons. A variety of spontaneously decaying pigments can be tested, as well as diverse irradiation conditions.

For this purpose, an absorber is exposed to a laser beam so that photons transfer their energy to it, and hence are de-energized. On the opposite side of the absorber is fixed a layer of a phosphorescent substance, which is excited with a pulse of an UV laser. The decay of the induced phosphorescence is then plotted. The process is afterward repeated, this time being submitted to irradiation with de-energized photons assumingly generated by the laser beam impinging on the absorber. Let us mention that the prototype set-up can be easily adequated to get the two measures at once.

After while the two plots of the phosphorescent decay are compared in order to look for any difference eventually created by the irradiation with the de-energized beam scattered through the absorber. The aim is to check the suitability of the assumption that photons de-energized through the full transfer of their energy to the absorber preserve their structure and subsist in an energy-less quantum state.

In order to avoid the heating of the absorber when irradiated by the laser beam and the subsequent IR emission, the target is cooled down to 77 K with liquid N or still better to 4 K with He. If it is assumed that the impinging photons are scattered through the absorber, having in between lost all their energy by transferring it to the absorber, on the back side of the absorber there would thus be a scattered invisible beam of de-energized photons. The challenge stands in trying to evidence it.

Space is presumed to be populated by energy-less photons, as by "vacuum" quanta in the quanto-mechanical framework. So, by means of the proposed set-up it is intended to locally produce a flux of de-energized photons in order to alter those processes not requiring energy, but susceptible to be affected by a change in their natural environment. The aim is to increase the density of energy-less photons in the immediate surrounding of spontaneous processes that does not need any energy input to ensue, such as e.g. the phosphorescent decay from a layer next to the absorber target.

Brief description of the set-up

The target, made of an absorber (e.g. graphite) and a phosphorescent layer, is stuck on one extremity of a glass tube, filled with liquid N or He, in order to deeply cool both the absorber and the phosphorescent substance, avoiding them to heat-up. The phosphorescent layer stuck on the absorber is irradiated with a pulse of an ultraviolet laser beam. Once the phosphorescent layer is glowing the absorber is irradiated on the opposite side with a laser beam in the visible range. It is assumed that through absorption, the resulting de-energized photons are scattered throughout the absorber, and so there would be a high density of de-energized photons on the back side of the absorber, and hence on the phosphorescent layer stuck on it.

It should be checked that the beam impinging on the absorber is fully absorbed on it and that there is no infrared emission. A photo-detector is placed over the phosphorescent target in order to plot its decay. The aim is to check if the decay of phosphorescence is affected by the de-energized photon environment. The vessel should be isolated from external irradiation and vacuum made inside it.

Let us point out that a negative result would however not disprove the assumption of the de-energized photons but instead it could just express that they have not influenced the decay process submitted to them, or that the alteration of the environment in the specific experimental setting was not high enough as to induce any detectable difference.


Paul Luizard

Erik Brauer

Stefan Weinzierl

Nathalie Henrich Bernardoni

Nathalie Henrich

Vocal adaptation of singers to room acoustics

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Vocal adaptation of singers to room acoustics P. Luizard 1 , E. Brauer 2 , S.Weinzierl 1 , N. Henrich 3 1 FG Audiokommunikation, TU-Berlin; 2 Universität der Künste, Berlin ; 3 GIPSA-Lab, Grenoble, France

  Characterize the influence of the acoustics of a room on the vocal production of a singer Determine which strategies are used by the singers to adapt to the room Estimate the preference of room acoustics to various singing styles, for a singer and a listener Z. Schärer Kalkandjiev and S. Weinzierl, "The influence of room acoustics on solo music performance: An empirical case study," Acta Acustica united with Acustica, vol. 99, 2013. K. Kato, K. Ueno, and K. Kawai, "Effect of room acoustics on musicians' performance. part II: Audio analysis of the variations in performed sound signals," Acta Acustica united with Acustica, vol. 101, pp. 743-759, 2015. A. Marshall and J. Meyer, "The directivity and auditory impressions of singers," Acustica, vol. 58, pp. 130-139, 1985. D. Noson, S. Sato, H. Sakai, and Y. Ando, "Singer responses to sound fields with a simulated reflection," Journal of Sound and Vibration, vol. 232, no. 1, pp. 39-51, 2000. ISO3382-1, "Acoustics -Measurement of room acoustic parameters -Part 1: Performance spaces," 2009. Schröder, Dirk und Michael Vorländer, "RAVEN: A Real-Time Framework for the Auralization of Interactive Virtual Environments", Forum Acusticum, Aalborg, Denmark, S. 1541-1546, 2011. M. Yadav, D. Cabrera, and W. L. Martens, "A system for simulating room acoustical environments for one's own voice," Applied Acoustics, vol. 73, pp. 409-414, 2012.

•

  Classical room acoustic measurements (ISO-3382) for room acoustical parameters estimation • Binaural room measurements for dynamic auralization Numerical values for the 1 kHz octave band Aim of this step: provide the singers with real-time auralization so that they can sing in virtual rooms In each real room, impulse responses measured with a Neumann dummy head and a speaker at the mouth position, both placed on a rotating table Samples in the horizontal plane from -40° to +40°, 5° step Based on the monaural impulse response measurements (ISO-3382) in each real room, simulations calibrated within the RAVEN software Replication of the whole circular binaural measurements through simulations • 8 real rooms • 8 simulated rooms • Same musical program • Perform the statistical analysis on the acoustical and physiological data • Perform a verbal analysis on the questionnaires of the singers • Run a listening test to estimate which combination {singing ; acoustics} is preferred • Compare the performances in virtual conditions based on measured vs. simulated impulse responses 2. Kato et al, 2015 • 4 instrumentalists and 1 singer in virtual rooms • Sound rendering through 6 surrounding speakers in an anechoic chamber • Most acoustical features were varied beyond the corresponding just noticeable difference • Audio features and questionnaires showed that musicians consciously adjust their interpretation style 3. Marshall et al, 1985 • Choir singers and soloists in an anechoic chamber • Virtual sound field based on time delays and relative amplitudes of reflections and reverberation • Reverberation controls the comfort of singers • Early reflections are appreciated if they are short (< 40 ms) 2. Noson et al, 2000 • 2 to 4 singers ensembles in a church • Artificial early reflections added by means of couples microphone -speaker • Fast-tempo singing requires strong early reflections whereas slow-tempo singing not significantly impacted Church (V = 9360 m 3 ) T20 = 4.7 s, EDT = 4.6 s, C80 = -5.3 dB Philharmonie Berlin (V = 22000 m 3 ) T20 = 2.0 s, EDT = 2.0 s, C80 = 2.4 dB Joseph Joakim Saal (V = 3660 m 3 ) Without banners T20 = 2.2 s, EDT = 2.1 s, C80 = -1.3 dB With banners T20 = 1.5 s, EDT = 1.6 s, C80 = 0.9 dB Chamber music room (V = 590 m 3 ) T20 = 1.2 s, EDT = 1.0 s, C80 = 4.5 dB Cabaret theater (V = 1700 m 3 ) T20 = 0.7 s, EDT = 0.7 s, C80 = 8.1 dB Recording studio (V = 420 m 3 ) Without banners T20 = 0.7 s, EDT = 0.7 s, C80 = 6.7 dB With banners T20 = 0.5 s, EDT = 0.5 s, C80 = 9.8 dB Anechoic chamber V = 640 m 3 Cutoff frequency = 70 Hz


Paul Luizard

Erik Brauer

Stefan Weinzierl

Nathalie Henrich Bernardoni

Nathalie Henrich

Henrich Bernardoni

How singers adapt to room acoustical conditions

INTRODUCTION

The perception of musical performers with regard to their acoustical environment, which has received increasing attention since the late 1980s [START_REF] Gade | Investigations of musician's room acoustic conditions in concert halls. II: Field experiments and synthesis of results[END_REF][START_REF] Dammerud | Objective Assessment of Acoustic Conditions for Symphony Orchestras[END_REF] , is different from the perception on the part of the audience. The motor activity of the musician, which absorbs much of his attention, the proximity of his instrument and other instruments as sound source(s), as well as the need to hear him-or herself and other ensemble players sufficiently well for a high quality performance can entail substantially different needs and expectations towards the room acoustical conditions on stage.

Recent research has also addressed the question, to what extent and in which way not only the perception but also the way of playing of musical performers depends on the acoustical environment [START_REF] Bolzinger | A study of the influence of room acoustics on piano performance[END_REF][START_REF] Meyer | Acoustics and the performance of music: Manual for acousticians, audio engineers, musicians, architects and musical instrument makers[END_REF][START_REF] Kawai | Experiment on adjustment of piano performance to room acoustics: Analysis of performance coded into MIDI data[END_REF] . In one of these studies, musicians were exposed to different acoustical environments simulated by a multichannel loudspeaker system in an anechoic chamber. As a result, significant differences in terms of tempo, vibrato, and sound level were observed under the different acoustical conditions [START_REF] Ueno | Effect of room acoustics on musicians' performance. Part I: Experimental investigation with a conceptual model[END_REF] . A comparison of features extracted from the audio signal with the response of the musicians given in questionnaires filled out after the recording indicated that many of the adaptations of the interpretation style were made consciously, such as a more pronounced staccato and longer pauses between notes with increasing reverberation time [START_REF] Kato | Effect of Room Acoustics on Musicians' Performance. Part II: Audio Analysis of the Variations in Performed Sound Signals[END_REF] .

Another study [START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Empirical Case Study[END_REF] has been carried out in seven real halls in which a cellist played the same musical program during a concert tour. Room acoustical measurements were performed in the empty rooms, and computer simulations were conducted to estimate the acoustics in the occupied condition. The musical recordings were analyzed in terms of tempo, dynamics, loudness, and timbre characteristics. A statistical analysis of the relationship between room acoustical parameters according to ISO-3382-1 [START_REF]Acoustics -Measurement of room acoustic parameters -Part 1: Performance spaces[END_REF] and audio features representing qualities of the musical performance revealed that more than 50% of the variance in performance features could be explained by the acoustical properties of the environment. With a similar test design, the same authors invited 12 musical performers playing six musical instruments to perform the same pieces of music in different acoustical conditions, simulated by dynamic binaural synthesis, thus avoiding the confusion with other influencing factors such as the visual impression, audience reactions, or the design of the stage [START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Experimental Study[END_REF] . The statistical analysis again revealed a significant influence of room acoustical parameters on the way of playing, but also very individual patterns of reaction, which strongly varied not only between different musical instruments but also between different performers on the same instrument.

The interaction of singers with their room acoustical environment was investigated mainly in the context of choir performance. Marshall and Meyer placed an ensemble of singers in a semianechoic room, where they were exposed to synthetic sound fields including discrete reflections arriving from the side and rear, combined with an artificial reverberation tail [START_REF] Marshall | The Directivity and Auditory Impressions of Singers[END_REF] . The experiment demonstrated the strong influence of reverberation on the comfort of singers. Early reflections were appreciated if their time delay did not exceed 40 ms, beyond which they became unpleasant. A more recent work [START_REF] Noson | Singer responses to sound fields with a simulated reflection[END_REF] focused on the effect of early reflections when singing in small ensembles (2 to 4 singers) in a church. Artificial early reflections were added to the natural acoustics of the venue by means of microphones and loudspeakers. The different groups of singers performed musical pieces with various tempi and as a result, it appeared that singing in fast musical tempi benefits from strong early reflections whereas they do not significantly impact slow-tempo singing.

These results complement well with the work of Ternström [START_REF] Ternström | Choir acoustics: an overview of scientific research published to date[END_REF] , who analysed the performance of three choirs in three different rooms, and demonstrated the influence of room acoustics both on the longtime average spectrum (LTAS) and the sound power produced by the singers [START_REF] Ternström | Long-time average spectrum characteristics of different choirs in different rooms[END_REF] . The way a singer or a speaker adapts his voice production during a performance or a communicative situation in general strongly depends on the auditory feedback the singer or speaker has from his own voice. As a measure for this, Ternström introduced the self-to-other ratio that accounts for the sound level difference between a singer's voice and his surrounding environment, including other singers in choir and sound reverberation in the room (Ternström, 1999). More recent studies have shown that auditory feedback affects not only the power, but also the quality and the pitch accuracy of singers, with the loudness decreasing and voice quality (singing power ratio, SPR) and the pitch accuracy increasing with the degree of auditory feedback [START_REF] Bottalico | Effect of training and level of external auditory feedback on the singing voice: volume and quality[END_REF][START_REF] Bottalico | Effect of Training and Level of External Auditory Feedback on the Singing Voice: Pitch Inaccuracy[END_REF] .

Previous investigations have thus been mainly focused on the interaction between instrumentalists and rooms acoustical conditions, while the singing voice was mainly considered in the context of how choir performance is related to room acoustical qualities [START_REF] Ternström | Physical and acoustic factors that interact with the singer to produce the choral sound[END_REF] . The adaptation of solo singers to the room acoustical conditions, however, has not been investigated in detail. Following the methodology of a previous study 8 conducted at TU Berlin, the present work investigates, to what extent and in which way the performance of singers (as described by features of the audio signal) is related to properties of the performance space (as described by room acoustical parameters). For this purpose, we have invited 4 solo singers to perform an identical repertoire in 6 different rooms covering a wide range or room acoustical conditions. In the following, the measurement approach towards the properties of the rooms and the musical performance is described, as well as a statistical analysis of the interrelation of the two domains.

METHODS

Room acoustics measurements

Stage acoustical measurements were carried out in each of the 6 rooms, documenting a broad range or acoustical characteristics in the selected sample of rooms for the current study (Tab. 1). A system of heavy banners could be deployed in two of these rooms, leading to a considerable change of most acoustical parameters beyond their just noticeable threshold (JND) [START_REF] Seraphim | Investigations on the difference limen of exponential decay for broadband impulse[END_REF][START_REF] Martellotta | The just noticeable difference of center time and clarity index in large reverberant spaces[END_REF] , and resulting in 8 different acoustical environments for the singers. The rooms include a symphonic concert hall (Philharmonie Berlin), a church of medium size (St Eduard Church) a cabaret theater (Distel), a recording studio, a small (Kammersaal) and a medium size chamber music hall (Joseph Joachim Saal), the last three being used by the Berlin University of the Arts (UdK). The measurements were performed by means of a dodecahedron speaker (Norsonic Nor276 and amplifier Nor280) generating logarithmic sine sweeps (3 sec, 20 Hz-20 kHz) and placed on stage at the usual position of a solo singer. A class-1 measurement microphone (NTi Audio M2230) was placed 1 m away from the acoustical center of the source in its horizontal plane. The interaural cross correlation (IACC) was measured on stage by means of a dummy head (Neumann KU100) and a small broadband speaker (Fostex 6301B) with a directivity close to the human voice, and placed just before the mouth of the head.

Musical recordings

In each room, the 4 solo singers involved in this study were invited to sing a musical program, including excerpts of three musical pieces with various tempo and dynamics. The characteristics of the singers and musical pieces are shown in Tab. 2. Recordings were done with a miniature omnidirectional microphone (DPA 4060) positioned 3 cm away from the singer's mouth edge, with an external audio interface (RME Fireface UFX) allowing to capture the voice signal with a very low level of reverberation.

Room name Volume (m 3 ) T30 (s) EDT (s) C80 (dB) G (dB) IACC

Recording 1 -Main parameters of the rooms measured on stage, averaged over the 500 Hz and the 1 kHz octave bands, except for IACC: 125 Hz to 1 kHz octave bands, according to ISO3382-1 [START_REF]Acoustics -Measurement of room acoustic parameters -Part 1: Performance spaces[END_REF] .

Room acoustic parameters

Based on the measured impulse responses from both the omnidirectional microphone and the dummy head, standard room acoustical parameters were calculated according to ISO 3382-1 [START_REF]Acoustics -Measurement of room acoustic parameters -Part 1: Performance spaces[END_REF] . To analyse the interrelation of room acoustical conditions and performance qualities, four parameters were selected which represent the most important perceptual properties of the room [START_REF] Weinzierl | A measuring instrument for the auditory perception of rooms: The Room Acoustical Quality Inventory (RAQI)[END_REF] and which were shown to be largely independent of each other in earlier studies [START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Empirical Case Study[END_REF][START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Experimental Study[END_REF] . These include the early decay time (EDT) as a good predictor of the perceived reverberance [START_REF] Barron | Subjective study of British symphony concert halls[END_REF] , IACCearly as a measure of the perceived source width [START_REF] Beranek | Concert Halls and Opera Houses: Music, Acoustics and Architecture[END_REF] , Glate as a relevant estimation of the subjective level of late sound 2 , and the bass ratio (BR) as a measure of the perception of the tonal color of the room [START_REF] Beranek | Music, acoustics, and architecture[END_REF] . The indices early and late correspond to time intervals before and after 80 ms respectively. The parameters were calculated using the ITA Toolbox 24 (T30, EDT, C80) and custom made codes based on the ISO3382-1 9 standard (G, IACC) and related publications (Glate 2 , IACCearly, and BR [START_REF] Beranek | Concert Halls and Opera Houses: Music, Acoustics and Architecture[END_REF] ).

Voice type Age Chosen musical pieces Pace

Musical descriptors

Following the aim of determining correlations between room acoustics and musical behavior, it is necessary to compare the room acoustical parameters to musical features extracted from the recordings of each singer in each room. Therefore, each recorded note had to be analyzed in order to estimate a number of musical features which could describe a given performance and make it objectively comparable. The first step was to detect the onset of each note. This was performed by a program [START_REF] Lerch | Software-based extraction of objective parameters from music performances[END_REF] previously developed at TU Berlin, which compares a MIDI score with the corresponding recording by means of a dynamic time warping algorithm. The obtained onsets must be manually corrected because onset detection algorithms still present a certain rate of mistakes, especially when dealing with a capella singing [START_REF] Böck | Maximum filter vibrato suppression for onset detection[END_REF] . This correction was based on visual inspection with the SonicVisualiser 27 software. The second step is the estimation, between two consecutive onsets, of the musical features as commonly performed in the music information retrieval (MIR) community. A collection of 4 features was generated, related to tempo at bar level, loudness as defined by Zwicker [START_REF]Method for objective measurements of perceived audio quality[END_REF] , timbre brightness, and dynamics rate as presented in Tab. 3.

Room acoustical parameters

Statistical analysis

To analyse the interrelation of room acoustical parameters and features of the musical performance, the latter variables were z-transformed, according to Xz= X-mean( X ) std ( X )

, ( 1
)
with mean and std as the arithmetic mean and standard deviation of the series X, thus avoiding the influence of the units and the scaling of each variable, only retaining the influence of their relative variation. All possible pairs of variables from both domains were then tested by linear regression, yielding values between 0 (no correlation) and ±1 (perfect positive or negative correlation), along with a p-value, denoting the asymptotic significance of the regression model. These results can be interpreted as quantitative measure to which extent the musical interpretations of the singers are influenced by the acoustics of the room.

RESULTS

General trends

As a first step of analysis, 16 pairwise linear regressions were calculated for all room acoustic and musical descriptors and for all singers and musical pieces. Figure 1 shows all relations with a probability value (statistical significance) of p < 0.2, which were considered as serious trends. They show significant correlations between the spectral centroid, corresponding to the brightness of the singing voice, and the early decay time (EDT). The singers obviously tend to produce a brighter timbre of their voice in rooms with longer reverberation. At the same time, the spectral centroid significantly decreases with increasing bass ratio, i.e. the singers tend to follow the tonal color of the room response instead of compensating for it. The other timbre descriptor, namely the spectral flux, measuring the temporal variation of the power spectrum of the sound, is significantly decreasing with increasing EDT, suggesting that the singers lower the timbral variation of their performance as successive notes tend to overlap due to longer reverberation. This timbre descriptor also tends to decrease as Glate increases, which can be similarly interpreted as a reduced tendency to modulate the timbre of the sound in rooms with high late support. Both the loudness and the tempo of the singers tend to increase with the values of the early interaural cross correlation (IACCearly), considered as a measure of the apparent width of the sound source. Hence, the singers tend to sing louder and faster as they perceive their own voice as more spacious. A reduction of the tempo with increasing reverberation time, as reported in earlier studies [START_REF] Ueno | Effect of room acoustics on musicians' performance. Part I: Experimental investigation with a conceptual model[END_REF] , has been observed in the present study only as a weak trend (p = 0.29).

Figure 1 -Musical descriptors against room acoustical parameters, all in normalized units, for all singers and all pieces. The statistical significance of the linear regressions is indicated as the probability values (p).

Individual strategies

The fact that relatively weak correlations between room acoustical parameters and performance features have been observed for the complete sample of singers and musical pieces can be the result of two different causes: Either there is no strong correlation between room acoustics and the way of singing, or these correlations are very individual, both for the personality of the musician and for the character of the musical piece, so that the dependencies can partly compensate each other.

We have thus put a focus on interrelations which are obviously different for particular singers and particular musical pieces. Some of these individual patterns of reaction can be found for the loudness descriptor used. Considering the relation between loudness and reverberation time (EDT) for the three musical pieces and three different singers (Fig. 2A), it can be seen that singer 3 increased his loudness with increasing EDT, while singers 1 and 4 reduced their loudness, with singer 1 additionally adapting the extent of this reduction to the character of the musical piece.

Similarly individual patterns can be observed for the interrelation between loudness and bass ratio (BR). While singer 3 reduces his loudness with an increasingly dark timbre of the room, singer 4 behaves exactly the other way around, and the reaction of singer 2 seems to depend on the musical piece he sings. It should be noted that most of the linear regressions in Fig. 2A and B do not reach statistical significance due to the small sample of singers, rooms, and musical pieces. They nevertheless demonstrate how individual the strategies of different singers can be, also confirming the results of earlier studies with instrumentalists [START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Experimental Study[END_REF] .

CONCLUSION

This study presents the recordings of four singers who each performed three musical excerpts a capella in eight different room acoustical environments. From a comprehensive analysis of room acoustical parameters and musical performance descriptors extracted from the musical recordings, four parameters from each group were selected, which were shown to be musically and acoustically relevant by earlier studies. The statistical analysis of global interrelations, which are valid for all singers and all musical pieces, show significant correlations for the spectral centroid, which increases with increasing early decay time (EDT) and decreases with increasing bass ratio (BR), and for the spectral flux, which decreases with increasing EDT. The timbral setting of the voice thus seems to be the primary form of adaption to the room acoustical conditions, while the variations in tempo and loudness are either quite weak (Fig. 1), or they are individual for specific singers or specific pieces of music (Fig. 2), and cannot be generalized. This is in line with the analysis made for an instrumental musician in earlier work [START_REF] Schärer Kalkandjiev | The Influence of Room Acoustics on Solo Music Performance: An Empirical Case Study[END_REF] . The duration (EDT) and the tone color (BR) of the reverberation tail have been identified as significant factors triggering performative adjustments, while IACCearly and Glate seemed to be of minor relevance.

Beyond these general tendencies, several adaptation strategies were shown to be quite individual, both for the specific singer and for the specific piece of music. This was particularly noticeable for the individual adjustment of the sound level of the voice, both to the duration and the timbre of the room response. As shown for instrumental musicians 10 , also singers seem to have their own strategies to adapt, consciously or not, to a variety of room acoustical conditions. In future work, these dependencies will be considered both through a larger sample of singers, rooms, and musical pieces, and through advanced statistical analysis, using for instance hierarchical or mixed linear models to account for the complexity of the problem.

Figure 2 -

 2 Figure 2 -Loudness of 3 different singers against early decay time (EDT, row A) and bass ratio (BR, row B).

Table 2 -

 2 Singers and musical pieces

			Mozart -Ach ich liebte (Aria from Entführung aus dem Serail)	Fast
	Soprano	23	Puccini -Aria from Manon Lescaut	Medium
			Verdi -Aria from Rigoletto	Slow
			Schubert -Nur wer die Sehnsucht kennt	Slow
	Mezzo soprano	27	Bach -Esurientes implevit bonis (Aria from Magnificat)	Medium
			Mozart -Aria from Le nozze di Figaro	Fast
			Schubert -Ganymed	Medium
	Tenor	21	Cesti -Intorno all idol mio	Slow
			Händel -Aria from Tamerlano	Fast
			Fauré -Au cimetière	Slow
	Baritone	26	Händel -Honor and arms (Aria from Samson)	Fast
			Humperdinck -Aria from Hänsel und Gretel	Medium

Table 3 -

 3 The selected parameters used for the statistical analysis

	Musical descriptors

Vol. 40. Pt. 3. 2018

Vol. 38. Pt. 3. 2018

ACKNOWLEDGEMENTS

This work was partially funded by the Alexander von Humboldt Foundation with a grant to P. Luizard. The authors would like to thank the singers for their interest in this study, as well as the room managers for letting us use the venues.