Pablo Arias

Laura Rachman

Marco Liuni

Jean-Julien Aucouturier
email: aucouturier@gmail.com

Keywords: voice, emotion, transformation, methods

While acoustic analysis methods have become a commodity in voice emotion research, experiments that attempt, not only to describe, but to computationally manipulate expressive cues in emotional voice and speech have remained relatively rare. We give here a non-technical overview of voice-transformation techniques from the audio signal processing community that we believe are ripe for adoption in this context. We provide sound examples of what they can achieve, examples of experimental questions for which they can be used, and links to open-source implementations.

We point at a number of methodological properties of these algorithms, such as being specific, parametric, exhaustive and real-time, and describe the new possibilities that these open for the experimental study of the emotional voice.

Introduction

A typical approach to study how humans and other animals communicate emotions vocally uses acoustic analysis to quantify the physical features of vocalizations, such as their fundamental frequency (F0), intensity, or spectrum, and seek how they relate to the affect of the speaker [START_REF] Bachorowski | Vocal expression of emotion: Acoustic properties of speech are associated with emotional intensity and context[END_REF] or listener (for a review, see e.g. [START_REF] Scherer | Vocal communication of emotion: A review of research paradigms[END_REF][START_REF] Juslin | Communication of emotions in vocal expression and music performance: Different channels, same code?[END_REF]. Several well-known tools exist in the community for this purpose, including PRAAT 1 for speech data [START_REF] Boersma | Praat, a system for doing phonetics by computer[END_REF], OpenSMILE 2 [START_REF] Eyben | Recent developments in opensmile, the munich open-source multimedia feature extractor[END_REF], MIRToolbox 3 for song and musical data [START_REF] Lartillot | A matlab toolbox for musical feature extraction from audio[END_REF], and Sound Analysis Pro 4 [START_REF] Tchernichovski | Sound analysis pro user manual[END_REF] or Seawave 5 [START_REF] Sueur | Seewave, a free modular tool for sound analysis and synthesis[END_REF] One limitation of this methodology however is that it is intrinsically correlational. Analysing large corpora of speech or vocalizations to establish e.g. that happy voices have statistically higher F0s, faster rate and more animated intonations [START_REF] Banse | Acoustic profiles in vocal emotion expression[END_REF]; for a recent review, see [START_REF] Kamiloglu | Speech representation and transformation using adaptive interpolation of weighted spectrum: vocoder revisited[END_REF] do not allow us to conclude that these features are biological signals that are causally involved in either the decoding of these emotions. For instance, it is now relatively well-described that smiling, the bilateral contraction of the zygomatics facial muscles, has perceivable acoustic consequences on the speaking voice that can be heard, e.g., on the phone [START_REF] Tartter | Hearing smiles and frowns in normal and whisper registers[END_REF]. How listeners recognize smiles in speech, however, is [START_REF] Ohala | The acoustic origin of the smile[END_REF] demonstrated that the resonant frequencies (formants) of a cylindrical clay model of the vocal tract are lowered when the shape of a smile is carved into it (A, middle), and that these new frequencies are similar to that of a non-smiling, but shortened cylinder (A, bottom). He concluded that smiling only has mechanistic consequences on the resonances of the vocal tract filter (A). This is confirmed by the fact that listeners are able to recognize smiled speech in both pitched and non-pitched (whispered) vocalizations (B). However, it is typical that speakers simultaneously raise their pitch while smiling (C), and thus listeners use higher pitch as a cue to identify smiled speech (D), even though it is not causally implicated in smiling. The sole acoustic analysis of speech corpora therefore cannot fully elucidate what cognitive mechanisms are involved in how smile is communicated vocally. Figure adapted from Ohala 1980;[START_REF] Tartter | Hearing smiles and frowns in normal and whisper registers[END_REF][START_REF] Tartter | Happy talk: Perceptual and acoustic effects of smiling on speech[END_REF]; [START_REF] Lasarcyk | Spread lips+ raised larynx+ higher f0= smiled speech?-an articulatory synthesis approach[END_REF] remarkably complicated. On the one hand, physical models show that labial spreading, which reduces vocal tract length, has no mechanistic consequence on the F0 of the glottal source, but only on formant frequencies (Figure 1A;[START_REF] Ohala | The acoustic origin of the smile[END_REF][START_REF] Drahota | The vocal communication of different kinds of smile[END_REF]; and thus, listeners are able to identify smiled speech in the whisper register (Figure 1B; [START_REF] Tartter | Hearing smiles and frowns in normal and whisper registers[END_REF], which has no audible F0.

On the other hand, corpus analyses typically find strong associations between smiled speech and F0 (Figure 1C; [START_REF] Tartter | Happy talk: Perceptual and acoustic effects of smiling on speech[END_REF][START_REF] Barthel | Acoustic-phonetic properties of smiling revised-measurements on a natural video corpus[END_REF], which suggests that it is neurologically difficult for speakers to smile without simultaneously raising their pitch; and thus, in the normal register, listeners reliably use pitch as a cue to identify smiled speech, even though it is not causally implicated in its production (Figure 1D; [START_REF] Lasarcyk | Spread lips+ raised larynx+ higher f0= smiled speech?-an articulatory synthesis approach[END_REF][START_REF] Haddad | An hmm-based speech-smile synthesis system: An approach for amusement synthesis[END_REF]. It is therefore clear, in this field of study like in any other [START_REF] Casadevall | Descriptive science[END_REF] Rather than describing them, one would like the ability to manipulate the acoustic factors of interest in stimuli, in order to confirm experimentally that they causally lead to a change of behavior in the predicted direction when they are perceived. The manipulation of acoustic cues provides an approach complementary to corpus analyses, where the latter can establish a relation between two phenomena (e.g. shifted formants when people smile) and the former can be used to build a model and test for their involvement in perception6 (see also [START_REF] Goldstone | Discovering psychological principles by mining naturally occurring data sets[END_REF]). Yet, while analysis tools are many, experiments that attempt to manipulate acoustic dimensions computationally in complex stimuli such as speech [START_REF] Scherer | Acoustic concomitants of emotional dimensions: Judging affect from synthesized tone sequences[END_REF], music [START_REF] Ilie | A comparison of acoustic cues in music and speech for three dimensions of affect[END_REF], or animal vocalizations [START_REF] Hienz | The discrimination of baboon grunt calls and human vowel sounds by baboons[END_REF]) have been,

until recently, remarkably rare. Perhaps because acoustic transformation tools are perceived to be too technical or of unsufficient quality, a steady stream of research has even preferred less flexible, but more ethologically-valid ways to manipulate vocal characteristics, such as immersing animals in heliox [START_REF] Nowicki | The communicative significance of tonality in birdsong: responses to songs produced in helium[END_REF][START_REF] Rand | Frogs in helium: the anuran vocal sac is not a cavity resonator[END_REF].

Two lines of research have significantly advanced the quest for acoustic control and causal inference in voice and speech research: vocal morphing and speech synthesis. On the one hand, morphing -an algorithmic method to combine two voices by interpolating their spectral features [START_REF] Kawahara | Auditory morphing based on an elastic perceptual distance metric in an interference-free time-frequency representation[END_REF] -, has allowed researchers to describe, e.g., how formants are processed to represent vocal identity [START_REF] Latinus | Normbased coding of voice identity in human auditory cortex[END_REF], whether averaged voices are perceived as more attractive [START_REF] Bruckert | Vocal attractiveness increases by averaging[END_REF] or whether vocal emotion are perceived categorically [START_REF] Laukka | Categorical perception of vocal emotion expressions[END_REF]. However, morphing is generally performed between voices that differ in more than one acoustic dimension. For example, two morphed vocal identities that differ in their fundamental frequency, formant dispersion, and Harmonic-to-Noise Ratio (HNR), will inevitably generate experimental conditions where these acoustic features co-vary [START_REF] Latinus | Normbased coding of voice identity in human auditory cortex[END_REF].

On the other hand, speech synthesis -a vast family of methods allowing to create artificial vocal stimuli from scratch by specifying part or all of their physical parameters (for a review, see e.g. [START_REF] Govind | Expressive speech synthesis: a review[END_REF][START_REF] Malisz | Modern speech synthesis for phonetic sciences: A discussion and an evaluation[END_REF]) -, has been used e.g. to reveal that vocal emotions can be recognised in isolated pitch contours [START_REF] Scherer | Cue utilization in emotion attribution from auditory stimuli[END_REF], to compare the emotional impact of various forms of non-linearities [START_REF] Anikin | The perceptual effects of manipulating nonlinear phenomena in synthetic nonverbal vocalizations[END_REF], or test the effect of formant frequencies on the recognition of smiled speech [START_REF] Quené | Audible smiles and frowns affect speech comprehension[END_REF]. While allowing theoretically unlimited control over the physical properties of the stimuli, synthesis methods have the caveat of of decontextualising acoustic features (e.g. when replacing full speech with isolated pitch contours) and may also suffer from sonic artefacts (e.g. voices sounding robotic and artificial -although see this article's final prospective note on deeplearning techniques).

In our view, voice transformation -the technique to manipulate an original, natural vocal utterance in order to alter a specific acoustic dimension -provides a useful alternative7 to the morphing and synthesis approaches. By giving experimenters the ability to predict how behavioral, physiological or neural reactions vary depending on specific acoustic changes, while leaving all other features unchanged, transformations are well adapted to the nature of hypothesisdriven experimental research. Furthermore, because of recent improvements in vocal transformation algorithms, and their increasing availability in the computer-science communities [START_REF] Stylianou | Voice transformation: a survey[END_REF]Mohammadi and Kain 2017a), it may be time to consider these technologies part of the toolbox of vocal emotion researchers.

In this article, we review the neurological and acoustic bases of emotional voice production, and show that there are in fact well-established, high-quality algorithms that are able to parametrically transform a prerecorded vocal signal at all levels of the voice production pathway. We provide sound examples from our own recent work that illustrate these techniques, examples of experimental questions for which they can be used, and links to open-source software to replicate and extend these studies. Finally, we point at a number of methodological properties of these algorithms, such as being specific, parametric, exhaustive and real-time, and describe the new possibilities, and questions, that these entail for experimental research.

Voice transformations along the vocal production pathway

Voice is produced when the expiratory airflow from the lungs, generated by thoracic and abdominal muscles, sets the vocal folds of the larynx into oscillations. This sound wave, or glottal source, is resonated through the vocal tract, filtered by oral articulators such as the tongue and lips which amplify certain bands of energy (or formants) in its frequency spectrum, and is finally radiated from the mouth and nose (Figure 2; Titze 1994). While the vibration of the vocal folds is a passive process, their oscillatory properties, the airflow that sets them into motion, and the resonance characteristics of the vocal tract are all controlled by over 100 respiratory, laryngeal, and oro-facial muscles [START_REF] Simonyan | Laryngeal motor cortex and control of speech in humans[END_REF], whose motoneurons originate from the spinal cord and brainstem.

In emotional vocalizations, neural control over these muscles involve a hierarchy of cortical and subcortical systems including the periaqueductal gray (PAG), anterior cingulate cortex (ACC) and laryngeal motor cortex (LMC).

Glottal source transformations

Changes in the subglottal pressure due to the contraction of thoracic and abdominal muscles, which are controlled from the anterior horn of the spinal cord, primarily lead to modulations of voice intensity. At moderate intensities, such as in e.g. happy, aroused voices compared to calm or sad voices, the effect of the modulation is carried linearly through the vocal pathway and can be simulated with a simple scalar multiplication of the recording's root mean square (RMS) intensity (see e.g. [START_REF] Ilie | A comparison of acoustic cues in music and speech for three dimensions of affect[END_REF] or, for arbitrary intensity profiles, a piece-wise linear function as implemented e.g. in the reverse-correlation toolbox CLEESE8 [START_REF] Burred | Cleese: An open-source audio-transformation toolbox for datadriven experiments in speech and music cognition[END_REF].

Increased airflow, such as in pain cries or anger shouts, but also possibly altered neurological control over the laryngeal muscles, such as in stress or anxiety, may drive the vocal folds into non-linear/chaotic oscillatory regimes and, more generally, change the shape and periodicity of glottal pulses, which resulting in audible alterations of sound quality such as roughness, noisiness or breathiness (Figure 2C). In voice measurements, such non-linearities are often analysed in terms of jitter and shimmer (cycle-to-cycle variations in the period and amplitude of glottal pulses, respectively) and harmonic-to-noise ratio (HNR) [START_REF] Boersma | Praat, a system for doing phonetics by computer[END_REF]. Such modulations of vocal source quality are important in emotional behaviors [START_REF] Johnstone | The effects of emotions on voice quality[END_REF][START_REF] Gobl | 11 voice source variation and its communicative functions[END_REF] and have been related, in listeners, to subcortical processing by the amygdala [START_REF] Arnal | Human screams occupy a privileged niche in the communication soundscape[END_REF].

Not all glottal source changes are easily simulated with voice transformations. In theory, analysis-resynthesis techniques can be used to, first, estimate the recording's series of glottal pulses [START_REF] Degottex | Mixed source model and its adapted vocal tract filter estimate for voice transformation and synthesis[END_REF]) and, then, resynthesize the vocal signal from a manipulated series of pulses with artificially-varied amplitude and period [START_REF] Verma | Introducing roughness in individuality transformation through jitter modeling and modification[END_REF][START_REF] Ruinskiy | Stochastic models of pitch jitter and amplitude shimmer for voice modification[END_REF][START_REF] Bõhm | Transforming modal voice into irregular voice by amplitude scaling of individual glottal cycles[END_REF]). However, these techniques rely on an explicit model of pulse variability, which is typically learned from one or several target examples of naturally rough voices [START_REF] Bonada | Generation of growl-type voice qualities by spectral morphing[END_REF], and it is unclear how such predetermined patterns should be selected for arbitrary voices. Moreover, because of the computational complexity of the initial stage of glottal source estimation, these techniques cannot operate in real-time. Alternative approaches can also simulate variations in pulse periodicity by overlapping randomly time-shifted copies of the original recording [START_REF] Loscos | Emulating rough and growl voice in spectral domain[END_REF] or modulating it at a divider of F0 to create subharmonics, as implemented e.g. in the ANGUS toolbox9 [START_REF] Gentilucci | Composing vocal distortion: A tool for real-time generation of roughness[END_REF]). However, these only allow exploring a subset of all possible non-linearities (e.g. subharmonics, but not biphonation in general), and vocal source spectrum is one area of vocal production for which pure speechsynthesis approaches, in which variability in the glottal shape or periodicity can be specified explicitly [START_REF] Brady | Synthesizing affect with an analog vocal tract: glottal source[END_REF][START_REF] Anikin | Soundgen: An open-source tool for synthesizing nonverbal vocalizations[END_REF]), may provide more experimental control than transformations.

Sound example S1: Female singing voice, first: original; second: manipulated with subharmonics with the ANGUS toolbox [START_REF] Gentilucci | Composing vocal distortion: A tool for real-time generation of roughness[END_REF].

Note that, because vocal folds or analog anatomical structures are present in a large number of species, source non-linearities are not unique to human vocalizations but are used as a signal of threat and alarm by a wide range of animals, including primates [START_REF] Fitch | Calls out of chaos: the adaptive significance of nonlinear phenomena in mammalian vocal production[END_REF], but also rodents [START_REF] Blumstein | The sound of arousal: The addition of novel non-linearities increases responsiveness in marmot alarm calls[END_REF], canids [START_REF] Wilden | Subharmonics, biphonation, and deterministic chaos in mammal vocalization[END_REF], whales [START_REF] Tyson | Nonlinear phenomena in the vocalizations of north atlantic right whales (eubalaena glacialis) and killer whales (orcinus orca)[END_REF], and birds [START_REF] Fee | The role of nonlinear dynamics of the syrinx in the vocalizations of a songbird[END_REF].

The use of glottal source transformations can therefore be extended to the study of animal behavior.

Changes in the oscillatory properties of the vocal folds, linked e.g. to their length and opening, are mostly controlled by the intrinsic laryngeal muscles, innervated from the vagal nerve originating in the nucleus ambiguus (Amb.) of the medulla, and lead to modulations of vocal source timbre (as above) but also, and perhaps most importantly, of vocal F0 (Figure 2B). A range of techniques exist to manipulate F0. Simple algorithms, as used for example in altered auditory feedback research [START_REF] Hain | Effects of delayed auditory feedback (daf) on the pitch-shift reflex[END_REF] Overlap and Add (PSOLA) method as implemented e.g. in PRAAT [START_REF] Boersma | Praat, a system for doing phonetics by computer[END_REF], the Phase-Vocoder method [START_REF] Moulines | Non-parametric techniques for pitch-scale and time-scale modification of speech[END_REF] as implemented e.g. in CLEESE [START_REF] Burred | Cleese: An open-source audio-transformation toolbox for datadriven experiments in speech and music cognition[END_REF], or pitch-adaptive analyses techniques such as the Adaptive Interpolation of Weighted spectrum method as implemented in STRAIGHT 11 (Kawahara 1997). These transformation methods not only allow raising or lowering the mean pitch of a recording, which may correspond to a baseline change of valence (Figure 3A, see e.g. [START_REF] Ilie | A comparison of acoustic cues in music and speech for three dimensions of affect[END_REF], but can also manipulate the difference between the instantaneous and mean F0 to exaggerate or lessen variations, as seen for example in fearful vs sad vocalizations (Figure 3B, see e.g.

Pell and Kotz 2011), create parametric F0 contours such as vibrato in anxious voices (Figure 3C, see e.g. Bachorowski and Owren 1995), or local intonations at the start or end of an utterance, as in surprised or assertive speech (Figure 3D, see e.g. Jiang and Pell 2017). In addition, most pitchshifting methods can also be used to manipulate the duration or speech rate of utterances (a process known as timestretching), producing faster or slower speech in positive or negative emotional states [START_REF] Scherer | Cue utilization in emotion attribution from auditory stimuli[END_REF].

Sound example S2: Male speech, first: original; second: pitch increased by 50 cents; third: pitch decreased by 50 cents; fourth: pitch modulated with a 8Hz vibrato. All transformations made with delay lines, using the DAVID toolbox [START_REF] Rachman | David: An open-source platform for real-time transformation of infra-segmental emotional cues in running speech[END_REF].

Sound example S3: Female speech, first: original; second-six: random pitch intonations, generated on 6 successive time-windows, using gaussian distributions centered at ±0 cents, SD=±200cents. All transformations made with a phase-vocoder, using the CLEESE toolbox [START_REF] Burred | Cleese: An open-source audio-transformation toolbox for datadriven experiments in speech and music cognition[END_REF].

Sound example S4: Female speech, first: original; second-sixth: random variations of speed rate, generated on 6 successive time-windows, using gaussian distributions centered at ±0%, SD=±10% original duration. All transformations made with a phase-vocoder, using the CLEESE toolbox [START_REF] Burred | Cleese: An open-source audio-transformation toolbox for datadriven experiments in speech and music cognition[END_REF].

Vocal tract transformations

The shape and resonating characteristics of the vocal tract, and thus, the spectral properties of the sound, are modulated by the articulators of the supraglottal region (e.g. lips, tongue, jaw), which are controlled by the oro-facial motoneurons of the facial and trigeminal nuclei of the pons (Figure 2A), but also by the extrinsic laryngeal muscles, whose motoneurons originate from the hypoglossal nucleus of the medulla and which have the ability to raise or lower the position of the larynx within the neck [START_REF] Titze | Principles of Voice Production[END_REF].

As for the glottal source, a number of techniques exist to transform vocal tract characteristics, without altering other aspects of vocal production. Simple techniques, such as the PRAAT "Change Gender..." method [START_REF] Boersma | Praat, a system for doing phonetics by computer[END_REF], exploit the side-effects of the resampling pitchshifting method to reduce or increase formant dispersion, thus simulating changes in vocal tract length and physical dominance [START_REF] Boidron | Emergency medical triage decisions are swayed by computer-manipulated cues of physical dominance in callers voice[END_REF].

Other techniques enabling the manipulation of individual formant frequencies include formant resynthesis methods [START_REF] Quené | Audible smiles and frowns affect speech comprehension[END_REF], spectral envelope manipulations by frequency warping (Arias et al. 2018b) and neural networks [START_REF] Narendranath | Transformation of formants for voice conversion using artificial neural networks[END_REF]. While vocal tract characteristics have been mainly studied in the context of speaker identity (Mohammadi and Kain 2017b), recent work has suggested they are also actively manipulated by emotional oro-facial gestures such as those involved in the expression of disgust [START_REF] Chong | Disgust expressive speech: The acoustic consequences of the facial expression of emotion[END_REF] or smiling (Arias et al. 2018a).

Sound example S5: Female speech, first: original; second-sixth: random variations of vocal tract filter, generated with spectral envelope frequency warping using the CLEESE toolbox [START_REF] Burred | Cleese: An open-source audio-transformation toolbox for datadriven experiments in speech and music cognition[END_REF]. As for glottal source above, vocal tract transformations are also relevant beyond human voice and speech, for the study of animal communication. Formant dispersion correlates with body size in rhesus macaque monkeys [START_REF] Fitch | Vocal tract length and formant frequency dispersion correlate with body size in rhesus macaques[END_REF], and dominant frequency (that of the formant with highest amplitude) inversely correlates with body size across 91 mammalian species [START_REF] Bowling | Body size and vocalization in primates and carnivores[END_REF]. Researchers have used formant manipulations of animal vocalizations in playback experiments to show e.g. that red deer stags are more attentive and reply more to calls from bigger conspecifics [START_REF] Reby | Red deer stags use formants as assessment cues during intrasexual agonistic interactions[END_REF], or that female koalas spend more time attending to vocalizations simulating larger males [START_REF] Charlton | Female koalas prefer bellows in which lower formants indicate larger males[END_REF].

Properties of voice transformations for experimental research

Voice transformation techniques have a number of methodological properties that make them well suited for experimental research, and allow experimental paradigms that would not be otherwise possible with traditional analysis methods or using actor-recorded vocalizations. We list and comment here on a number of these properties, namely that of specificity, parametricity, exhaustivity, and real-time behavior. Despite their advantages for research, assessing whether a given acoustic transformation possesses each of these properties also brings a number of methodological and theoretical difficulties, which we discuss below.

Specificity

Vocal behavior is highly multidimensional and, either for anatomical or neural reasons, it is nearly impossible for speakers to produce for instance only the pitch aspects

Parametricity

A common research question aims to compare how emotional expressions are processed across stimulus conditions, testing e.g. whether listeners perceive emotional speech differently in their native or a foreign language [START_REF] Scherer | Emotion inferences from vocal expression correlate across languages and cultures[END_REF][START_REF] Pell | Recognizing emotions in a foreign language[END_REF], on familiar on nonfamiliar speakers [START_REF] Chen | Impact of acoustic similarity on efficiency of verbal information transmission via subtle prosodic cues[END_REF], male or female voices [START_REF] Bonebright | Gender stereotypes in the expression and perception of vocal affect[END_REF], or even across speech and music [START_REF] Juslin | Communication of emotions in vocal expression and music performance: Different channels, same code?[END_REF]. When using emotional stimuli produced by actors, differences in decoding performance across groups may arise either because of production or perception differences. For instance, if French listeners have difficulties processing emotional cues spoken by Japanese speakers, it may be because their auditory representations of the Japanese phonetic inventory are poor (a perception effect, [START_REF] Dupoux | Epenthetic vowels in japanese: A perceptual illusion?[END_REF], because one does not use the same cues to express joy in Japanese and in French (a production effect, [START_REF] Kitayama | Cultural affordances and emotional experience: socially engaging and disengaging emotions in japan and the united states[END_REF], or both. Balanced designs, in which stimuli produced by more than two culture groups are presented to all participants cross-culturally, is a useful strategy to disentangle such decoding and encoding differences: encoding differences should result in poor recognition across non-native groups, while decoding differences should result in impaired recognition only for a given group, but across stimuli from the other groups [START_REF] Matsumoto | Methodological requirements to test a possible in-group advantage in judging emotions across cultures: Comment on elfenbein and ambady (2002) and evidence[END_REF]Yoshie and Sauter 2019). Once an in-group advantage has been established however, transformation tools that can parametrically manipulate stimuli then become useful to examine the causal mechanisms that underlie such crosscultural differences.

In programming language theory, parametricity is the property of a function that can handle input values identically without depending on their type [START_REF] Pierce | Types and programming languages[END_REF].

We use the term here to describe the property of voice transformations having a uniform acoustic effect regardless of the characteristics of the original signal, their speaker, content or culture. Parametricity opens the possibility to create emotional voice stimuli which utilize exactly the same prosodic cues in exactly the same manner (e.g. a 50-cent pitch increase in the first 2 seconds), and thus to separate the effect of production and perception in intergroup differences.

Data in Figure 4B illustrates this possibility by comparing the effect of the same smile transformation on a variety of recorded sentences by both male and female speakers.

Even though the acoustics of male and female utterances widely differ, notably because of pitch dimorphism [START_REF] Titze | Principles of Voice Production[END_REF], listeners' ratings of speaker positivity in both smile and unsmile transformations can be normalized with respect to original ratings, which allows us to compare the effect of identical cues between sexes. In the same experimental logics, audio transformations have been used to compare identical cues on speech, music and environmental sounds [START_REF] Ilie | A comparison of acoustic cues in music and speech for three dimensions of affect[END_REF][START_REF] Ma | Human emotions track changes in the acoustic environment[END_REF].

Caveat: As for specificity, a number of technical considerations may reduce a given algorithm's parametricity. Some may result from algorithmic design choices. For instance, because F0/pitch is ill-defined on unvoiced portions of speech, pitch transformations, such as vibrato, are often designed to only operate on voiced sections of the signal, leaving transients untransformed. Transformations like vibrato therefore rely on the availability of relatively long voiced portions in phonemes (e.g., 250 ms for two cycles of an 8-Hz-vibrato to be perceived) and, even if specified with identical parameters, may not have as much physical effect on speakers with a faster rather than slower speech rate, or languages with a larger rather than smaller consonant/vowel ratio (see e.g. the failed generalization to Swedish in [START_REF] Rachman | David: An open-source platform for real-time transformation of infra-segmental emotional cues in running speech[END_REF]. Other breaches of parametricity may result from algorithmic limitations. to smaller physical changes of formant frequency F2 (as measured by PRAAT) in male voices than in female voices, making the higher ratings of perceived smiliness measured on transformed female voices difficult to attribute to a purely perceptual effect. It is therefore crucial for researchers to develop a technical understanding, as well as the methodological know-how, to critically assess a transformation's parametricity.

Exhaustivity

Although the human vocal apparatus can produce many different vocal sounds, languages generally use only a small subset of these sounds, and these are not uniformly distributed between and within languages [START_REF] Boer | Self-organization in vowel systems[END_REF].

Many psychophysical procedures, such as adaptive staircase methods but also reverse correlation [START_REF] Jack | Toward a social psychophysics of face communication[END_REF], require presenting participants with random distributions of stimuli in feature space, which is often impractical using naturalistic vocalizations [START_REF] Belin | The sound of trustworthiness: Acoustic-based modulation of perceived voice personality[END_REF]. With voice transformations, experimenters can uniformly or adaptively sample a large space of prosodic variations, e.g. all vibrato frequencies between 1-10Hz, regardless of how common these may be in actual behavior, and are thus able to draw better inferences about how these features are processed.

For instance, in [START_REF] Ponsot | Cracking the social code of speech prosody using reverse correlation[END_REF], we used a phasevocoder transformation to generate more than 70,000 random prosodic variations from a single recording of the word 'bonjour' (hello), and used reverse correlation to uncover participants' mental representations of a dominant or trustworthy way to pronounce that word. Similar paradigms can be applied to study how healthy participants and patients mentally represent emotional prosody, and address e.g. emotion perception deficits in pathologies like ASD or amusia.

Caveat: While their parameters can be explored exhaustively, most transformation techniques do not implement boundaries on what is or isn't physiologically possible and thus, beyond a certain parameter range, may not not sound like authentic human speech. In addition, even within realistic parameter ranges, transformations may generate artefacts, such as unnatural timbres, doubling of F0 or smearing of transients, and give the transformed sound a robotic, artificial tone. Even if the impact of artificiality on emotional judgements remains unclear [START_REF] Burleigh | Does the uncanny valley exist? an empirical test of the relationship between eeriness and the human likeness of digitally created faces[END_REF], one can rightly question the ecological validity of behaviors measured in response to such stimuli. While most artefacts can be avoided with simple heuristics (e.g. basing parameters on the range of variation measured in natural voices, or clipping sampling distributions at ±2SD), quantifying how natural, or easily detectable, a given transformation sounds to participants can be remarkably complicated. First, the acceptance of transformed voice as authentic is heavily dependent on context.

For instance, transformations may be more easily detected in situations where variants can be compared with the original sound. Second, judgements of naturalness are multifaceted, and likely incorporate evaluations of biological plausibility (this doesn't sound human), agentivity (this sound tampered with), vocal or social typicality (no one in their right mind would do this) or even semantics (it does not make sense to say this with a happy voice). Future research will benefit from more principled ways to measure transformation naturalness and its impact on participants.

Real-time behavior

One typical way to study the role of emotional expressions in social interactions is either to explicitly instruct social partners to display a certain emotion (e.g. Tice 1992), or to indirectly lead them to express it using a cover story (e.g. [START_REF] Van Doorn | Emotion and the construal of social situations: Inferences of cooperation versus competition from expressions of anger, happiness, and disappointment[END_REF]. Because many voice-transformation techniques allow real-time processing (typically, 50-100ms latency for transformations based on phase-vocoder, [START_REF] Lee | An analysis of startup and dynamic latency in phase vocoder-based time-stretching algorithms[END_REF], they open the possibility to control emotional expression in continuous, real-time interactions (e.g. on the phone) with no experimental demand, and possibly even without participants' awareness of the manipulation. For instance, we have used a vocal tract transformation to manipulate the perceived body size of mock patients calling a medical call center simulator [START_REF] Boidron | Emergency medical triage decisions are swayed by computer-manipulated cues of physical dominance in callers voice[END_REF], and found that callers whose voice were perceived as indicative of physical dominance obtained a higher grade of response, a higher evaluation of medical emergency and longer attention from physicians than callers with strictly identical medical needs whose voice signaled lower dominance. Similar paradigms can be used e.g. to study how congruent or incongruent emotional expressions influence the outcome of group decision-making, or group creativity.

In addition, some voice-transformation techniques, such as those operating in the time-domain [START_REF] Juillerat | Low latency audio pitch shifting in the time domain[END_REF], can be so fast that they can not only manipulate a social partner's voice without disrupting the flow of interaction, but also manipulate the participant's own voice without disrupting their sensorimotor feedback (e.g. with less than 20ms latency between the original input voice and the manipulated output). This opens the possibility to build altered auditory feedback paradigms and test how e.g. hearing one's voice with a happier or sadder tone influences one's emotional experiences, judgments or decisions. For instance, we used a time-domain transformation of pitch to modify participants' voices in a happy or sad direction as they read out an emotional neutral text, and found that participants who heard themselves with emotionally manipulated voices reported significantly different moods, as well as elevated levels of skin conductance [START_REF] Aucouturier | Covert digital manipulation of vocal emotion alter speakers emotional states in a congruent direction[END_REF]. Similar paradigms can be used to study e.g. The DNN architecture, which directly models waveforms using a neural network method trained with recordings of real speech, provides superior acoustic quality for several high-level vocal applications such as Text-to-Speech [START_REF] Wang | Tacotron: Towards end-to-end speech synthesis[END_REF]) and speech enhancement [START_REF] Pascual | Segan: Speech enhancement generative adversarial network[END_REF]), but also to generate expressive voices [START_REF] Lee | Emotional end-to-end neural speech synthesizer[END_REF][START_REF] Akuzawa | Expressive speech synthesis via modeling expressions with variational autoencoder[END_REF] or convert their emotions [START_REF] Luo | Emotional voice conversion with adaptive scales f0 based on wavelet transform using limited amount of emotional data[END_REF]). However, although highly flexible, these systems have so far failed to exhibit the level of parametric control that we argue here is needed for experimental applications. In this article, we reviewed a wide range of recent (or notso-recent) technologies that allow researchers to manipulate specific acoustic features along the voice production pathway. From the glottal source to the vocal tract, we presented the acoustic consequence of each of the building blocks of the vocal apparatus, as well as corresponding acoustic models and transformation algorithms from the signal-processing literature.

We suggest that using such transformations to control the content of vocal stimuli in experimental research is a promising line of work. This methodology allows researchers to formulate and test computational predictions about the behavioral, physiological and neural consequences of specific acoustic changes, enabling them to draw strong, causal links between the anatomic mechanisms involved in voice production and their subsequent reactions in listeners.

Transformation technologies can easily be shared between research groups, made open source, and deployed across several types of studies (e.g. cross-linguistic, cross-species) and auditory modalities, such as non-verbal behaviour, speech and music. When possible, we gave here links to some of these tools that are available freely, as well as examples of studies that uses them, and hope that this list will only be growing. However, in order for these technologies to be useful in an experimental context, they must deliver the proper type of acoustic control. We identified four of such constraints.

  for animal communication. The availability of dedicated software has an important impact on research: it gives access to audio signal processing techniques such as F0 extraction without needing a technical background; it helps standardize the definition of vocal features, by providing reference implementations (when one studies jitter while referencing PRAAT, others know what is meant and how to reproduce the work); and it provides an interdisciplinary interface between the research that creates these tools and the research that uses them, to share new techniques and new research needs.

Figure 1 .

 1 Figure 1. The confusing case of fundamental frequency in smiled speech. Ohala (1980) demonstrated that the resonant

  , that correlation does not imply causation and that the sole acoustic analysis of what is incidentally present in voice and speech may obfuscate the mechanisms with which emotions are produced, or recognized (see Armstrong et al. 2019 for a similar argument on the signaling of body size by low F0 voices).

  These subcortical and cortical influences on muscle actuators at every stage of the vocal production pathway have different, complementary effects on the final acoustic properties of the vocal signals, and specific voice-transformation techniques exist to reproduce these changes in ecological voice and speech recordings.

Figure 2 .

 2 Figure 2. Neurological, anatomical and acoustic characteristics of the vocal production pathway. Voice is produced when the expiratory airflow from the lungs sets the vocal folds of the larynx into oscillations (right). This sound wave, or glottal source, is resonated through the vocal tract and filtered by oral articulators such as the tongue and lips (left) and is finally radiated from the mouth and nose. Neural control at different levels of this pathway involve a hierarchy of cortical and subcortical systems including premotor nuclei of the brainstem such as the nucleus ambiguus (Amb.), periaqueductal gray (PAG), anterior cingulate cortex (ACC) and laryngeal motor cortex (LMC). Voice-transformation techniques exist to parametrically manipulate a prerecorded vocal signal at all levels of the voice production pathway: vocal tract filter (A), pitch and intonation (B), and glottal source timbre (C).

  ) and implemented in the DAVID toolbox 10 (Rachman et al. 2018), are based on resampling or multiple delay lines (a technique that introduces a small delay to an audio signal in order to play it faster/slower, thus raising/lowering its pitch, Dattorro 1997) and may alter vocal tract filtering or formants unrealistically beyond small parametric changes. State-of-the-art techniques that allow separating source and filter information to avoid such artifacts are based on reconstructions of the signal's short-time Fourier transform (STFT) at non-uniform rates, such as the Pitch Synchronous

  Sound example S6: Female speech, first & third: original; second & fourth: formant shifted upwards to simulate smiling, generated with frequency warping(Arias et al. 2018b).

  10 http://forumnet.ircam.fr/product/david 11 http://www.wakayama-u.ac.jp/ ˜kawahara/ STRAIGHTadv/index_e.html

Figure 3 .

 3 Figure 3. Four examples of pitch transformations on a single recording of the sentence "I would like a new alarm clock", produced by a male English speaker. The original pitch values are presented in black and the transformed pitch values in magenta. The speech waveform is shown on the x-axis and the dotted line indicates the mean F0 of the recording. (A) 100-cent upward pitch shift, applied uniformly over time. (B) Exaggerated pitch dynamics by a 15% increase of pitch values with respect to the mean pitch. (C) Vibrato applied with a 100 cent depth and a rate of 8.5 Hz. (D) Increasing pitch at the end of the utterance.

  of a sad expression without simultaneously varying timbre or duration, or timbre aspects without simultaneously varying pitch (Figure 1). Co-variation of features in naturalistic recordings means that one cannot conclusively establish what acoustic property drive listeners' emotional evaluations. The ability to transform voice along specific signal dimensions, such as pitch or timbre, while preserving all other aspects of the original, opens the possibility to create pairs of stimuli (original and transformed) that differ only in one experimental factor. Comparing behavioral or physiological measures within pairs thus allows controlling for co-varying factors and identify causal relations that would otherwise be missed in noise or, worse, spuriously attributed to the wrong factor.

Figure

  Figure4Aillustrates this situation with recent data fromArias et al. (2018a), in which we manipulated 20 original male and female recordings with a vocal tract transformation designed to simulate the acoustic effect of smiling, and asked 35 participants to evaluate their perceived happiness of the speaker. Each recording was transformed into two variants, smiled and unsmiled versions, respectively expanding or compressing formants by the same amount. These two versions are thus two opposite transformations relative to a neutral, non-manipulated stimulus. Because the transformation did not affect glottal source properties, the F0s of the original recordings were preserved by the two transformations, and ranged from 117 to 300 Hz. While there

Figure 4 .

 4 Figure 4. Attributing ratings of speaker happiness to manipulated vocal tract cues in smiled speech. (A) Mean rating of happiness for 20 male and female recordings, as a function of the recording's mean pitch and sound manipulation : smile (dark green), neutral (black), unsmile (light green); each dot represents one separate recording; black arrows link unsmile with smile transformations of the same original recording. (B) Normalized ratings of happiness for male and female recordings manipulated with smile (light green) and unsmile (dark green) transformations; asterisks indicate significant statistical differences between the smile and unsmile distributions, computed with paired t-tests (p<0.01); all ratings normalised by the corresponding neutral (non-manipulated) rating. (C) Difference of formant frequency (in cents) between smile (dark green) and unsmile (smile green) transformations and the corresponding non-manipulated sound (formant 1: F1, formant 2: F2) ; asterisks indicate significant statistical difference between the distributions, computed with paired t-tests (p<0.01)

Figure

  Figure 4C illustrates this situation: despite the identical transformation parameters, the smile transformation as implemented in Arias et al. (2018a) unintentionally leads

  whether personal emotional memories can be re-encoded with different valence when healthy participants or patients hear themselves narrate them with a transformed tone of voice. Caveat: Once given the possibility to transform continuous speech in real-time interactions, the immediate next question concerns the contextual adaptation of transformations to speech content. While transformation parametricity is methodologicaly useful, applying the same pitch increase on spoken sentences regardless of, say, their original prosody or stress words (which are also marked with pitch -Pell 2001) may create unnatural or misadaptative expressions, and it is computationally unclear how to adapt transformations to speech content, especially in a real-time context. In that respect, voice transformations should not be considered experimental materials, but experimental methods. They do not replace actor-produced stimuli nor provide ready-made expressions with which to study the impact of emotions on behavior (in truth, transformed vocalizations may be less intense, less natural and less well-recognized than natural expressions, see e.g. Rachman et al. 2018). Rather, by manipulating signal properties at all stages of the vocal production pathway, they provide control over the physical properties of the stimuli, bringing unprecedented precision on the neurological, anatomical and acoustic components of what makes voice and speech emotional, but leaving it to the experimenter to construe how speakers and listeners integrate these components with other aspects of affective and cognitive processing. A prospective note on deep learning techniques In recent years, artificial voices have become an integral part of consumer electronic appliances (e.g. smart assistants, car navigators, augmentative and alternative communication) and the amount of funding private companies such as Amazon, Google or Apple have injected into the speech synthesis community has transformed the field into a fastpaced and competitive domain. In just a few years, neural machine learning (deep neural network, DNNs) operating on waveform samples, as opposed to e.g. spectrogram features, has become the de-facto standard in voice synthesis (Oord et al. 2016).

  First, machine-learning speech generally emulates vocal patterns learned from large sets of recordings where all vocal features co-vary, and therefore typically lacks feature specificity and exhaustivity. Second, deep-learning architectures do not allow easy introspection into how information is represented in the network, making it difficult to know what exact vocal features are being manipulated and lacking interpretability.As far as we know, these limitations have so far prevented the application of DNNs for the kind of experimental work reviewed here (although see[START_REF] Sun | Brain2char: A deep architecture for decoding text from brain recordings[END_REF]). However, progress in the field is fast. One promising line of research uses post-hoc, data-driven methods to reveal how stimulus information is encoded into network layers[START_REF] Hsu | Learning latent representations for speech generation and transformation[END_REF]). In the visual sciences, these methods have been used to compare what visual features human and machine use to achieve face classification[START_REF] Xu | Deeper interpretability of deep networks[END_REF], and similar approaches could be used for speech. Another relevant line of research aims to create DNN-based vocoders[START_REF] Wu | Quasi-periodic wavenet vocoder: a pitch dependent dilated convolution model for parametric speech generation[END_REF], in which, like in traditional vocoders reviewed above, speech synthesis can be controled with specific pitch or duration parameters, while conserving the acoustic performance of deep learning models. Finally, Generative Adversarial Networks (GANs), a special class of DNN architecture capable of learning a deterministic mapping from one style of stimulus to another[START_REF] Goodfellow | Generative adversarial nets[END_REF], are increasingly used to create visual transformations (e.g. smiles[START_REF] Wang | Every smile is unique: Landmark-guided diverse smile generation[END_REF] and have also started to be applied to speech transformations. For example, GANs were recently used to transform a voice into its Lombard counterpart (a particular type of vocal effort which makes the voice more intelligible in background noise;[START_REF] Seshadri | Augmented cyclegans for continuous scale normal-to-lombard speaking style conversion[END_REF]). All such advances open exciting new possibilities to create emotional voice and speech transformations, which will certainly find their way to the community in the upcoming years.ConclusionSo far, the experimental study of emotional voice and speech has largely relied on acoustic analyses of datasets of natural emotional vocalisations, or the use of these recordings as stimuli in experimental research. While natural vocalizations have the advantage of being realistic and ecological, they often vary in several acoustic dimensions, making mechanistic conclusions difficult to establish. Yet, much is known about the essential anatomic building blocks involved in emotional vocal production (e.g. laryngeal muscle tension, vocal fold oscillatory regime, oro-facial gestures). These anatomic mechanisms are controlled by increasingly wellidentified neural structures and have specific, and to some extent independent, acoustic consequences that can be modeled computationally -because they have a physical basis.

  First, in our view, transformations should be specific, i.e. transform sound in a single acoustic dimension mirroring an isolated anatomic mechanism (e.g. vocal fold saturation or zygomatic muscle contraction). Second, transformations should be parametric, i.e. have a uniform acoustic effect regardless of the original signal (e.g. semantic content, age, identity, sex, species), thus allowing comparative studies. Third, transformations should be exhaustive, i.e. unconstrained by what speakers usually produce, but rather by what they can produce, in order to reduce sampling bias for psychophysical research. Finally, in an era where the study of social interactions is at the top of the cognitive-science research agenda, the community should favor transformations that can operate in real-time. We hope that these recommendations can be used as a 'check-list' for machine-learning and signal-processing researchers involved in creating new vocal transformations. If new tools, included those emerging from the recent trend of deep-learning research, follow these constraints, they will be more easily used for experimental research..

It should be noted that transformation do not help to establish the involvement of the cues in the production/expression of emotion; for this, one has to manipulate the actual emotional state of the signalers (e.g.[START_REF] Bachorowski | Vocal expression of emotion: Acoustic properties of speech are associated with emotional intensity and context[END_REF]

Note that, strictly speaking, there is a technological overlap between voice transformation and certain forms of (re-)synthesis techniques, which analyse an original signal down to its generative parameters and then resynthesize a variant of the signal by altering these parameters. We review some of these techniques in the rest of the paper under the encompassing term of transformation.

http://forumnet.ircam.fr/product/cleese Prepared using sagej.cls

http://forumnet.ircam.fr/product/angus Prepared using sagej.cls

Prepared using sagej.cls

Acknowledgements P.A., L.R. and M.L. produced the experimental data and software on which this review was based. JJ.A. wrote the manuscript, with contributions from all other authors. Data in Figure 4 collected at the Sorbonne-Universit INSEAD Center for Behavioural Sciences. Work funded by ERC StG 335536 (CREAM) and ANR REFLETS to JJA. We thank Gilles Degottex for feedback on Figure 1. Facial expressions in Figure 1 are from the extended Cohn-Kanade dataset (ck+), S106 and S113 (Lucey et al. 2010).

Funding

Work funded by ERC StG CREAM (335536) and ANR REFLETS (2017-2021) to JJA.


Caroline Michel

Fabienne Battaglia-Brunet

Catherine Joulian

Preservation of microbial consortia: impact of

Keywords: consortium preservation, mixed microbial community, arsenic bio-oxidation, activity, diversity

come

Preservation of microbial consortia: impact of storage at 4°C and -80°C on an As(III)-oxidizing community

Caroline MICHEL, Fabienne BATTAGLIA-BRUNET, Catherine JOULIAN BRGM, DEPA/GME (Geomicrobiologie et Monitoring Environemental), 3 avenue Claude Guillemin, BP 36000, 45060 Orléans cedex 2 (France)

The use of mixed microbial communities in biotechnologies offers many advantages compared to single/pure strains (Brenner et al., 2008):

-They can face environmental changes in an easier manner than monocultures can do.

-They can perform complicated tasks/functions such as that requiring multiple steps, which can be difficult or even impossible for many individual strains. -Microbial consortia represent a source of genes and functions, as well as resources for future and novel biotechnological applications as they involve non-cultivable and/or uncharacterized microorganisms. -Microbial communities are naturally mixed in the environment and consortia are thus more representative of the "real" life.

However, the preservation, especially over the long-term, of stable microbial consortia is still underlined as a challenge. It has become a necessity to safeguard the accessibility to microorganisms already recognized to have interesting features and to be a pool of microbial resources. Several factors are known to be critical for a good storage, such as cell size and type, temperature, amount of cells, physiological state, the use of a cryoprotectant for cryoconservation and lyoprotectants for lyophilisation, the period of time of cryoprotectant/cells contact before cooling, the rates of cooling and thawing… These parameters have been identified as critical for the storage of pure microbial strains but little is known on how to preserve microbial consortia.

Here, the impact of two ways of storage widely used in laboratory, cryoconservation at -80°C in 20% glycerol and 4°C storage, was studied on a microbial community selected for its ability to oxidize As(III) into As(V), an important function in the frame of As bioremediation purposes. The community was characterized before and after storage, over a period of up to 12 months, focusing on the following parameters: viability (Live and Dead labelling approach), cell growth (microscopic counts), activity (As(III) oxidation and carbon sources utilization Biolog profile), and diversity fingerprints. Results showed no impact of both ways of storage on As(III) oxidation ability and cell growth, but storage led to an impact on cells viability. An increase of biodiversity after storage at -80°C was observed, probably due to the presence of glycerol that may have allowed the growth of strains that were in minority before storage. On the opposite, storage at -4°C tended to decrease biodiversity. Results also showed an evolution in carbon source utilization that varied according to the temperature and the time of storage. In conclusion, the function of interest, As(III) oxidation, was well preserved whatever storage temperature and duration, but microbial diversity and viability were impacted, even during short term storage.