Hugo Scurto
email: hugo.scurto@ensad.fr

MUSICKING DEEP REINFORCEMENT LEARNING

In this paper, I relate an auto-reflexive analysis of my practice of designing and musicking deep reinforcement learning. Based on technical description of the Co-Explorer, a deep reinforcement learning agent designed to support sonic exploration through positive or negative human feedback, I discuss how deep reinforcement learning can be seen as a form of sonic comprovisational agent, which enables musicians to compose a parameter sound space, then to engage in embodied improvisation by guiding the agent through sound space using feedback. I then relate on my own musicking experiments led with the Co-Explorer, which resulted to the creation of the aego music performance, and build on these to sketch a music representation for deep reinforcement learning, highlighting its original aesthetics, as well as its ontological shifts between performer and agent, and epistemological tensions with engineering-oriented representations. Rather than discrediting the latters, my wish is to create space for practicebased approaches to machine learning in a way that is complementary to engineering-oriented approaches, while contributing to further music representations and discourses on artificial intelligence.

INTRODUCTION

Reinforcement learning defines a computational framework for the interaction between a learning agent and its environment [START_REF] Sutton | Reinforcement learning: An introduction[END_REF]. The framework provides a basis for agents that learn an optimal behaviour within their environment by taking actions in it, then receiving positive or negative feedback from it, as a reward or punishment signal. Recent advances in deep learning enabled reinforcement learning to be applied to high-dimensional spaces, through the socalled deep reinforcement learning framework [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF]. Such a framework actively contributed to the growing field of artificial intelligence, with application domains ranging from robotics and finance to healthcare and science [START_REF] Franc ¸ois-Lavet | An introduction to deep reinforcement learning[END_REF].

Deep reinforcement learning was recently explored in the domain of music. Kotecha used deep reinforcement learning to generate symbolic polyphonic music [START_REF] Kotecha | Bach2bach: Generating music using a deep reinforcement learning approach[END_REF]. Karbasi et al. explored deep reinforcement learning to create rhythms for a collective of interactive robots [START_REF] Karbasi | A generative model for creating musical rhythms with deep reinforcement learning[END_REF]. Ramoneda et al. applied deep reinforcement learning to learn optimal piano fingerings based on simulated piano performances [START_REF] Ramoneda | Piano fingering with reinforcement learning[END_REF]. Yet, all these works privileged an engineering-oriented approach to deep reinforcement learning, using it as a computational model for existing symbolic music representations. In addition, they did not include musicians in the research and design of these generative models, leaving both analytic and performative aspects of music practice aside.

As a musician, designer and researcher, I was interested in adopting a design-oriented approach to deep reinforcement learning. I was especially interested in exploring novel forms of musicking where a deep reinforcement learning agent would learn interactively from a musician, that is, by receiving positive or negative feedback from them. I was expecting that such a creative process could in turn lead to new designs and representations for deep reinforcement learning that originate from music practice as much as from engineering. I was notably inspired by previous works from Bevilacqua et al., who pioneered such interactive approaches to machine learning for gestural control of sound [START_REF] Bevilacqua | Mnm: a max/msp mapping toolbox[END_REF], and by Fiebrink et al., who highlighted musical attributes of machine learning by leading in-depth studies of the creative process of musicians creating gesture-sound mappings with machine learning [START_REF] Fiebrink | Toward understanding human-computer interaction in composing the instrument[END_REF].

In this paper, I relate an auto-reflexive analysis of my practice of designing and musicking deep reinforcement learning. In Section 2, I describe the Co-Explorer, a deep reinforcement learning agent that supports sonic exploration based on positive or negative human feedback, designed in collaboration with sound designers. In Section 3, I discuss how deep reinforcement learning may be seen as a form of sonic comprovisational agent, enabling musicians to compose a parameter sound space, then to engage in embodied improvisation by guiding the agent through sound space using feedback. In Section 4, I relate on my own musicking experiments with the Co-Explorer, which resulted in the creation of aego, a music performance for one human improviser and one learning machine, presented at this year's TENOR music track. I end by discussing in Section 5 how musicking deep reinforcement learning helped me sketch a music representation for this computational framework, highlighting the epistemological, ontological and aesthetic shifts produced by musicking compared to its standard, engineering-oriented applications. Rather than discrediting the latters, my wish is to create space for practice-based approaches to machine learning in a way that complements engineering-oriented approaches, with the hope that it will contribute to further music representations and discourses on artificial intelligence.

CO-EXPLORER

In this section, I describe the Co-Explorer, a deep reinforcement leaning agent designed to support sonic exploration based on positive or negative feedback provided in real-time by a musician. The Co-Explorer was developed as part of my doctoral thesis, which sought to approach machine learning as design material in the context of new interfaces for musical expression [START_REF] Scurto | Designing with machine learning for interactive music dispositifs[END_REF]. Specifically, we adopted a human-centred design approach to deep reinforcement learning, involving sound designers in diverse steps of our design process, and studying their creative processes with our software agent [START_REF] Scurto | Designing deep reinforcement learning for human parameter exploration[END_REF]. The next sections describe technical foundations of deep reinforcement learning, and more specifically, the exploration method and interaction modalities that we developed within the Co-Explorer 1 . I refer the reader to my previous papers for technical details and qualitative evaluation of implementation.

Interactive Deep Reinforcement Learning

Deep reinforcement learning is a generic computational framework for the interaction between a learning agent and its environment. Our first design step thus consisted in defining a model of the environment and the agent that could be adapted to the use case of sonic exploration. We opted for an elementary model of the environment, consisting of a parameter space of arbitrary dimension (e.g., a synthesis space). Technically speaking, let S = {S} denote the state space constituted by all possible parameter configurations S = (s 1 , ..., s n ) reachable by the agent, with n being the number of parameters, and s i ∈ [s min , s max ] being the value of the i th parameter living in some bounded numerical range. Let A(S) = {A} denote the corresponding action space as moving up or down one of the n parameters by one step a i , except when the selected parameter equals one boundary value. The resulting agent would thus iteratively explore the parameter space while producing continuous sound synthesis variations.

Crucially, we assumed that a musician observes the stateaction trajectories taken by the agent in real-time, and interactively provides positive of negative feedback, or reward R, to the agent. As such, the agent would progressively learn a mapping between states and actions, leveraging deep learning to tackle learning and generalisation in high-dimensional parameter spaces. The resulting trained model can be used as a representation of a musician's subjective preferences toward a parameter space.

Exploration Method

In addition to learning musician's preferences, reinforcement learning agents have a second aim, which is to maximise feedback received from the musician. As such, they may help musicians find the best state-action in the parameter space as they explore it.

To do so, agents rely on exploration methods that enable them to find optimal state-action trajectories in their environment. Intuitively speaking, an agent has to balance exploitation of their computational knowledge (e.g., taking 1 https://github.com/Ircam-RnD/coexplorer the best actions as defined by previous feedback to maximise future feedback) with exploration of their environment (e.g., taking sub-optimal actions in terms of previous feedback, possibly leading to better actions in the future).

For the Co-Explorer, we developed a novel exploration method that builds on an intrinsic motivation technique, which pushes the agent to "explore what surprises it". Specifically, it has the agent direct its exploratory actions toward uncharted parts of the space, rather than simply making random moves, as in most reinforcement learning [START_REF] Sutton | Reinforcement learning: An introduction[END_REF].

Thus, deep reinforcement learning agents may have a dual role: on the one hand, their learned model can be used as a representation of a musician's subjective preferences toward a parameter space; on the other hand, their exploration behaviour can be used to foster the creative process of a musician toward some parameter space.

Interaction Modalities

Our next step consisted in designing interactions with deep reinforcement learning to let musicians experiment with both its learning and exploration abilities. We collaborated with sound designers to iteratively design and implement these interactions within the Co-Explorer (see Figure 1).

The first interaction modality is positive or negative feedback. We distinguished guiding feedback, which enables to provide feedback toward actions taken by the agent, and zone feedback, which enables to provide feedback toward states reached by the agent. While each type of feedback relies on a different implementation, they both consist of a scalar with continuous positive or negative value.

The second interaction modality is state commands. State commands enable to control the agent's trajectory more directly, that is, without relying on feedback. A first state command is changing zone, which enables to command the agent to make an abrupt jump to an unexplored parameter state. Another state command is start/stop autonomous exploration mode. In autonomous exploration mode, the agent takes actions in the parameter space at a regular time interval, whether the musician provides feedback (thus learning in real-time), or not (thus relying on its learned model and exploration behaviour).

The third and last interaction modality is direct parameter manipulation. It enables to explore the parameter space by hand, as in most sound synthesis workflows. Additionally, it enables to choose a given parameter state from which the agent would start its autonomous exploration.

DEEP REINFORCEMENT LEARNING AS SONIC COMPROVISATIONAL AGENT

In this section, I discuss how deep reinforcement learning may be seen as a form of sonic comprovisational agent, enabling musicians to compose a parameter sound space, then to engage in embodied improvisation by guiding the agent through sound space using positive or negative feedback. I detail possible strategies to compose such parametric sound spaces, as well as possible configurations for musical improvisation through positive or negative feedback, which let me argue for deep reinforcement learning as a form of technology for comprovisation [START_REF] Bhagwati | Notational perspective and comprovisation[END_REF].

Composing Sound Spaces

Composing a sound space consists in defining the timbral features and diversity of sounds to be produced by the learning agent. Sound spaces are context-independent, and may be fixed before interacting with the agent. As we will see in Section 4.1, they may be successively explored by the agent to create dramaturgy along a musical piece. Technically speaking, composing sound spaces consist in linking the parameters of the model to parameters of a sound synthesis engine. Concretely, one may first choose one given synthesis engine, then curate n synthesis parameters from it, and set the numerical bounds within which the agent would lead exploration. As the environment's model is generic, one may connect the Co-Explorer to any parameter sound synthesis engines, including commercial VSTs, physically-inspired sound synthesis, descriptorbased sound synthesis, or custom Max/MSP patches. The resulting sound morphologies would continuously evolve across time, as the agent would iteratively take actions on parameters and thus reach new parameter states.

While the Co-Explorer was initially designed to explore sound spaces, the genericity of its environment model makes it theoretically applicable to other music representations. For example, the Co-Explorer was used to explore rhythmic structures, by approaching states as discrete rhythmic patterns of size n, and agent actions as activations or deactivations of beats within the pattern [START_REF] Scurto | Prototyping machine learning through diffractive art practice[END_REF]. Other musical applications could lie in the creation a chord parameter space, and have the agent learning to modify note combinations.

Improvising Through Feedback

Beyond sonic exploration for sound design, I argue that deep reinforcement learning opens new approaches for musical improvisation due to its relying on positive or negative feedback. Below I detail how feedback may be used as a contingent element of a performance, supporting realtime instructions toward sound, symbolic communication with sound, and embodied responses toward sound, all contributing differently to the agent's learning.

Feedback as Instructions Toward Sound

A first musical use of feedback follows that which is technically defined by deep reinforcement learning: namely, enabling performers to provide instructions toward sound to guide the agent's learning and exploration of the space.

As a generic, scalar value, feedback may be directed toward various dimensions of sound. For example, positive or negative feedback may be used to evaluate timbral attributes of sound, so that the agent learns a model of timbre from its parameter environment. Or, feedback may be used to communicate subjective preferences toward sound, so that the agent learns a model of the composer's or performer's tastes toward sound.

In both cases, feedback-based instructions toward sound may be fixed before the performance by the composer. In this case, improvisation would be led by the agent, essentially through its exploration behaviour, while the performer would communicate accurate feedback to teach the agent to reach some goal sound fixed by the composer. Alternatively, such instructions toward sound could be opted for in real-time by the performer. In this case, improvisation would be essentially led by the performer as they would guide agent exploration in real-time through feedback. Specifically, the performer may use feedback to convey spontaneous subjective preferences toward sound, or rely on some sonic scenario, decided before, or emerging from, improvisation, to guide agent exploration. In this case, the reaching of a goal sound may both depend on accurate feedback provided by the performer, as well as on agent learning and exploration of the parameter space.

Feedback as Symbolic Communication With Sound

Rather than sound-oriented instructions, feedback may be reappropriated by the performer to communicate with sound at a symbolic level. For example, a performer may use positive or negative feedback to express personal semantics or imagery toward sound, rather than to evaluate timbral features of sound. In this case, the performer might start to imagine that they are controlling sound production, even if the agent may not be able to properly learn such a high-level representation. Alternatively, a performer may consciously communicate contradictory feedback as a way to hijack the agent's learning, and thus, its trajectory in the sound space. In this case, the performer may have no pre-conceived scenario toward improvisation, except that of discovering unexpected sounds, due to the agent's struggling in interpreting the performer's feedback.

Feedback as Embodied Response To Sound

In addition to instructions or symbolic communication, feedback may be produced by the performer as an embodied response to sound generated by the agent. For example, a performer may produce feedback involuntarily, as errors toward instructions provided by a composer, or as an emotional response toward timbral or symbolic features of sound. Or, a performer may produce feedback to expressively accompany sounds generated by the agent, in a way similar to ancillary gestures produced by musicians with their instruments [START_REF] Godøy | Musical gestures: Sound, movement, and meaning[END_REF]. In the latter case, the performer may approach feedback as an abstract thread that connects them with the agent, thus creating space for expressive improvisation with sound, in a way similar to dance, where movements that accompany music can lead performers to feel that they have control over sound production [START_REF] Leman | Embodied music cognition and mediation technology[END_REF].

Comprovising with Deep Reinforcement Learning

I believe that the combination of context-independent with contingent elements makes deep reinforcement learning a new technology for computer-based comprovisation. In its current formalisation, deep reinforcement learning highlights sound listening as a main feature, where it be in the composition of sound spaces, or during improvisation with the agent. Its second feature is the enabling of musical improvisation through a high-level communication channel, that is, positive or negative feedback, which can be used as either an indirect control modality toward sound generation (in the case of instructions and symbolic communication), or as a direct engagement modality with sound (in the case of symbolic communication and embodied responses). While recent, the framework was already explored by other musicians, specifically, to compose and improvise with musical gestures [START_REF] Visi | Interactive machine learning of musical gesture[END_REF].

MUSICKING EXPERIMENTS

In this section, I relate my own musicking experiments led with deep reinforcement learning, made in collaboration with composer-researcher Axel Chemla-Romeu-Santos between 2019 and today. They resulted in the creation of aego, a music performance for one human improviser and one learning machine. The piece was performed one time in 2019 [START_REF] Scurto | Machine learning for computer music multidisciplinary research: A practical case study[END_REF]; we produced a reworking in 2022, which we will premiere at this year's TENOR music track.

aego started by the wish to experience comprovisation with the Co-Explorer, possibly leading to the discovery of alternative music representations for deep reinforcement learning. We adopted a practice-based approach to the Co-Explorer, that I propose to describe as musicking [START_REF] Small | Musicking: The meanings of performing and listening[END_REF], since it essentially relied on listening to, and performing with, the sounds and music produced by deep reinforcement learning, without assuming any pre-established musical form. The next sections details the compositional, improvisational, and comprovisational experiments led through aego. I refer the reader to our previous paper for aesthetic and technical details on the performance itself [START_REF] Scurto | Machine learning for computer music multidisciplinary research: A practical case study[END_REF].

Composing Latent Sound Spaces

A first aspect of musicking deep reinforcement learning lied in composing parameter sound spaces that the agent will navigate through. For aego, we opted for latent sound spaces, that is, sound spaces created by generative deep learning, another machine learning framework that enables to produce new data that resembles existing data [START_REF] Esling | Bridging audio analysis, perception and synthesis with perceptually-regularized variational timbre spaces[END_REF]. Latent sound spaces have interesting musical features for comprovisation. Specifically, their parameters are not necessarily interpretable as technical synthesis parameters, such as frequency, amplitude, or modulation. Rather, they should reflect perceptual variations of timbre of sound datasets used for learning. Thus, improvising in a latent sound space should generate continuous timbre variations, interpolating between recognisable timbres contained in the training dataset, while also generating ambiguous timbral artifacts typical of generative deep learning [START_REF] Ghisi | Music across music: towards a corpusbased, interactive computer-aided composition[END_REF].

For aego, we opted for two latent sound spaces built over two training datasets: synthesis sounds and acoustic instrument recordings. We stress that we consciously chose these latent spaces in terms of the training datasets they relied on to be created. Yet, we underline that we could not exactly define, nor control, the types of sounds contained in these latent spaces, due to the intrinsic generativity of deep learning [START_REF] Esling | Bridging audio analysis, perception and synthesis with perceptually-regularized variational timbre spaces[END_REF]. As such, we opted for an experimental approach to composing sound spaces, first crafting generative models through their sound dataset, then curating the latent dimensions to be explored by Co-Explorer. This process was highly recursive, as composing latent sound spaces required improvising through gestural feedback to fully grasp their musical attributes.

Improvising Through Gestural Feedback

Indeed, a second aspect of musicking deep reinforcement learning consisted in improvising through feedback with the agent. As a performer, I opted to develop a gestural controller to communicate positive and negative feedback. Specifically, I used inertial measurements units, placed on top of velcro rings, to measure my hands' orientations. I added both angular values and scaled the resulting numerical scalar so that it goes from -1 to 1. My wish was that such a bodily interface would allow for more intuitive and creative musicking with deep reinforcement learning.

In early experiments, I was able to discover the bodily vocabulary enabled by this gestural controller to communicate feedback. The most elementary and illustrative gestures consisted in turning my hands front to communicate positive feedback, and turning them back to communicate negative feedback, by only pivoting wrists. Through improvisation, I discovered other gestures to be explored to provide instructions toward sound, as well as to symbolically communicate with sound. Asymmetric hand postures, for example, enabled to obtain neutral feedback, since the sum of the two angular values would be zero. Yet, the resulting gesture would not be neutral, and would produce expectation and tension for both the performer and the audience. I also explored somatics-based gestures, focusing on internal bodily sensations as I was listening to sound, and producing free-form aerial gestures as embodied response to sounds, resulting in varying feedback values.

All along our experiments, I witnessed myself entering in a state of heightened listening toward sound. Specifically, I observed myself oscillating between two approaches: one that was performative, where I attempted to grasp control over sound by producing precise instructions or symbolic communications, and one that was meditative, where I carefully listened to sound as if it existed by itself, detached from my very own influence, even if my body responding to it in spite of me. Both cases almost had me forgetting about the agent's learning abilities for the benefit of discovering novel sound morphologies, at times witnessing my light influence on it. In short, feedback-based improvisation pushed me to consider both optimal and non-optimal behaviours of deep reinforcement learning as relevant for music performance, while simultaneously contributing to a feeling of spiritual identification with music [START_REF] Scurto | Machine learning for computer music multidisciplinary research: A practical case study[END_REF].

Comprovising a Musical Structure

Based on our musicking experiments, we sought to compose a musical structure for aego (see Figure 2). We chose to start the performance with the latent sound space built over synthesised sounds, since its timbral variations were less chaotic than those produced by the instrumental latent space. We also wrote series of latent dimensions for each space, going from one to eight for the first latent space, then eight to one for the second. As such, widening or narrowing timbral richness in sound spaces let us create dramaturgy across performance for both the performer and the audience.

For our premiere of aego in 2019, the composer was responsible for choosing the moments where the agent would switch latent sound spaces. In our reworking, we renounced to this idea, and had the agent autonomously change zone within a sound space based on its exploration behaviour. This decision was shown to create space for improvisation for the performer, since they would discover the new sound space at time of performance only, thus dealing with even more indeterminacy from the agent during performance.

Similarly, for our premiere of aego in 2019, we had set the time interval between agent's action at a fixed value, which resulted in the agent navigating sound spaces at a slow speed, thus producing a continuous drone sound. In our reworking, we wrote this time interval to produce diverse spectromorphologies along performance, alternating between continuous drone sounds through slow speeds, and glitchy sounds through higher speeds. This let us compose musical tension for both the performer and the audience.

Last but not least, we directed the performance so that the performer progressively relinquishes communication of accurate feedback to the agent, that is, going from instructions to symbolic communication and eventually to embodied response to sound. On the one hand, this choice aimed at improving audience comprehension of the performance, as they would first witness the agent being optimally guided by the performer, then progressively observe the blurring threads of influence between the agent and the performer. On the other hand, this choice enabled us to critically engage with deep generative learning, as embodied responses to sound will lead the agent to produce non-optimal behaviours within the sound spaces. Displaying such indeterminate behaviours as one defining musical attribute of deep reinforcement learning do not conform to established engineering-oriented applications; yet, from our perspective, it speaks of their material engagement with musicians and the world.

FROM COMPUTATIONAL FRAMEWORK TO MUSIC REPRESENTATION

In this section, I sketch contours of a music representation for deep reinforcement learning, which emerged from my musicking experiments with the Co-Explorer. I detail the epistemological tensions over learning models, the ontological shifts between performer and agent, and the aesthetics of feedback-based improvisation, produced by such a music representation for deep reinforcement learning.

Epistemological Tensions over Learning Models

As described in Sections 1 and 2, deep reinforcement learning defines a computational framework for agents that learn by interacting with their environment. Typical applications of deep reinforcement learning seek to learn an optimal behaviour in relation to the goal of a task. Engineeringoriented approaches thus seek to optimise an agent's learning by constructing some synthetic reward function that will yield the best results in terms of learning [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF]. Thus, every other feedback functions can be seen as sub-optimal, or even incorrect, from this engineering perspective [START_REF] Sutton | Reinforcement learning: An introduction[END_REF].

The music-oriented approach to deep reinforcement learning suggests that imperfect human feedback functions for engineering may in turn yield rich forms of improvisation for music research and practice. In fact, I argue that "optimal behaviour" may be a dynamic and emergent attribute of musicking and improvisation, as opposed to the static and pre-existing definition of engineering sciences. For example, one could argue that indeterminacy, as a musical feature of deep reinforcement learning, is what contributes the most to musicking, beyond agent learning or exploration behaviour. Yet, indeterminacy remains a variable that needs minimising in engineering-oriented approaches to deep reinforcement learning. Thus, goals of music and engineer practices may sometimes be opposed.

I believe that such epistemological tensions should be taken seriously by music researchers and practitioners, especially in the current growing applications of artificial intelligence to music, which often reinforce static representations of music through symbolic modelling of existing languages [START_REF] Kotecha | Bach2bach: Generating music using a deep reinforcement learning approach[END_REF]. I suggest that musicking can be one such practice-based approach to discover material attributes of machine learning and illuminate their emerging properties. Rather than discrediting engineering-oriented approaches, I see this highlighting as an opportunity for interdisciplinary collaboration, enabling to iterate the design and implementation of learning models that are entangled with music.

Ontological Shifts Between Performer and Agent

As described in Section 4, musicking deep reinforcement learning enabled me to enter in a state of heightened listening toward sound. This heightened listening had me witness my oscillation between two different postures toward sound and the agent. On the one hand, I would aim at instrumental control over both sound and the agent, using feedback as both sonic instructions and symbolic communications. On the other hand, I would witness the existence of sound beyond myself and the agent itself, as feedback would only help me believe that I control sound.

I argue that this oscillatory phenomenon reveals an ontological displacement of the notions of performer and agent toward sound. In fact, this displacement may drastically differ from other computational frameworks for music improvisation based on machine learning [START_REF] Assayag | Omax brothers: a dynamic yopology of agents for improvization learning[END_REF][START_REF] Nika | Improtek: introducing scenarios into human-computer music improvisation[END_REF][START_REF] Borg | Somax 2: A real-time framework for humanmachine improvisation[END_REF]. The latters often rely on anthropomorphic representations of sound and music, such as MIDI signals, and inject these in the design of the agents. Simultaneously, the performer may also rely on their joint technical and embodied knowledge of music to produce sound with their instrument and interact with the agent. As such, the role of the performer remains clearly defined as that of a musician. The agent, on the other hand, can be described as intelligent, or even as creative, as it builds on the same anthropomorphic music representation than that of the performer, while simultaneously being equipped with a greater musical agency compared to other software for music composition.

In deep reinforcement learning, however, no anthropomorphic representation of sound or music are injected in the design of the agent. Simultaneously, the role of the performer slightly moves away from that of a musician, since they do not rely on their instrumental knowledge to interact with the learning agent, nor do they actually produce sound directly. As such, the role of the performer oscillates between that of a musician and that of a listener, while also creating space for observation of the fluid boundaries that operate between these two roles, thus fostering spiritual identification with the produced sound. In parallel, I suggest that the role of the agent may progressively move away from that of an anthropomorphically-creative agent, to that of a non-human form of intelligence that produces music by conveying temporal form to sound. I believe that such an analysis should be deepened from a musicological perspective to produce alternative discourses toward artificial intelligence: rather than seeking to imitate or replace musicians, machine learning may enable to produce rich forms of musicking that foster human creativity while heightening their listening to their environments.

Aesthetics of Feedback-based Improvisation

As described in Section 3, deep reinforcement learning enables to engage in sonic improvisation by only relying on positive or negative feedback. The resulting interactions include instructions toward sound, symbolic communication with sound, as well as embodied response to sound. In a sense, they move away from standard instrumental techniques to music, as they privilege material attributes of sound over languages usually employed to describe it; indirect influence on sound over precise control and mastery of it; but also and crucially, identification with music over actual sound production. As a result, the values encapsulated in feedback-based music improvisation may differ from values of improvisation found in more traditional written music. In fact, they may lead to ethical encounters with certain communities of music practice and research, for example debating the quality or "truthfulness" of the produced music, or criticising the entertaining aspect or "seriousness" of the designed interactions with sound.

I argue on the contrary that feedback-based music improvisation create novel embodied interactions with sound that produce as "true" music as any other approaches to music composition or performance, and as "serious" interactions with sound than other physical or computer technology for music. If required, I would situate the aesthetics produced by deep reinforcement learning in line with the experimental music movement, in the sense that they push boundaries of existing genres, definitions, or disciplines of music, through their ontological shifts of performer and agent, epistemological tensions over learning models, but also and essentially, through their concrete approach to sound, and their reliance on free improvisation and indeterminacy processes to produce music [START_REF] Nyman | Experimental music: Cage and beyond[END_REF]. In this sense, I suggest that debates toward aesthetics produced by musicking deep reinforcement learning may not fundamentally differ from those opposing conventional and nonconformist music practices with computer technology, long before the current trend for artificial intelligence.

Furthermore, I believe that feedback-based music improvisation summon "serious" social, cultural, bodily and spiritual phenomena related to embodied interaction with sound. Feedback shares similarities with gesture, in the sense that both may be used to translate sound using embodied knowledge and somaesthetic appreciation. In fact, feedback may be closely linked to biosignals, such as muscle tension or heart beats, in the sense that both may sometimes reflect involuntary responses of a musician toward sound. In this sense, feedback as an interaction modality for sound may be shared among diverse communities of people, be they musicians or non-musicians. Going further, I would suggest that the belief of controlling sound, as fostered by feedback, could be of interest for music practices that engage with people with disabilities [START_REF] Parke-Wolfe | Sound control: Supporting custom musical interface design for children with disabilities[END_REF]. Rather than just a basis for entertainment, identification with music have been at the heart of musicking for centuries: I suggest that it may be actively summoned within machine learning design and engineering to empower people toward both music practice and artificial intelligence technology.

CONCLUSION

In this paper, I have reported an auto-reflexive analysis of my practice of designing and musicking deep reinforcement learning. I have described the computational framework of deep reinforcement learning, along with the Co-Explorer, an agent designed to support sonic exploration through positive or negative feedback. I have discussed how deep reinforcement learning may be seen as a form of sonic comprovisational agent, enabling musicians to compose a sound space, then to engage in embodied improvisation by guiding the agent through sound space using feedback. I have reported musicking experiments made with the Co-Explorer, which led to the creation of aego, a music performance for one human improviser and one learning machine. This enabled me to sketch a music representation for deep reinforcement learning, attempting to make its epistemological, ontological, and aesthetic aspects explicit for practitioners and researchers in music and ma-chine learning. I hope that the present work will contribute to further music representations and discourses on artificial intelligence within the TENOR community.

Figure 1 .

 1 Figure 1. Co-Explorer workflow.

Figure 2 .

 2 Figure 2. aego musical structure (2022 reworking).

Acknowledgments

I would like to thank Axel Chemla-Romeu-Santos for collaborating in the composition and performance of aego, as well as Bavo Van Kerrebroeck, Baptiste Caramiaux and Frédéric Bevilacqua for collaborating in the design and implementation of the Co-Explorer, and Raphaël Imbert for pushing me to write on my such musicking experiments with deep reinforcement learning.


Jonas Miguet

Yuan Fang

Florence Rouyer

Emmanuelle Rio

How does the presence of stevia glycosides impact surface bubbles stability?

How does the presence of stevia glycosides impact

surface bubbles stability?

Introduction

The stability of bubbles at a liquid/air interface has been more and more explored in the past ten years [START_REF] Lhuissier | Bursting bubble aerosols[END_REF][START_REF] Poulain | Ageing and burst of surface bubbles[END_REF][START_REF] Poulain | Biosurfactants Change the Thinning of Contaminated Bubbles at Bacteria-Laden Water Interfaces[END_REF][START_REF] Champougny | Life and death of not so "bare" bubbles[END_REF][START_REF] Kočárková | Film drainage of viscous liquid on top of bare bubble: Influence of the bond number[END_REF][START_REF] Lorenceau | Life time of a single bubble on the surface of a water and ethanol bath[END_REF] for the benefit of various applications ranging from the prediction of the climate [START_REF] Boucher | Clouds and Aerosols, book section 7[END_REF][START_REF] Murphy | Influence of sea-salt on aerosol radiative properties in the southern ocean marine boundary layer[END_REF] to the food industry [START_REF] Liger-Belair | Unraveling different chemical fingerprints between a champagne wine and its aerosols[END_REF] or bacterial circulation [START_REF] Poulain | Biosurfactants Change the Thinning of Contaminated Bubbles at Bacteria-Laden Water Interfaces[END_REF][START_REF] Duncan | The ejection of drops from the sea and their enrichment with bacteria and other materials: a review[END_REF].

The transfer of liquid from the oceans to the atmosphere and the production of condensation nuclei is indeed affected by the dispersion of aerosols by rupturing bubbles [START_REF] Johnson | Aerosol generation by modern flush toilets[END_REF][START_REF] Embil | With Exposure to Hypersensitivity Pneumonitis or[END_REF][START_REF] Ruud | Gas release and foam formation during melting and fining of glass[END_REF]. On the other hand, similar aerosols are produced when bubbles rupture at the surface of carbonated beverages and contribute to the sensations of the drinkers through the dispersion of the different flavors.

The starting point of the present work is the observation of the enormous difference between a foam produced by a sucrose-based carbonated beverage and a much more dilute sweetener-based one (see figure 1). It turns out that the foam produced by the latter is more stable, therefore possibly altering the experience of the consumer through two mechanisms. First, a greater foamability and foam stability enhance the probability that the reckless pouring of the beverage leads to a high foam to liquid ratio, a longer waiting time before actual consumption of the liquid, or even to the overflowing of the glass. Second, as shown by Lhuissier et al [START_REF] Lhuissier | Bursting bubble aerosols[END_REF], bubbles with a typical size beyond 1 mm are likely to produce so-called film aerosols upon bursting that are projected in the overlaying air above the glass and eventually deposited in gustatory sensors or evaporated, thus maximizing flavor sensations for the consumer [START_REF] Liger-Belair | Unraveling different chemical fingerprints between a champagne wine and its aerosols[END_REF]. The number of these aerosols depends on the thickness of the film upon bursting, and therefore on their lifetime. We propose here a set-up that allows to study single bubbles stability at the surface of a liquid at rest to compare quantitatively the lifetime of surface bubbles in beverages depending on their composition. Such physical systems need to be studied statistically, since the lifetime of a bubble may inherently not be deterministic [START_REF] Vrij | Free Liquid Films[END_REF]. Nevertheless, it has been shown to be on average well-defined under controlled conditions [START_REF] Saulnier | A study of generation and rupture of soap films[END_REF][START_REF] Champougny | Life and death of not so "bare" bubbles[END_REF][START_REF] Poulain | Ageing and burst of surface bubbles[END_REF]. We thus propose an experiment allowing to measure automatically the bubbles lifetime to extract statistically meaningful lifetime distributions in presence of sucrose and sweeteners. We also test the effect of antifoaming agents which are often added in industrial beverages to correct the enhanced foamability observed in presence of sweeteners.

Material and Methods

Material

Different sweetening products have been tested and compared. Classical "white table sucrose" has been purchased from Sigma Aldrich ((S0389), GC grade, purity ≥ 99.5 %). The different sweeteners are natural extracts of stevia rebaudiana: Stevioside and Rebaudioside A. The three Cram representations of these molecules are pictured in figure 2. More precisely, we use REBA 97 and REBA 99, which are Rebaudioside A with a respective purity of 97 and 99 %, the impurities being mostly other steviol glycosides. We also tested SG95, which is the natural extract of stevia rebaudiana purified up to 95 % and thus a mixture of various steviol glycosides. These three sweeteners are commercially available (PureCircles) and often used in industries. As an antifoam, we used a food grade silicon antifoam (purchased from Momentive Performance Material Inc.). m product m total . The concentrations are chosen to reproduce the industrial concentrations in carbonated beverages. Since the purpose is to obtain a resulting taste as close as possible to that of sucrose but with sweeteners, the right parameter for this choice is the so-called relative sweetness that, in our case, is in the range 250-450 [START_REF] J E Brandle | Stevia rebaudiana: Its agricultural, biological, and chemical properties[END_REF](the manufacturer gives 230 and 270 respectively for stevioside and Rebaudioside A). Therefore, since the concentration of sucrose in cabonated beverages is of the order of 10 wt%, we took a concentration of 0.03 wt% for all the sweetener-based ones. All solutions are prepared using ultra clean water. We systematically add two preservatives to the solutions: Citric Acid and Sodium Benzoate at respective concentrations of 0.13 wt% and 0.012 wt%.

All small masses (sweeteners, preservatives) are weighted using a precision scale (OHAUS Pionneer PA 214) and a weighting pan. Then the water is added and the full mass of the solution (or other big masses like for sucrose) is measured with a coarser scale (KERN 440-47N). Finally a micropipette is used to weight the antifoam, which is a liquid compound, the concentration of which is assessed using a precision scale. For the solutions containing 0.1 or 1 ppm, the final concentrations are obtained by successive dilutions. The solutions are then stirred using a magnetic stirrer until complete dissolution.

Surface tension measurements have demonstrated the presence of impurities in sucrose (see SI). To ensure reproducibility of the experiments, we thus chose to filter all the solutions using a 0.22 µm wide pore-size cellulose membrane with the help of a Büchner system (funnel and flask) and a vacuum pump. As shown in the Supplementary Information, this indeed allow to remove the impurities and to measure a constant surface tension during hours. When some compounds should not be filtered (antifoams), the solutions is filtered prior to adding the corresponding products.

Tensiometry

To measure the surface tension of our solutions of interest, we used a commercial tracker (Teclis Instrument). The associated method is known as the "pendant drop" method although we rather use it in the "rising bubble" mode so that we avoid evaporation or depletion effects and minimize risks of pollution of the interface. The method consists in analyzing the shape of a bubble attached to a syringe [START_REF] Daerr | Measuring liquid surface tension through the pendent drop method : description of a measurement bench and an ImageJ Plugin[END_REF]. All experiments are conducted at around 25 • C. The results are given in Table 1. The surface tension is averaged between 10 and 35 s, which is the lifetime of the surface bubbles.

Another instrument has also been used that relies on a method referred to as Du Noüy-Padday. A small cylinder, with a diameter below 1 mm so that we can neglect buoyancy, is pulled out of the solution and the maximum pulling force F max [N] ie the force at detachment is linked to the surface tension through:

γ = F max 2πr cylinder . ( 1
)
where γ [N/m] is the surface tension and r cylinder [m] the radius of the cylinder. Measurements are repeated 10 times and results are given together with the measured standard deviation in Table 1.

We show in Appendix B that both methods give comparable results.

Evaporation

We measured the evaporation rate for different solutions in order to check whether this parameter is critical or not. To do so, we pour the solution of interest up to the surface of a petri dish of diameter 98.63 mm. We then place it on a high precision scale in a closed chamber in which we can control the relative humidity. We set the humidity to 50 % and wait for at least half an hour for the regulation to proceed.

The weight of the petri dish m(t) [mg] is measured along time (Figure 3) and the mass evaporation rate F [mg/s] is directly the slope of the mass loss versus time data obtained by linear regression (Table 2). Air is injected at the bottom of the container by a pump to create bubbles, the presence of which is assessed using a laser and a photodiode. Images are recorded from the side during the experiments.

Experimental Set-Up for bubble lifetime measurement

Figure 4 shows the set-up that allowed us to repeat measurements of the bubbles lifetime under controlled conditions. The solution of interest is put in a cylindrical container of 4 cm of inner diameter. The liquid/air surface is positioned slightly above that of the container, taking advantage of the meniscus, so that the bubbles can be imaged from the side. At the bottom of the cylinder, a hole is pierced and sealed with a piece of rubber. A stainless steel needle is then vertically planted in the rubber. The syringe is placed so that the drop is generated at a distance of 1 cm from the surface. .This implies, following the results of Zawala et al [START_REF] Zawala | Influence of the Impact Velocity and Size of the Film Formed on Bubble Coalescence Time at Water Surface[END_REF], and given our bubble size, that the bubble velocity is close to its free ascension velocity and therefore closer to the application conditions. It is dressed with a PTFE (polytetrafluoroethylene, hydrophobic) tube of inner diameter 0.4 mm and 1 cm long to avoid any wetting of the injector. The bottom of the needle, outside the cylinder is plugged into a series of tubes going through electronically controlled valves that drive the injection of bubbles by a flow controlled aquarium air pump.

A laser beam is directed to the surface of the liquid bath, where it is directly reflected into a photodiode when the interface is at rest (Figure 4a) or diverges when a bubble is present (Figure 4b). This automated assessment of the presence of a bubble, allows for repeated measurements by creating a new bubble 10 s after the detection of the rupture of the previous one. We are aware that, when a bubble reaches the interface it is susceptible to bounce. However, we never observed this phenomena probably because it is too fast. Indeed, following the results of Zawala et al [START_REF] Zawala | Influence of the Impact Velocity and Size of the Film Formed on Bubble Coalescence Time at Water Surface[END_REF], we can estimate that this phenomenon lasts up to hundreds of milliseconds, which is of the order of our measurements resolution (the frame rate of the camera is 3.75 s -1 ) and can safely be neglected in the development that we propose. We conducted similar experiments, only varying the waiting time between two successive bubbles to check its impact and did not notice any sensible change.

The continuous capture of the side images provides both the size of the bubbles and a reliable measure of the lifetime. This direct measurement allows to avoid the artifacts inherent to the laser detection, i.e presence of very stable daughter bubbles [START_REF] Bird | Daughter bubble cascades produced by folding of ruptured thin films[END_REF] or loss of the laser alignment due to evaporation. The image processing is done using the scikit-image python library to get the lifetime ±0.4 s of the bubbles. As soon as a bubble is detected, its size is measured with a resolution of 30µm, making use of a fitting algorithm based on Hough transform. The full reconstruction of a given serie of measures, making use of the time given by the electronic device connected to the photodiode and that of the camera, allows to ensure that no bubble is missed and that the lifetime is always precisely measured within one image for the birth and the death of a given bubble. The frame rate of the camera being 3.75 s -1 , our resolution is the limiting factor for the measure which uncertainty is then estimated to be 0.4 s. Figure 5 shows the lifetime of individual bubbles (top chart) and their measured size (bottom) chart, as a function of the time elapsed since the beginning of the experiment. This typical experiment has been obtained using Reba 97. The size of the bubbles is very reproducible with an average bubble cap radius of 0.24 cm and a standard deviation of 0.013 cm. This parameter will be kept constant throughout the experiments. The lifetime of individual bubbles does not follow any particular trend during the whole experiment. This is a sign that the system is stable along time.

To clean the tank between each experiment, the system can be rinsed without having to open the box, in which the atmospheric humidity is controlled. Three rinses are made with ultrapure water and one with the next solution of interest, before filling the tank. After using anti-foaming products that are oil-in-water emulsions, we need to open the chamber, clean the funnel assembly of the tank hose with washing-up liquid, rinse thoroughly and re-establish the humidity control in the chamber before repeating the measurement.

Finally, we checked the property of the entire set-up before each experiment by testing the lifetime of surface bubbles in pure water. The surface bubbles then burst a second or less after their generation indicating that the pure water has not been polluted.

Results and Discussion

All the results for the bubbles stability are presented in the form of lifetime distributions (Figures 6,7). The number of bubbles n is written in the legend. The y-axis represents the normalized probability density function (labelled PDF). The number of bins N bins is 100. The integral of the PDF being equals to 1 by definition, this condition leads, in our discrete case, to N bins i=1 N i dt = 1 with N i the normalised probability to get a given value of the lifetime within a given interval and dt the width of the intervals. dt is constant and equals 0.5 s. The radii along with their standard deviations are summarized in table 3 together with the parameters obtained by fitting the distributions as explained in the following.

Solution

Comparison of bubbles stability for different sweetening products

In this section, we will compare the effect of the different sweeteners and of the sucrose on the stability of the bubbles. Figure 6 represents the lifetime distributions for bubbles created with the different products.

The first result is that the presence of sucrose stabilizes the surface bubbles, which live longer than in pure water. Indeed, as mentioned in the experimental set-up, the stability of bubbles made in pure water vanishes because no stabilizing effect exist in absence of surfactants which usually generate a Marangoni elasticity that can sustain the weight of the cap.

A comparison between the surface tension of water and that of 10 wt% sucrose solutions is summarized in table 1. The surface tension is only very little affected with 10 wt% sucrose as compared to water which is consistent with previous studies and the model of Docoslis et al. [START_REF] Docoslis | Influence of the water-air interface on the apparent surface tension of aqueous solutions of hydrophilic solutes[END_REF] including a depletion zone close to the interface. Additional differences between the sucrose and the water solutions are the viscosity and the evaporation rate, which can contribute to the drainage dynamics of the film and therefore affect its stability. The bulk viscosity of a sucrose solution of concentration 10 wt% is approximately 25 % higher than that of water at 20 • C [START_REF] Telis | Viscosity of aqueous carbohydrate solutions at different temperatures and concentrations[END_REF]. Second, as shown in figure 3 andtable 2, the addition of sucrose decreases the evaporation rate of the solution. Both pledge for a decreased thinning rate of the film.

A model has been proposed recently by Poulain et al [START_REF] Poulain | Biosurfactants Change the Thinning of Contaminated Bubbles at Bacteria-Laden Water Interfaces[END_REF] to give an estimation of the bubble lifetime in presence of surfactants. We proposed a modified model [START_REF] Miguet | Stability of big surface bubbles: impact of evaporation and bubble size[END_REF] by including a convective evaporation. This model includes the influence of the surface tension γ, of the viscosity η and of the evaporation rate J. The main physical ingredient is the comparison between the thinning rate due to drainage and the one due to evaporation. If h c is the thickness for which both thinning rates are equal, the lifetime τ is given by hc J . This model leads to a bubble lifetime τ R 7/5 η 2/5 ρ 3/5

J 3/5 γ 2/5 2/5 c ( 2
)
where c = γ ρg is the capillary length depending on the density ρ of the liquid and on the gravitational acceleration g.

We calculated τ for the different sweeteners using the surface tension in table 1, a density of 1000 kg/m 3 and the radius in table 3. The viscosity is η = 10 -3 Pa.s for every solution but sucrose for which we took η = 1.26 × 10 -3 Pa.s. The evaporation rate J is estimated by using the evaporation rate F given in table 2 as J = F Fwater × J water , with J water = 3.64 × 10 -5 kg/m 2 /s [START_REF] Boulogne | Convective evaporation of vertical films[END_REF]. The underlying hypothesis is that the local evaporation rate J scales as the measured global evaporation rate F . The calculated values of τ are reported together with the experimental value of the bubbles lifetime in Table 4. This scaling appears to overestimate the lifetime of surface bubbles in sucrose by a factor of almost 2.

Solution Experimental lifetime [s] τ [s] Sucrose

12.5 24.1 Reba97 25.9 16.6 The second important result here is that all the sweeteners have the same quantitative effect on the stability of bubbles. Indeed, the stability of the bubbles is enhanced in all cases for sweeteners as compared to sucrose, which is consistent with the first remark we made about carbonated beverages made of sweeteners featuring more stable foams (Figure 1). Moreover, it appears very clearly that no significant difference regarding bubbles stability can be found among the different sweeteners. Table 1 summarizes the results of surface tension measurements using the tracker for each sweetener. No noticeable effect of the proportion of the blend or the nature of the molecules used can be seen here. The decreased surface tension of these products as compared to pure water is a sign that they have an affinity for the interface. This is qualitatively explained by the structure of these molecules (Figure 2). Stevioside and Rebaudioside A indeed feature both a hydrophobic part (the aromatic compounds) and some hydrophilic parts (the other aliphatic chains surrounded by hydroxyl groups that can make hydrogen bonds with water). Moreover, as shown by the value calculated in Table 4, the model proposed by Poulain et al gives values very close to the experimental lifetime. Indeed, these amphiphilic groups can reach the interface so that we expect a behavior closer to the one of surface bubbles stabilized by surfactants.

The third result is that not only the average value but also the lifetime distribution is very different for sweeteners and for sucrose. In Figure 6, the

E(t) = 1 σ exp t -µ σ exp -exp t -µ σ . (3)
This distribution exhibit an asymmetric bursting probability. In our case, an interpretation is the presence of a well-defined characteristic time located around µ together with early bursting accidents, i.e. bubbles bursting at earlier time.

The Weibull function has been proposed by Lhuissier et al in [START_REF] Gilet | How long will a bubble be? Physics[END_REF][START_REF] Lhuissier | Bursting bubble aerosols[END_REF] to describe the bursting time distribution in dirty water. It contains one free parameter µ linked to the position of the maximum:

W (t) = 4 3
t 1/3 (0.92µ) 4/3 exp - t 0.92µ 4/3 , (4)
and well describes the data obtained in sucrose because of the long tail. This suggests a dominating stochastic behavior for these systems and therefore, in average, a longer time for a hole to nucleate than for the film to thin down. Finally, our interpretation is that the mechanism leading to bubble bursting in sucrose is closer to the one observed in dirty water, whereas the physical picture in the presence of sweeteners seems to be qualitatively different and closer to surfactant based systems. The higher surface tension of sucrose as compared to sweetener-based systems indeed pledges in favour of a mechanism similar to that proposed by Poulain [START_REF] Poulain | Ageing and burst of surface bubbles[END_REF] for dirty water. In this mechanism, a soluble impurity adsorbs to both sides of the film, locally reduces the surface tension and leads to a divergent Marangoni flow, a subsequent local thinning and the final rupture. This mechanism should lead to bursting events for a larger film thickness. On the other hand, the good correspondence of the sweetener solutions with equation 2 seems to be qualitatively in agreement with surfactantbased systems for which the lifetime is ultimately governed by evaporation [START_REF] Poulain | Biosurfactants Change the Thinning of Contaminated Bubbles at Bacteria-Laden Water Interfaces[END_REF][START_REF] Miguet | Stability of big surface bubbles: impact of evaporation and bubble size[END_REF].

This first experiment demonstrates that our experimental apparatus gives the right diagnostic concerning the prediction of foamability in a gazeous beverage. The sweeteners are indeed molecules, which can populate the interface and account for additional Marangoni stresses able to counterbalance the weight of the liquid film and to stabilize the surface bubbles as well as the foams. In the following, we will use our diagnostic at the scale of surface bubbles to test if antifoaming agents can destabilize surface bubbles so that a lifetime distribution similar to the one observed for sucrose can be achieved.

Bubbles stability in presence of an antifoam

To measure the efficiency of an antifoaming agent on the stability of sweetener solutions, we select Reba 97 as a sweetener. The results in presence of the antifoam at different concentrations are represented in figure 7. At high concentration, the antifoam unsurprisingly completely killed the stability of the bubbles. No difference can be measured in our system between these solutions and ultrapure water. The highest possible efficiency of the antifoaming properties of these products is therefore assessed in this system. In the case of the antifoam at smaller concentrations, the obtained distribution is intermediate and we identified that a concentration of 10 -4 wt% (0.1 ppm) allows to recover an average lifetime close to the one observed with sucrose. Nevertheless, the distribution is still better described by an extreme value distribution and not by a Weibull function. This makes this antifoam at this concentration a good candidate to be added to fizzy beverages in presence of sweeteners to recover a bubble stability close to the one observed in presence of sucrose.

Conclusion

The impact of the physical chemistry of different chemical components potentially found in sodas on the stability of single bubbles has been measured. The stability is assessed by measuring the lifetime of the bubbles. The temperature and the humidity of the environment are controlled, as these parameters are known to affect the system.

The results confirm that single bubbles made with sweeteners, despite a product concentration a few hundred times smaller as compared to sucrose, are significantly more stable. No stability difference was found among the tested sweeteners. This confirms the empirical observation of enhanced foamability and foam stability of sweetener-based sodas as compared to sucrose-based ones. Additionally, our work demonstrate that the replacement of the sucrose by sweeteners also affects the lifetime distribution. Different antifoaming agent concentrations were tested along with the same sweetener, Reba 97 and we showed that an intermediate concentration allows to reproduce the average lifetime of bubbles in a sucrose solution.

These result show how any variation of the recipe can affect the bubbles and foam stability in fizzy beverages, which in turn, can affect the consumer experience. Our experiment suggests that the stability of surface bubbles is a simple controlled system, which can help formulating fizzy beverages. Indeed, since the atmospheric conditions as well as the bubble size are well controlled, our conclusions really concern the formulation.

We would like to emphasize that such a study is only a first step towards a comprehensive understanding of the effect of sweeteners on carbonated beverages. For example, here we consider only air as a gas for obvious experimental reasons whereas, in carbonated beverages, the gas contained in the bubble is C0 2 [START_REF] Stoyan | Do Liquid Films Rupture due to the So-Called Hydrophobic Force or Migration of Dissolved Gases?[END_REF][START_REF] Peter | Surface Tension of Aqueous Solutions of Electrolytes: Relationship with Ion Hydration, Oxygen Solubility, and Bubble Coalescence[END_REF]. A validation of the results for a different gas is beyond the scope of this article but would be very interesting to validate the applicability of our conclusions in real fizzy drinks.

Acknowledgments

We acknowledge the technical contribution of Christophe Courrier for the electronic interface, David Hautemayou and Cédric Mézière for ensuring the mechanical support during the design of the experiment, Vincent Klein and

  (a) 10 wt% sucrose solution (b) 0.05 wt% sweetener solution

Figure 1 :

 1 Figure 1: Foam evolution after the quick pouring of a solution in a glass, top view of a ≈ 7cm diameter glass filled containing around 15 cl. For the concentrations, the notation wt% stands for a relative weight percent.

Figure 2 :

 2 Figure 2: Cram Representations of sweetening molecules

Figure 3 :Table 2 :

 32 Figure 3: Mass contained in a petri dish along time while evaporation occurs in an atmosphere controlled at 50 % humidity.

( a )

 a Undisturbed interface, the laser beam is reflected into the photodiode (b) As a bubble rises to the surface, the laser beam is reflected away from the detector

Figure 4 :

 4 Figure4: In a closed humidity controlled chamber, with a humidity fixed at 50 %, a container is filled through a funnel. Air is injected at the bottom of the container by a pump to create bubbles, the presence of which is assessed using a laser and a photodiode. Images are recorded from the side during the experiments.

Figure 5 :

 5 Figure 5: Example of raw data set for Reba 97. The relative humidity is regulated at 50%. Each point corresponds to a new bubble with a given lifetime and a given bubble radius. (Top) Lifetime of the bubbles as a function of the experimental time. (Bottom) Size of the bubbles as measured by image analysis, as a function of the experimental time. The pictures represent an example of the same raw (top) and processed (bottom) image together with the extracted radius of curvature of the spherical cap R.

Figure 6 :

 6 Figure 6: Lifetime distributions of the different tested sweetening products. n in the legend represents the number of bubbles for a given distribution. The relative humidity is regulated at 50%.

Figure 7 :

 7 Figure 7: Lifetime distributions of Reba97 and the antifoaming agents. "n" in the legend represents the number of bubbles for a given distribution. The relative humidity is regulated at 50%.

Figure 9 :

 9 Figure 9: Measurement of surface tension of pure water with both measurements methods.

Table 1 :

 1 Surface Tension of the different solutions.

	Solution	Method	Mean [mN/m]	Standard Deviation [mN/m]
	Water	Du Noüy-Padday	72.05	0.06
	Sucrose [10 wt%] (without preservatives)	Du Noüy-Padday	72.30	0.5
	Reba 97	Tracker	62.7	0.3
	Reba 99	Tracker	62.2	0.2
	SG 95	Tracker	62.4	0.3
	Reba 97 + Antifoam [0.01 w%]	Tracker	63.6	0.2

Table 3 :

 3 Recap chart of all lifetime experiments

		n	Mean Radius [mm]	Std Radius [mm]	µ [s]	Std Lifetime [s]
	Sucrose	575	2.89	0.17	13.60 not applicable
	Reba97	305	2.40	0.13	25.90	2.73
	Reba99	338	2.51	0.014	23.96	2.42
	SG95	199	2.24	0.015	25.30	2.60
	Reba97 + antifoam 0.1 ppm 306	2.33	0.06	8.69	4.31
	Reba97 + antifoam 1 ppm	313	2.13	0.01	2.34	0.85
	Reba97 + antifoam 50 ppm 103	2.51	0.15	1.32	0.29

Table 4 :

 4 Comparison of the experimental lifetime with the value of τ .

Jérémie Sanchez for building the humidity controller and Laura Wallon for surface tension measurements. This work was funded by PepsiCo R&D. The views expressed in this manuscript are those of the authors and do not necessarily reflect the position or policy of PepsiCo Inc.

Appendix A

We compared surface tension measurements for a high quality sucrose (S0389, GC grade, purity ≥ 99.5 %) before and after filtration. The data were obtained using the set-up with the cylinder in the dynamic surface tension mode described in the material and methods section. The surface tension decreasing with time for unfiltered sucrose is the sign of the presence of a pollution in the solution. On the other hand the constant surface tension observed in sucrose after filtration is a signature that the sucrose is pure enough for our purpose. We thus decided to filter every solutions.

Appendix B

We have used to very classical commercial setups to measure the surface tension. The first one, which is called the Tracker uses the rising bubble method. The second one, which we call the Kybron uses the du Noüy-Padday method. We measured pure water with both methods to show that they both give very similar results (Fig. 9).