Rania Abdelghani

Hélène Sauzéon

Pierre-Yves Oudeyer

Generative AI in the Classroom: Can Students Remain Active Learners?

Generative Artificial Intelligence (GAI) can be seen as a double-edged weapon in education. Indeed, it may provide personalized, interactive and empowering pedagogical sequences that could favor students' intrinsic motivation, active engagement and help them have more control over their learning. But at the same time, other GAI properties such as the lack of uncertainty signalling even in cases of failure (particularly with Large Language Models (LLMs)) could lead to opposite effects, e.g. over-estimation of one's own competencies, passiveness, loss of curious and critical-thinking sense, etc. These negative effects are due in particular to the lack of a pedagogical stance in these models' behaviors. Indeed, as opposed to standard pedagogical activities, GAI systems are often designed to answers users' inquiries easily and conveniently, without asking them to make an effort, and without focusing on their learning process and/or outcomes. This article starts by outlining some of these opportunities and challenges surrounding the use of GAI in education, with a focus on the effects on students' active learning strategies and related metacognitive skills. Then, we present a framework for introducing pedagogical transparency in GAI-based educational applications. This framework presents 1) training methods to include pedagogical principles in the models, 2) methods to ensure controlled and pedagogically-relevant interactions when designing activities with GAI and 3) educational methods enabling students to acquire the relevant skills to properly benefit from the use of GAI in their learning activities (meta-cognitive skills, GAI litteracy).

Introduction

The rise of generative artificial intelligence (GAI) represents an unprecedented opportunity for dealing with the multiple challenges facing the educational field today. Indeed, in a context where students are having more and more diverse needs, and educational establishments suffering from more and more teachers shortages, GAI can play a crucial role. One of the most straightforward implications is using LLMs to help educators create pedagogical content, personalized learning sequences and feedback, attractive and collaborative interactions [START_REF] Kasneci | ChatGPT for good? On opportunities and challenges of large language models for education[END_REF], etc. Beyond helping teachers be more efficient, these new opportunities can also facilitate students' intrinsic motivation and active engagement during learning [START_REF] Brett | Motivating students to engage in learning: the MUSIC model of academic motivation[END_REF]. This is indeed a strong argument in favour of using LLMs in education since such factors are shown to be fundamental for an efficient and enjoyable learning experience [START_REF] Borah | Motivation in learning[END_REF][START_REF] Divjak | The impact of game-based learning on the achievement of learning goals and motivation for learning mathematics-literature review[END_REF][START_REF] Lin | A study of the effects of digital learning on learning motivation and learning outcome[END_REF], ... . But although these seem as undeniable advantages, it is important to remember one of the core features of LLMs: they are designed to carry out users' instructions without the need -or with a minimal need-for a specific context [START_REF] Brown | Language models are few-shot learners[END_REF][START_REF] Zhou | A comprehensive survey on pretrained foundation models: A history from bert to chatgpt[END_REF][START_REF] Wei | Finetuned language models are zero-shot learners[END_REF]. This property, the forte of LLMs, needs to be taken NeurIPS '23 Workshop on Generative AI for Education (GAIED).

with a lot of caution in education. Indeed, in order to be efficient learners, it is very well documented that students need to be able to frame informative and precise inquiries [START_REF] Oudeyer | Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies[END_REF] and receive answers accordingly. In other words, in order for these models to be efficient learning companions, they should push students towards being active and in control of their learning by delivering information following specific pedagogical considerations (format, content, structure, etc). However, LLMs today are still lacking these considerations. For example, Macina et al. [START_REF] Macina | MathDial: A Dialogue Tutoring Dataset with Rich Pedagogical Properties Grounded in Math Reasoning Problems[END_REF] highlight the fact that, even when giving accurate answers in mathematics problems, these solutions are given very quickly by the LLM (ChatGPT), which doesn't give the opportunity for students to think, adapt, etc. And although using training techniques such as Reinforcement Learning from Human feedback (RLHF) can help move forward towards this goal, this method still presents several limitations mainly due to the absence of pedagogical stance in humans' feedback, the quality of their feedback (errors, wrong intentions, biases, etc), differences in the points of view between different evaluators, etc [START_REF] Casper | Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback[END_REF]. This can also be seen as a part of a more general challenge with LLMs which is their high controllability (i.e. the possibility to easily condition and sway their outputs towards a wanted behavior). This could be particularly risky in education as it means that, intentionally or not, students can make LLMs behave in opposite directions of pedagogical goals and methods, very easily. For example, research such as [START_REF] Rastogi | Supporting human-ai collaboration in auditing llms with llms[END_REF][START_REF] Hartmann | The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation[END_REF][START_REF] Kovač | Large Language Models as Superpositions of Cultural Perspectives[END_REF] highlights the idea that LLMs may be viewed as 'agents' having 'ideas', 'positions', or 'ethical values' is misleading. This makes it unclear whether the current training approaches can align LLMs' behaviors with specific pedagogical policies and goals.

If such challenges are not addressed when designing GAI-based pedagogical activities, they can be conducive to poor states of learning awareness (i.e. inability to understand and control the learning process), which is an important asset to favor motivation, activeness and to fully benefit from the learning activity. Several levels can be considered to face these challenges. First, on the training level: by grounding pedagogical methods and goals through the specific training data, alignment, etc. Second, on the application/ implementation level: by making informed decisions about how to use GAI to implement sufficient pedagogical activities (e.g. types of interactions allowed, controllability level allowed, etc). And third, on the usage level: by helping students develop the relevant skills that could help them be aware of and overcome the challenges of using GAI. These levels are summarized in Figure 1.

In this context, the current article proposes to start by over-viewing the opportunities and challenges that surround students' use of GAI during learning. In doing so, we focus on LLMs and their effects on active learning strategies and on predictors of learning such as critical thinking, metacognition, etc. Once laid out, we try to discuss some possibilities to face these challenges on the usage level, from an educational psychology point of view: how could we prepare students, especially the younger ones, to have informed and efficient use of such tools during learning? We focus on the role of metacognitive training and GAI literacy.

This work tries to suggest future research tracks that need to be explored in order to prepare for a safer and more efficient use of GAI in education. By doing so, it calls for joint effort from actors in the AI community, educational psychology research, educators and EdTech designers.

2 How can the use of GAI affect students' active learning ?

Adopting active learning strategies refers to a student's ability to stay engaged during learning by exploring the environment and 'working' for information rather than receiving it passively. This includes exhibiting curiosity, epistemic vigilance and critical thinking, metacognitive self-regulation, etc.

Immediately, we can think of several GAI properties that could have positive or negative effects on such dimensions. We discuss these properties in the next paragraphs.

Opportunities for active learning

One of the most straightforward applications of GAI -and particularly LLMs-in education, is the idea of taking advantage of their strong linguistic abilities to implement incentive interactions that can push students towards adopting more active learning strategies [START_REF] Kasneci | ChatGPT for good? On opportunities and challenges of large language models for education[END_REF]. This can be done by designing specific interactions to trigger deeper and higher-level thinking capabilities. Some previous work has indeed showed some promising results concerning such applications. For example, work in [START_REF] Abdelghani | Gpt-3-driven pedagogical agents to train children's curious questionasking skills[END_REF] Figure 1: Life-cycle of LLMs usage in educational applications and opportunities for alignment with pedagogical goals.

showed that GPT-3 can be efficient in prompting students' curiosity during reading-comprehension tasks. It helped them ask deeper-thinking and more divergent questions by proposing hints that help them identify their own uncertainties. Moreover, LLMs could constitute a strong tool to improve critical thinking skills. Indeed, such skills are shown to be supported by adding opportunities for dialogue during learning, exposure to authentic or situated problems/ examples, etc [START_REF] Philip C Abrami | Strategies for teaching students to think critically: A meta-analysis[END_REF]. All of these strategies could be much more easily implemented in the classroom by using LLMs, given their interactive and properties and access to huge amounts of scenarios and examples. Furthermore, LLMs can also facilitate the implementation of educational activities where children play active role, such as collaborators or teachers. Such roles can help them gain evaluative and monitoring metacognitive skills since they need to have a deep understanding about a given knowledge component in order to be able to teach it and/or use it to communicate with peers [START_REF] Loewenstein | The Psychology of Curiosity: A Review and Reinterpretation[END_REF].

On another hand, one strong advantage of LLMs is their ability to introduce personalized pedagogical guidance during learning [START_REF] Biswas | Role of Chat GPT in Education[END_REF][START_REF] Zhai | ChatGPT user experience: Implications for education[END_REF]. This could have several functions: informative feedback to assess the student's cognitive performance, metacognitive prompting to propose useful learning strategies to adopt, etc. Implementing such behaviors could be of a great benefit for students, both on the cognitive and metacognitive levels, as it could help them have the relevant information to 1) make accurate mental models of the pedagogical concepts at hand. And 2) to auto-regulate their learning and be more efficient in their strategies to reach their goals.

Overall, LLMs can present several important opportunities to be efficient learning companions. However, it is clear that in order to reach these opportunities, careful design work needs to be done in order to implement specific and complex interactions that could bypass the inherent lack of pedagogical stance in these models' behaviors. But what about allowing uncontrolled, spontaneous interactions ?

Challenges for active learning 2.3 GAI challenges the ability to work for information

One of the very relevant challenges of allowing uncontrolled interactions with LLMs is the risk of students becoming overly-relying on receiving information with minimal-to-none effort, and believing that solving any type of problem can be easily accessible without the need for any specific previous knowledge [START_REF] Kasneci | ChatGPT for good? On opportunities and challenges of large language models for education[END_REF]. This could lead to passiveness and undermine the intrinsic motivation to conduct investigations and active information-search. Furthermore, this apparent convenience and facility to obtain information with LLMs could lead to a negative learning control [START_REF] Robert A Bjork | Self-regulated learning: Beliefs, techniques, and illusions[END_REF][START_REF] Baidoo | Education in the era of generative artificial intelligence (AI): Understanding the potential benefits of ChatGPT in promoting teaching and learning[END_REF], meaning that students could start linking cognitive success with these powerful tools (LLMs), rather than with active learning strategies such as question-asking, exploration, etc. These are indeed crucial points to consider given their significant links with learning outcomes [START_REF] Gruber | Curiosity and learning: a neuroscientific perspective[END_REF][START_REF] Oudeyer | Intrinsic motivation, curiosity, and learning: Theory and applications in educational technologies[END_REF].

GAI challenges the development of critical thinking and self-reflective behaviors

Another LLMs' property that could lead to the loss of learning control is the lack of uncertainty signalling and the continuous confidence they exhibit even in cases of failure. This behavior means that students are not given the opportunity to judge in depth the quality of the information delivered by the model. This could result in two major problems:

1. An over-dimensioned representation of the knowledgeability of LLMs. This can lead to systematically accepting their behaviors without much analytic or critical thinking [START_REF] Kidd | How AI can distort human beliefs[END_REF]. It is particularly sensitive in education since evidence shows that individuals tend to track the knowledgeability of agents and use this signal to form their own beliefs and informationsearch behaviors [START_REF] Mark | Learning words from knowledgeable versus ignorant speakers: Links between preschoolers' theory of mind and semantic development[END_REF][START_REF] Heider | An experimental study of apparent behavior[END_REF][START_REF] Orticio | Children flexibly adapt their evidentiary standards to their informational environment[END_REF]. This means that, when mis-representing an agent's knowledgeability, individuals can form wrong and/or biased mental models of the knowledge component at hand. Once transferred to them, these misinformation, biases, etc cannot be easily corrected [START_REF] Thompson | Human biases limit cumulative innovation[END_REF]. 2. An over-estimation of students' own knowledge. The stream of affirmation during interactions with LLMs means that students will have very little chance to reflect on their knowledge states and analyze their progress, strengths and weaknesses for the task. Having an over-estimation of one's own competencies is one of the first brakes that can keep individuals from engaging in exploratory and active information-search behaviors [START_REF] Loewenstein | The Psychology of Curiosity: A Review and Reinterpretation[END_REF].

These are major and realistic challenges given that LLMs are subject to several failures in their behaviors, such as unsubstantiated content, hallucinations, incomplete information, ambiguity, cognitive and/or social biases, etc [START_REF] Chen | Can LLM-Generated Misinformation Be Detected?[END_REF][START_REF] Jones | Capturing failures of large language models via human cognitive biases[END_REF]. In this line, empirical studies such as [START_REF] Hill | Taking the help or going alone: ChatGPT and class assignments[END_REF] show indeed that even university students fail to detect failures in ChatGPT's answers and tended to use them in exams without analyzing/ correcting them. Furthermore, work in [START_REF] Chen | Can LLM-Generated Misinformation Be Detected?[END_REF] shows that LLM-generated misinformation can be harder to detect for humans, compared to human-generated misinformation. This suggests that LLMs can potentially cause more harm and expose students to adopting more wrong/ biased information compared to standard learning settings.

We believe that all of these challenges we raise, and more, are essentially coming from one higherlevel barrier: as opposed to the primary objectives of standard pedagogical activities, LLMs are not designed -and do not behave in a way-to carry out pedagogical goals and support users' learning.

Even with training methods such as RLHF that are put in place to align LLMs with human goals, it is still not clear what is the pedagogical intent (if any) behind the training of these models. Indeed, RLHF involves human evaluators who can be of various backgrounds, opinions, goals and expertise.

3 How can we promote active learning in a GAI era ?

As we mention in the introduction and illustrate in Figure 1, facing active learning-related challenges can be initiated by different actors and on different levels:

• Role of the AI community on the training level: consists of refining the training policies and content of the models, in order to acheive a pedagogical methods/ goals-grounding.

Concretely, this could be done by training the model on more educational and learning principles data, recruiting teachers as annotators during the RLHF process, etc.

• Role of educators/ EdTech designers on the application/ implementation level: consists of making informed decisions when using LLMs to design educational activities (e.g. by ensuring enough control, pedagogically-oriented interactions, etc).

• Role of researchers on the usage level: consists of preparing children to be aware of GAI's challenges and be able to use it in an informed and positive way. It relies on supporting them to develop the relevant knowledge and skills that could help them stay active and critical when using this particular technology.

In this next section, we focus on the third point, i.e. the usage level. We argue that in order to stay active and on control of their learning while using GAI, students should be able to understand the latter's behaviors rationale and have strong metacognitive skills.

Role of GAI literacy

As already discussed, one of the biggest challenges of using GAI is the misleading over-hype around its agency and knowledgeability. To have a more realistic representation of these tools, we think that it is important for students to see their real potential, strengths and weaknesses and understand the motivation and rationale behind their design. For example, explaining that these models rely heavily on statistical methods to predict their next behaviors, can help students see the difference with how biological brains work [START_REF] Brown | Language models are few-shot learners[END_REF]. At its turn, this can help have a more accurate representation of these models' power: they are likely to show several problems related to cognitive reasoning, senses of the physical world, logic, etc that are difficult for users to disentangle [START_REF] Jones | Capturing failures of large language models via human cognitive biases[END_REF].

We also think that it is important to raise students' attention to the different failures that GAI -and LLMs specifically-can generate. Indeed, problems such as outdated information, unsubstantiated content, incomplete information, lack of context, complete fabrication,... can easily occur when interacting with LLMs [START_REF] Chen | Can LLM-Generated Misinformation Be Detected?[END_REF][START_REF] Novelli | Taking AI Risks Seriously: a Proposal for the AI Act[END_REF]. Furthermore, students need also to be aware of the existence of cognitive and/or social bias in these models' behaviors [START_REF] Casper | Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback[END_REF] that can emerge from the initial training datasets, the human annotators implicated in the RLHF process, etc [START_REF] Bender | On the dangers of stochastic parrots: Can language models be too big?[END_REF]. Understanding such specific limitations can push students to be more cautious, analytic and critic of GAI's behaviors and therefore, to think deeper before adopting them. Such explanations can take the form of a series of educational videos for example. Several contents introducing GAI to the general public are currently being developed [START_REF] Torres-Leguet | ChatGPT en 5mn: une série pédagogique pour le grand public[END_REF] and can be adapted to students' levels.

On another hand, it is important to remember that the ability to conduct efficient information-search strategies in rich online environments depends on students' ability to form specific and goal-directed inquiries [START_REF] Rouet | What was I looking for? The influence of task specificity and prior knowledge on students' search strategies in hypertext[END_REF]. In GAI environments, this translates to the importance of using specific and clear prompting strategies. For example, work in [START_REF] Wei | Chain-of-thought prompting elicits reasoning in large language models[END_REF] shows that Chain-Of-Though prompting leads LLMs to a better performance in complex reasoning tasks. For this reason, we think that teaching students efficient prompting strategies is important, not only because it can enhance GAI's behaviors, but also because it can help students understand their tasks better. Indeed, an efficient prompting process requires individuals to think about how to explain the task well and introduce its context clearly to the system, decompose it into several steps if necessary, specify the nature of outcome that is wanted, etc. This means that teaching the good strategies of interacting with GAI can also lead to better metacognitive states and awareness about the learning process and goals.

Finally, we think it is also very relevant to propose such trainings to teachers and the educational designers/ developers in order to help them have clearer views about the potential of GAI and how to use it appropriately in order to implement efficient pedagogical activities that can support students' active and meaningful learning.

Role of metacognitive training

Although very important, GAI literacy may not be sufficient for nurturing an informed use of these systems in education. For example, work in [START_REF] Theophilou | Learning to Prompt in the Classroom to Understand AI Limits: A pilot study[END_REF] shows that interventions targeting prompting strategies led to enhanced states of trust in the system by students. Although this can be seen as a positive result, it can also mean that students are systematically trusting the system's behaviors as long as they're using the prompting strategies they learnt are 'correct'. This could lead to the same challenges we pointed above, i.e. over-reliance on GAI, loss of critical/ analytic instincts, etc.

To face this, we think that GAI literacy should be coupled with the training of higher-level skills such as metacognition, critical-thinking or intellectual vigilance, that are key components to active control during learning [START_REF] Livingston | Metacognition: An Overview[END_REF]. Indeed, metacognition is argued to be crucial to engage in active knowledge-acquisition behaviors: Loewnstein [START_REF] Loewenstein | The Psychology of Curiosity: A Review and Reinterpretation[END_REF] explains that curiosity-driven information-search is primarily motivated by the individual's ability to detect a knowledge gap in their own knowledge. This step mobilizes the metacognitive ability to evaluate one's own knowledge and decide about the next relevant cognitive behaviors to adopt. This means that if we want students to be more at ease with asking further information when they feel surprised or/and uncertain about GAI's behavior, supporting them to develop evaluative metacognitive skills can be an efficient strategy. Indeed, preliminary work in [START_REF] Abdelghani | Interactive environments for training children's curiosity through the practice of metacognitive skills: a pilot study[END_REF] shows that training specific metacognitive skills resulted in more epistemic curiosity behaviors in 9-to-11 yo children.

Moreover, critical thinking skills also depend on metacognition as they require the individual to be able to pursue their goals through self-directed search and interrogation of knowledge, link the information they find with their prior knowledge, hypotheses, etc [START_REF] Kou Murayama | Process account of curiosity and interest: A reward-learning perspective[END_REF]. Thus, it becomes imperative to introduce students to these skills, enabling them to discern the need for corroborative evidence, both supporting and challenging GAI-generated information, prior to acceptance. Such behaviors can help students maintain a purposeful learning trajectory when using GAI for resolving specific cognitive tasks.

It is to be noted that several studies have shown that metacognition can be trained using specific methods and prompting approaches such as inciting self-reflection, evaluating newly-acquired information, etc, [START_REF] Abdelghani | Conversational agents for fostering curiosity-driven learning in children[END_REF][START_REF] Abdelghani | Interactive environments for training children's curiosity through the practice of metacognitive skills: a pilot study[END_REF][START_REF] Alaimi | Pedagogical agents for fostering question-asking skills in children[END_REF], ... . However, to the best of our knowledge, no studies have begun exploring applying such metacognitive-prompting approaches in GAI learning environments where interactions are more complex and challenging. For this reason we think that it is important to work on adapting these approaches to the properties and challenges of GAI and design new interventions.

Conclusion

Generative AI presents huge opportunities for enhancing the teaching and learning experiences and outcomes. It has the potential to address several challenges facing the educational sector today, from teachers' shortages to taking charge of individual differences in students. However, using these systems in the classrooms still raises a lot of questions, especially concerning the lack of pedagogical intentions in these systems, which can lead to interactions that foster passiveness and loss of learning control.

With this paper, we try to lay out some of the opportunities and challenges associated with using GAI in the classroom, with a focus on how it may impact student's active learning strategies and metacognition. From there, we discuss the roles of the different stakeholders in facing these challenges, from the AI community, to educational psychologists and educators. This joint effort can indeed lead to having more pedagogically-balanced GAI and better-skilled students that can use these tools to learn in an active and responsible way.

Acknowledgments and Disclosure of Funding

This work benefited from funding from the EvidenceB company (CIFRE Grant for Rania Abdelghani) and from ANR AI Chair DeepCuriosity ANR-19-CHIA-0004. Pierre-Yves Oudeyer is also scientific consultant at EvidenceB.


Jonathan Bell
email: belljonathan50@gmail.com

MAPS AS SCORES: "TIMBRE SPACE" REPRESENTATIONS IN CORPUS-BASED CONCATENATIVE SYNTHESIS

The present study investigates ways in which the "timbre space" metaphor may be used in creative ways for instrumental composition. Numerous tools for concatenative sound synthesis share today the ability to represent in an ndimensional space large quantities of sound, thus displaying on a map data which originally unfolded in time. If the potential of such systems for creating interactive instruments is an evidence, their affordance as musical scores needs further assessment, for control over time becomes unknown territory. When porting to VR such representation of sonic data, the score becomes a 3-dimensional map (or world) in which the user typically navigates freely. Experimentation through composition, instrument design and improvisation have shown a potential of simulation of plausibly automatised acoustical instrument, using machine learning techniques to model virtual instruments out of relatively small quantities of data (e.g. 20 minutes of audio to model a clarinet). The method offers promising avenues for the exploration of instrumental fragments clustered by timbre, register, dynamic, instrumental techniques. Whether or not such maps identify as musical scores, they contribute to addressing a problem formulated by Lev Manovitch: "how to merge database and narrative into a new form".

INTRODUCTION 1.1 Are scores maps?

The question raised in this article is in no small part inspired by a study by Daniel Miller, Are scores maps? A cartographic response to Goodman [START_REF] Miller | Are scores maps? a cartographic response to goodman[END_REF], in which a dialectical tension between two concepts (score and map) leads to interesting questions on the role and function of musical notation. Miller proposes that, in spite of surface-level conventions, the underlying structure of scores closely relates to maps: "notational components of scores are better understood as contingent surface-level features leveraged by an underlying map-like representational structure. On this account, scores are seen to be highly conventionalized maps, and the notational symbols of scores consti-Copyright: c 2022 Author Name. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. tute just one of multiple modes of representation and depiction harnessed by this framework". Scores may therefore be conceived as a mere subset of maps. The rapprochement between both notions is best illustrated with a historical example: famous works of the post-war avantgarde (by M. Feldman, J. Cage, E. Brown, P. Boulez, K. Stockhausen and A. Boucourechliev to cite a few) took the form of graphical scores which the performer could freely navigate. Boucourechliev named many of his compositions "Archipels" (archipelago) which evokes the same metaphor. Those scores typically presented common notation fragments spread out all over the page, so as to emancipate the work from the linearity imposed by traditional notation, an idea thoroughly discussed in U. Eco's Open Work [START_REF] Eco | The Open Work[END_REF]. In Boulez's second sonata for piano for instance, this co-existence of snippets of conventional notation with a map-like layout on the page illustrates how maps allow for compelling hybrid cases, in which not every symbol on the page functions as a map. Similarly, in more conventional/linear scores, Miller underlines that only some features are isomorphic (or maps-like): "Scores are maps that are isomorphic with the spatial and temporal structures of the musical works they represent, while other graphical features may be purely contingent or incidental. This highlights an interesting property of maps: they need only be isomorphic with regard to a subset of the properties of the space they represent". Maps, in a manner reminiscent of scores, aim to guide a user or prompt a performer for action, and this goal needs not obey strict, systematic, one to one mapping relationships. For Miller indeed, this mixture of isomorphism and contingency responds to Goodman's famous attack against John Cage, formulated as follows: "Without stipulation of minimal significant units of angle and distance, p. 53 from John Cage's Concert for Piano and Orchestra from 1960 is not syntactically differentiated". The philosopher's observation led him to vitriolic criticism of John Cage's approach to graphic notation: "Under the proposed system there are no disjoint and differentiated characters or compliance classes, no notation, no language, no score" [START_REF] Goodman | Languages of Art: An Approach to a Theory of Symbols[END_REF], a point of view which, in return, has proven widely unpopular.

Reflexions on maps and paper scores may seem here overly theoretical. The digital age, however, urges composers to think about scores in new ways, in which hybridization between interactive systems and notation leaves more room to our notion of map as score. In "The digital score, musicianship, creativity and innovation" [START_REF] Vear | The Digital Score: Musicianship, Creativity and Innovation[END_REF], Craig Vear pro-poses that: "The core purpose of any digital score is to communicate ideas between musicians using digital technology.", thus placing technology at the center contemporary notation, and expanding at the same time the range of systems hitherto coined as "scores". Vear also proposes that "Some digital scores might feel like computer games where the performer makes decisions about what happens next." The image of the computer game, just as the one of a map which the performer can browse, evokes freedom in the first place, but also an important shift of focus when considered through the perspective of a traditional composer. His craft then becomes closer to the one of an instrument designer, which Vear still considers belonging to the realm of digital scores. In a chapter entitled "The nature of digital scores: expanding the core signatures", he states: "the score might be embedded within the design of an instrument, instrument might be a system-score of electronics controlled by generative software". Under such circumstances, the frontier between score and digital instrument design blurs, and diverse forms of interactive system querying a database will be considered a score.

In some early accounts of artistic uses of CataRT [START_REF] Schwarz | Real-Time Corpus-Based Concatenative Synthesis with CataRT[END_REF], Diemo Schwartz described his 2d representations of a sound corpus as "navigable score" or "score instruments": "The [piece's] subject was "navigable scores" or score-instruments, in which different kinds of users would play the sound or music, cruising through the score." [START_REF] Schwarz | Principles and Applications of Interactive Corpus-Based Concatenative Synthesis[END_REF] Schwartz's discovery then presents this paradox that it proposes a score that does not represent time.

Maps do not represent time

Musical scores are used to convey information about musical material function of time.

Maps, on the other hand, are static representations of an area. They do not represent time or changes over time, and are typically used to convey information about geographical features and spatial relationships, and are also a powerful tool for visualisation in data science, as will be developed in Chapter 5.

"Many new media objects do not tell stories; they don't have a beginning or end" [START_REF] Manovich | Database as symbolic form[END_REF]. This quote by Manovitch helps us here introducing a challenging idea for a traditional composer: ignoring the temporal dimension of a score may lead to innovative approaches to composition. We will now expose how such data plot, or more globaly databases can also be considered as an art form. Finally, the absence of time inherent to these concept encourages to use in Section 1.2.2 aesthetic arguments to understand what might stay creatively interesting within a loose control over time in musical composition.

Databases as an art form

Databases and data mining in music can be thought of as an art form in the sense that they involve using creativity and analytical skills to extract insights and knowledge from large sound datasets. This requires the ability to inform musical creativity by identifying patterns, trends, and relationships in data that may not be otherwise apparent.

Novels and film, are, as music, dance, or theatre, temporal art forms. Manovitch's insight on database art form suggests that new media isn't subordinate to time, or narrative, in the same manner; this consideration may help a composer alter some preconception on his approach to form, articulation, or narrative: "After the novel, and subsequently cinema, privileged narrative as the key form of cultural expression of the modern age, the computer age introduces its correlate -database. Many new media objects do not tell stories; they don't have a beginning or end; in fact, they don't have any development, thematically, formally or otherwise which would organise their elements into a sequence." [START_REF] Manovich | Database as symbolic form[END_REF] This absence of narrative in new media art, which Manovitch relates to post-modernity, finds an interesting echo in the thought of Morton Feldman, who saw in the European Avant-Garde an excessive desire for control over time.

Morton Feldman and the European clock makers

In his collected writings [START_REF] Feldman | Give My Regards to Eighth Street: Collected Writings of Morton Feldman, ser. Exact Change[END_REF], Morton Feldman recalls a discussion he once had with Karlheinz Stockhausen: "He was convinced that he was demonstrating reality to me. That the beat, and the possible placement of sounds in relation to it, was the only thing the composer could realistically hold on to. [. . . ] Frankly, this approach to time bores me. I am not a clockmaker. I am interested in getting Time in its unstructured existence. That is, I am interested in how this wild beast lives in the jungle -not in the zoo". Feldman, often with humour, insisted on that idea : "Let the sounds alone, Karlheinz, don't push them -not even a little bit?". This almost passive approach to composition echoes the zen-inspireed thought of John Cage, and also importantly took his inspiration from painters: "A painter will perhaps agree that a color insists on bieng a certain size, regardless of his wishes [...] He can simply allow it to 'be'. In recent years we realize that sound has a predilection for suggesting its own proportions [...] Any desire for differentiation must be abandoned"

Feldman's music explored this idea of surface in a number of ways, such as through the use of long, slow and static musical lines, the repetition of simple motifs, absence of contrast (which he called differenciation), and the use of unconventional temporal dimensions (some works such as the second string quartet last over four hours). This unconventional approach to form relates more broadly to an opposition between European and American avant-garde in the 50-60s.

The tools described in Chapter 5 inclined the author to think of musical form as long static time canvases, as would M. Feldman call them: " "My obsession with surface is the subject of my music. In that sense, my compositions are really not "compositions" at all. One might call them time canvases in which I more or less prime with an overall hue of the music". Listening to almost any of his works, one realises that the entire piece most often shares the same atmosphere from its beginning until its end. A piece with a recognisable instrumental combination such as why patterns for flute, piano and celesta or Clarinet and Percussion have a strong acoustical footprint and illustrate how machine machine listening could be used on such materials.

CORPUS-BASED CONCATENATIVE SOUND SYNTHESIS (CBCS) TODAY

Corpus-Based Concatenative Sound Synthesis (CBCS) is a technique used in computer music that involves constructing a sound or music piece by concatenating (joining together) smaller units of sound, such as phonemes in speech synthesis or musical phrases in music synthesis. It can be used to model an improvising instrumental musician by creating a database of recorded musical phrases or segments that can be combined and rearranged in real-time to create a musical performance that sounds like it is being improvised.

Today nearly 20 years old if one refers to the first CataRT publications [START_REF] Schwarz | Real-Time Corpus-Based Concatenative Synthesis with CataRT[END_REF], CBCS today enjoys an increasing popularity. Various apps today are based on similar principals (AudioStellar, Audioguide, LjudMAP or XO). The democratisation of audio analysis and machine learning tools such as the FluCoMa package (for Max, SuperCollider and Pure Data) encourages computer music practitioners to engage in this field at the crux between music creation and data science/machine learning.

Timbre Space

In spite of promising advances in the domain of deep learning applied to sound synthesis [START_REF] Briot | Deep Learning Techniques for Music Generation -A Survey[END_REF] [10], CBCS tools may earn their popularity from a metaphor which leads back to the early days of computer music: the notion of timbre space, developed by Wessel [START_REF] Wessel | Timbre space as a musical control structure[END_REF] and Grey [START_REF] Fitz | Multidimensional perceptual scaling of musical timbre by hearingimpaired listeners[END_REF], according to which the multi-dimentional qualities of timbre may be better understood using spatial metaphors (e.g. the timbre of the English horn being closer to basson than is it of trumpet).

The heritage of the Timbre Space metaphor can also be found in various iterations of the Orchidea [START_REF] Cella | Orchidea: a comprehensive framework for target-based computer-assisted dynamic orchestration[END_REF] project, and continues to inspire generations of composers and music technologists. 1 Pioneers in the perception of timbre studies such as Grey [START_REF] Fitz | Multidimensional perceptual scaling of musical timbre by hearingimpaired listeners[END_REF], J.C. Risset, D. Wessel, [START_REF] Risset | Exploration of timbre by analysis and synthesis[END_REF] or Stephen McAdams [START_REF] Mcadams | Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes[END_REF] [18] most often define timbre by underline what it is not. Risset and Wessel, for instance, define it as follow: It is the perceptual attribute that enables us to distinguish among orchestral instruments that are playing the same pitch and are equally loud.

The co-variance such parameters (pitch, loudness and timbre), however, leads Schwarz to distinguish timbre space and CBCS notions: 'Note that this concept is similar but not equivalent to that of the timbre space put forward by Wessel and Grey [START_REF] Manovich | Database as symbolic form[END_REF][START_REF] Bevilacqua | A gesture follower for performing arts[END_REF], since timbre is defined as those characteristics that serve to distinguish one sound from another, that remain after removing differences in loudness 1 Daniele Ghisi, co-author of the bach [START_REF] Agostini | A Max Library for Musical Notation and Computer-Aided Composition[END_REF] package for Max, occupies a role role here as he's worked both on the Orchidea and FluCoMa projects. Some objects of his later dada library [START_REF] Ghisi | dada: Non-standard user interfaces for computer-aided composition in max[END_REF] also show an influence of CataRT (the dada.catart object was later renamed dada.cartesian). The dada.base based, finally, might have been a source of inspiration for the manipulation of databases in Max. An extract of one of his presentation is available here: https://youtu.be/LD0ivjyuqMA?t=3032 Figure 1. Multidimensional perceptual scaling of musical timbres (John M. Grey [START_REF] Fitz | Multidimensional perceptual scaling of musical timbre by hearingimpaired listeners[END_REF]). Sounds available at: https://muwiserver.synology.me/timbrespaces/grey.htm. and pitch. Our sound space explicitly includes those differences that are very important to musical expression." [START_REF] Schwarz | The Sound Space as Musical Instrument: Playing Corpus-Based Concatenative Synthesis[END_REF] The workflow described in Chapter 5 gave in practice strong evidence of inter-dependance between register, timbre and dynamics, particularly when the analysis run over a single instrument sound file (e.g. 30 minutes of solo flute), and chopped in short samples. The system will then precisely be able to find similarity between instrumental passages played in the same register, same dynamic, and same playing technique (e.g. a flute playing fast trills mezzo forte, in mid-low register, with air).

Corpus-Based Concatenative Synthesis -State of the art

A wide array of technologies today can be called corpusbased concatenative synthesis, in the sense that they allow, through segmentation and analysis, to explore large quantities of sound. Some of them are presented as "ready-made" solutions, such as the recent Audiostellar [START_REF] Garber | Audiostellar, an open source corpus-based musical instrument for latent sound structure discovery and sonic experimentation[END_REF], or SCMIR 2 for SuperCollider. Hackbarth's AudioGuide [START_REF] Hackbarth | Composing Morphology: Concatenative Synthesis as an Intuitive Medium for Prescribing Sound in Time[END_REF] offers a slightly different focus because it uses the morphology/timeline of a soundfile to produce a concatenated output. Within the Max world finally, two environments appear as highly customizable: IRCAM's MuBu [START_REF] Schnell | Mubu and friends -assembling tools for content based real-time interactive audio processing in max/msp[END_REF] and the more recent EU funded FluCoMa [START_REF] Tremblay | Enabling Programmatic Data Mining as Musicking: The Fluid Corpus Manipulation Toolkit[END_REF] project. CataRT is now fully integrated in MuBu, whose purpose encompasses multimodal audio analysis as well as machine for movement and gesture recognition [START_REF] Bevilacqua | A gesture follower for performing arts[END_REF]. This makes MuBu extremely general purpose, but also difficult to grasp. The data processing tools in MuBu are mostly exposed in the pipo plugin framework [START_REF] Schnell | Pipo, a plugin interface for afferent data stream processing operators[END_REF], which can compute for in-stance mfcc analysis on a given audio buffer 3 by embedding the pipo.mfcc plugin inside the mubu.process object.

FluComa also aims to be general purpose, but seems particularly suited to perform two popular specific tasks. With only limited knowledge of the framework nor of theory laying behind the algorithms it uses (such as those dimentionality reduction, mfcc analysis, or neural network training), the framework allows: 1/ to segment, analyse and represent/playback a sound corpus 2/ to train a neural network to control a synthesizer, in a manner reminiscent of Fiebrink's Wekinator [START_REF] Fiebrink | The wekinator: A system for real-time, interactive machine learning in music[END_REF].

Only the tools for segmentation, analysis representation and playback (described in detail in Chapter 5) were used here, for they precisely fit the needs of corpus-based synthesis.

FIRST ATTEMPTS

Some of the early versions of the FluCoMa package already provided efficient onset detection algorithms (onsetslice 4 ), which encouraged the author to further dig into their environment.

At that time, all the compositional material generated with the package used the fluid.bufstat 5 object to run statical pitch analysis (and pitch confidence analysis) on each slice of a given pre-existing sound file. As can be heard in this accompanying example 6 , slices were classified by pitch (the lower the index, the lower the register), although some inaccuracy could occur due to the simplicity of the analysis (section 5 will describe more elaborate strategies).

Onset detection combined with statistical pitch analysis were first used on a large scale in a piece (Chef 2.0 7 ) in which all the instrumental parts had been generated with comparable rudimentory Music Information Retrieval techniques. Some experiments of various forms of VR control over such analysis tools already seemed promissing at that time 8 . The PatchXR software will be discussed in section 5.3.

MOTIVATIONS

One of the goals of the present study is to take advantage of the numerous capacities of visualisation, interaction and motion gesture data available within a VR environment. 3 MFCC stands for Mel-Frequency Cepstral Coefficients. It is a type of feature extraction method that is commonly used in speech and speaker recognition systems. MFCCs are used to represent the spectral characteristics of a sound in a compact form that is easier to analyze and process than the raw waveform. They are calculated by applying a series of transformations to the power spectrum of a sound signal, including a Mel-scale warping of the frequency axis, taking the logarithm of the power spectrum, and applying a discrete cosine transform (DCT) to the resulting coefficients. The resulting coefficients, which are called MFCCs, capture the spectral characteristics of the sound and are commonly used as features for training machine learning models for tasks such as speech recognition and speaker identification. 4 https://learn.flucoma.org/reference/onsetslice/ 5 https://learn.flucoma.org/reference/bufstats/ 6 https://youtu.be/UNj7 TI8SVs 7 Simulation: https://www.youtube.com/watch?v=MkMKVm3G3W8 Result: https://youtu.be/Sc4Ye rnSO8?t=9893. Although not relevant to the discussion here, the arm movement of the conductor was mapped to the speed of the cursor on the performers' screens with the help of INScore [START_REF] Fober | INScore -An Environment for the Design of Live Music Scores[END_REF] and Gesture Follower [START_REF] Bevilacqua | Continuous realtime gesture following and recognition[END_REF] in MuBu for Max. 8 https://youtu.be/DC BL HGPLA

Porting to VR analysis made in Max/FluCoMa has its challenges but I am currently exploring various possible ways of interacting with a corpus-based analysis in VR. After a few trials in which the x y z coordinates of a world directly represented audio descriptors such as loudness pitch and centroid 9 , I more systematically used mfcc analysis and dimensionality reduction, as described in the following section. The use of machine learning (dimensionality reduction) in the latter case renders a world in which the absolute coordinates of each point has no more link to the descriptor space (the high sounds cannot be mapped to the y axis for instance), but offers compelling results for clustering information relating to the different playing styles of the instrument that is being analysed: as an example, in this extract 10 based on flute sounds, the opening shows a clear opposition between two types of gestures: 1/ staccato notes and 2/ legato scale-like type of material. This contrast in sound is made explicit by a movement of the avatar which jumps from a cluster of buttons to another.

WORKFLOW

After numerous attempts at listening to and playing with this system11 , I am now investigating how to diversify in VR metaphors for exciting the corpus based synthesis engine, as well as the different ways in which the synthesis may be rendered on an orchestra of RaspberryPi-equipped loudspeakers (see Chapter 6).

Corpus Selection

My experiments have focussed on musical instrument corpora almost exclusively. The tools presented here can efficiently generate plausible virtuosic instrumental music (as was sometimes the case in the piece Chef 2.0 discussed earlier), but recent uses found more satisfying results in slower, quieter, "Feldman-like" types of textures. Various limitations on the playback side (either in VR, or on a Pure Data sampler for RaspberryPi described in Chapter 6) have imposed restrictions in the first stages on the amount of data it could handle (less than 5minutes in AIFF in VR) or the number of slice the sample could be chunked into (256 because of limitation of lists in Max). Both limitations were later overcome (use of the ogg format in VR, increase of internal buffer size in fluid.buf2list), thus allowing for far more convincing models.

Analysis in FluCoMa

Using concatenative synthesis to model an improvising instrumental musician typically involves several steps:

1. Segmentation of a large soundfile: This involves dividing a large audio recording of the musician's performance into smaller units or segments.

2. Analysis: These segments are then organised in a database according to various descriptor data (mfcc in our case).

3. Scaling/pre-processing: scaling is applied for better visualisation.

Dimension reduction:

Based on mfcc descriptors, the dimensionality of the data is reduced in order to make it more manageable and easier to work with. This can be done using techniques such as principal component analysis (PCA) singular value decomposition (SVD), or Uniform Manifold Approximation and Projection (UMAP, preferred in our case).

5. Near neighbours sequencing: Once the segments have been organised and analysed, the software selects and combines them in real-time based on certain input parameters or rules to create a simulated musical performance that sounds like it is being improvised by the musician. We use here a near neighbours algorithm, which selects segments that are similar in some way (e.g., in terms of pitch, loudness, or timbre -thanks to similarities revealed by umap on mfccs in our case) to the current segment being played.

We will now describe these steps in further detail:

Slicing

We saw in Chapter 3 how slicing musically trigger possibilities. In MuBu onset detection is done with pipo.onseg or pipo.gate. FluCoMa expose five different onset detection algorithms: Onsetslice only was extensively tested. The only tweaked parameters were a straigh-forward "threshold" as well as a "minslicelength" argument, determining the shortest slice allowed (or minimum duration of a slice) in hopSize. This introduce a common limitation in CBCS: the system strongly biaises the user to choose short samples for better analysis results, and more interactivity, when controlling the database with a gesture follower. Aaron Einbond remarks in the use of CataRT how short samples most suited his intention: "Short samples containing rapid, dry attacks, such as close-miked key-clicks, were especially suitable for a convincing impression of motion of the single WFS source. The effect is that of a virtual instrument moving through the concert hall in tandem with changes in its timbral content, realizing Wessel's initial proposal." [START_REF] Einbond | Spatializing timbre with corpus-based concatenative synthesis[END_REF] A related limitation of concatenative synthesis lies in the fact that short samples will demonstrate the efficiency of the algorithm 12 , but at the same time moves away from the "plausible simulation" sought in the present study. A balance therefore must be found between the freedom imposed by large samples, and the refined control one can obtain with short samples. A direct concatenation of slices clicks in most cases on the edit point, which can be avoided through the use of ramps. The second most noticeable glitch on concatenation concerns the interruption of low register resonances, which even a large reverb fails making sound plausible. Having a low threshold and large "minslicelenght" results in equidistant slices, all of identical durations, as would do the pipo.onseg object in MuBu.

Because we listen to sound in time, this parameter responsible for the duration of samples is of prior importance.

mfcc on each slice -across one whole slice/segment

Multidimensional MFCC analysis: MFCC (Mel-Frequency Cepstral Coefficient) analysis is a technique used to extract features from audio signals that are relevant for speech and music recognition. It involves calculating a set of coefficients that represent the spectral envelope of the audio signal, and can be used to capture the spectral characteristics of the musician's playing style.

statical analysis over each slice

BufStats is used to calculate statistical measures on data stored in a buffer channel. A buffer here is a type of data structure that holds time-series information, audio descriptor data in this case. BufStats calculates seven statistics on the data in the buffer channel: mean, standard deviation, skewness, kurtosis, low, middle, and high values. These statistics provide information about the central tendency of the data and how it is distributed around that tendency. In addition to calculating statistics on the original buffer channel, BufStats can also calculate statistics on up to two derivatives of the original data, apply weights to the data using a weights buffer, and identify and remove outlier frames. These statistical measures can be useful for comparing different time-series data, even if the original data is different lengths, and may provide better distinction between data points when used in training or analysis. The output of BufStats is a buffer with the same number of channels as the original data, with each channel containing the statistics for its corresponding data in the original buffer.

Normalization

The FluCoMa package proposes several scaling/preprocessing tools, amongst which normalization and standardization were used. Standardization and normalization are techniques used to transform variables so that they can be compared or combined in statistical analyses. Both techniques are used to make data more comparable, but they work in slightly different ways.

Standardization involves scaling a variable so that it has a mean of 0 and a standard deviation of 1. This is done by subtracting the mean of the variable from each data point and then dividing by the standard deviation. Standardization is often used when the variables being compared are on different scales or have different units of measurement. It allows for comparison of variables that would otherwise be difficult to compare directly.

Normalization involves scaling a variable so that it has a minimum value of 0 and a maximum value of 1. This is done by subtracting the minimum value of the variable from each data point and then dividing by the range (i.e., the difference between the maximum and minimum values). Normalization is often used when the variables being compared have a skewed distribution, or when the variables are not normally distributed. It allows for comparison of variables that would otherwise be difficult to compare directly due to the skewness of their distribution.

Standardization scales a variable to have a mean of 0 and a standard deviation of 1, while normalization scales a variable to have a minimum value of 0 and a maximum value of 1. Normalization ccaling was found easier to use both in 2-D (in FluCoMa, the fluid.plotter object), as well as in the VR 3D world in which the origin corresponds to a corner of the world. The fluid.nomalize object features an "@max" attribute (1 by default), which then maps directly to the dimensions of the VR world.

Dimensionality Reduction

Dimensionality reduction is a technique used in machine learning to reduce the number of features (dimensions) in a dataset. The goal of dimensionality reduction is to simplify the data without losing too much information. Various dimensionality reduction algorithms are presented in an early FluCoMa study [START_REF] Roma | Adaptive mapping of sound collections for data-driven musical interfaces[END_REF], with interestingly no mention of UMAP, later favoured. SOM in one of the most popular algorithms for dimensionality reduction. It is implemented in the ml.star [START_REF] Smith | Unsupervised play: Machine learning toolkit for max[END_REF] library for Max, a simple hands-on library for machine learning, just one amongst the vast ammount of frameworks and machine learning algorithm famous across the NIME community [32] [33] [34] [START_REF] Caramiaux | Machine learning of musical gestures[END_REF]. SOM (Self-Organizing Map) and UMAP (Uniform Manifold Approximation and Projection) are both techniques for dimensionality reduction. SOM is a type of neural network that is trained using unsupervised learning. It consists of a grid of neurons, each of which is associated with a set of weights. The SOM is trained by presenting it with input data and adjusting the weights of the neurons so that similar input patterns are mapped to nearby neurons on the grid. The resulting map is a low-dimensional representation of the input data that preserves the topological structure of the original data. UMAP, on the other hand, is a non-linear dimensionality reduction technique that is based on the principles of topological data analysis. It uses a combination of techniques such as k-nearest neighbours, weighted graph construction, and low-dimensional embedding to produce a low-dimensional representation of the input data. Unlike SOM, which is limited to a fixed grid structure, UMAP can produce a continuous, flexible representation of the data. Both SOM and UMAP can be useful for visualising high-dimensional data and for discovering patterns and relationships in the data. However, UMAP has some advantages over SOM, including the ability to handle large datasets more efficiently and the ability to produce more interpretable results. UMAP (Uniform Manifold Approximation and Projection) can be used to visualize high-dimensional data in a lower-dimensional space. When applied to sound data analysed with MFCC (Mel-Frequency Cepstral Coefficients), UMAP reduces the dimensionality of the data and creates a visual representation of the sound in a 2-or 3-dimensional space. MFCCs, again, are a feature extraction technique commonly used in speech and audio processing. They involves decomposing a sound signal into a set of frequency bands and representing the power spectrum of each band with a set of coefficients. The resulting MFCC coefficients capture important spectral characteristics of the sound signal (albeit hardly interpretable by the novice user), such as the frequency and magnitude of the spectral peaks. By applying UMAP to the MFCC coefficients of a sound signal, it is possible to create a visual representation of the sound that preserves the relationships between the different MFCC coefficients (see Fig. 2). This can be useful for tasks such as exploring the structure of a sound dataset, identifying patterns or trends in the data, and comparing different sounds. UMAP is therefore used for its clustering abilities in the first place, helping for classification purposes. It helps identifying patterns or trends that may not be evident from the raw data. Most importantly, the non-linear dimensions proposed by UMAP (whether in 2d in Max or in 3 dimensions in PatchXR, and when compared to linear analyses in which, for instance, x, y and z correspond to pitch, loudness and centroid) gave far more "intelligent" clustering than more conventional parameter-consistent types of representations.

Neighbourhood queries

The neighbourhood retrieval function in a slightly different way each time, but is based in FluCoMa on K-d trees and and the knn algorythm. In MuBu , the mubu.knn object, as well as the ml.kdtree object ml.star, give very comparable result than those achievable with fluid.kdtree. K-d trees (short for "k-dimensional trees") and k-nearest neighbours (k-NN) are two algorithms that are related to each other, but serve different purposes. A k-d tree is a data structure that is used to efficiently store and query a set of points in a k-dimensional space. It works by partitioning the points into a binary tree, with each node in the tree representing a hyperplane that splits the space into two halves. The points are recursively partitioned into the left and right subtrees based on which side of the hyperplane they fall on. By organising the points in this way, it is possible to quickly find the nearest neighbours of a given point by searching only a subset of the tree rather than the entire set of points. On the other hand, the k-NN algorithm is a machine learning algorithm that is used for classification or regression. Given a set of labeled points and a new, unlabelled point, the k-NN algorithm determines the k points in the set that are nearest to the new point, and then uses the labels of those points to predict the label of the new point. The value of k is a hyper-parameter that is chosen by the user, and it determines the number of neighbours that are considered when making the prediction.

In summary, a k-d tree is a data structure that is used to store and efficiently query a set of points in a k-dimensional space, while the k-NN algorithm is a machine learning algorithm that is used for classification or regression. Both algorithms are often used in applications such as pattern recognition, image classification, and data mining.

While CataRT or Audiostellar are typically used for generation of electronic textures/sound design, I have most often used FluCoMa to generate monophonic instruments (one performer plays one instrument at a time), in which the avatar reproduces what knn would do with an automated instrument: he will privilege in his choice the sample he can reach at hand, rather than jump large distance between 2 items (see Fig. 3) .

PatchXR

PatchXR [START_REF] Bauer | First steps towards augmented reality interactive electronic music production[END_REF] is a playful digital audio workstation for making music in VR. It's core metaphor corresponds to what the FluCoMa team calls CCE (creative coding environments) insofar as it functions in many ways like Max or Pure Data.

One reason for using VR to explore a 3D dataset is that it allows users to interact with the data in a more natural and immersive way, using it as a tool for data visualisation and analysis. Users can move around and explore the data from different angles, which can help them to better understand the relationships between different data points and identify patterns. Users get a more intuitive sense of the data and better understand how it is structured and how the different data points relate to one another.

The structure of a .patch file (a patchXR world) follows the syntax of a .maxpat (for Max) or .pd file (for pure data) in the sense that it first declares the objects used, and then the connexions between them. This structure made it relatively trivial to generate a javascript routine taking as input a dictionary (json file) with each segment's 3d coordinates, and as output a new .patch file (a world accessible in VR, see general workflow on Fig. 4).

Interaction and OSC communication

PatchXR exposes a wide range of blocks (a block corresponds to an object in Max or Pure Data) making it simple to access gesture data such as:

• The position/distance between hands/controllers and a reference.

• The rotation angles (x y z) of both hands' controllers

• 2-d touchscreen like controllers, where the user moves the xy position of a selector across a plane by manually grabbing it.

• 2-d lazer-like controllers, where the user moves the xy position of a selector remotely, as if using a lazer pointer towards a remote screen

• 2-d pads, which allow to access the velocity at which the pad is hit

• 3-d theremine like controllers, where the user moves the xyz position of a selector across a plane by manually grabbing it.

• 1-d sliders, knobs, buttons... One of the current challenges consists in diversifying the ways in which the corpus is queried. One to one mapping of UMAP results such as those described in Chapter 5.3 favour simulations for soloists, or duo in a multiplayer mode, in which the button interface buttons are facing each other, in order to prompt the players to face each other (see https://youtu.be/LP1g79BdIpY). A simulation for more instruments, particularly when playing alone, encourages to use higher level type of control over automata, most importantly the simple ability to automatically concatenate: play the next sample as soon as the previous one has stopped.

FUTURE WORKS: THE RASPBERRY PI ORCHESTRA

In the frame of an artist residency at UCA (Université côte d'azur), the investigation questions how the tools presented above (those concern with the domain of Music Information Retrieval -MIR) may serve the control an immersive platform made of an orchestra of 64 Pré modules. [START_REF]Pr É : connected polyphonic immersion[END_REF] At the time of writing this report, most satisfying results were achieved by sending messages to each RaspberriPi independently, according to its specific (static) IP address, with a simple syntax of a 2-integer list corresponding to: 1/which buffer to lookup 2/ which slice in this buffer to play. Pursuing on elaborations of timbre-space illustrations, the Pré modules, with the different acoustics its mobility allows, will encourage contrasted density of events according to the acoustics of the space in which the listening experience is happening.

CONCLUSIONS

After a discussion on the valuation of scores as maps, we've proposed a workflow for corpus-based concatenative synthesis CBCS, arguing that machine learning tools for data visualisation offer revealing and exploitable information about the timbral quality of the material that is being analysed. From a composer's perspective, the disappearance of the x time axis prompts to envisage composition not in a narrative sense (understood according reflections on new media and the notion of data-based art developed by [START_REF] Manovich | Database as symbolic form[END_REF]), but rather, as "time canvasses", as understood by Morton Feldman.

The discussed tools for "machine listening" (FluCoMa, MuBu) help building intelligent instruments with relatively small amounts of data, the duration of samples appear crucial in CBCS. A balance must be found between 1/ short duration sample analysis which are easier to process and categorise and 2/ long samples which sound more natural in the context instrument-based simulations.

1 .

 1 fluid.ampslice: Amplitude-based detrending slicer 2. fluid.ampgate: Gate detection on a signal 3. fluid.onsetslice: Spectral difference-based audio buffer slicer 4. fluid.noveltyslice: Based on self-similarity matrix (SSM) 5. fluid.transcientslice: Implements a de-clicking algorithm

Figure 2 .

 2 Figure 2. Dimensionality reduction of MFCCs help revealing spectral similarities. UMAP outputs coordinates in 2d or 3d.

Figure 3 .

 3 Figure 3. A VR interface in which each button in the world corresponds to a slice of the sound file. Machine learning helps bringing closer sounds that share common spectral characteristics.

Figure 4 .

 4 Figure 4. General Workflow: from an input audio file to its .patch 3d representation in PatchXR.

A demo is available at: https://youtu.be/jxo4StjV0Cg

https://youtu.be/1LHcbYh2KCI?t=19

https://youtu.be/777fqIIJCY4

For cello: https://youtu.be/L-MiKmsIzjM For various instruments: https://www.youtube.com/playlist?list=PLc WX6wY4JtnNqu4Lwe2YzEUq9S1IMvUk For flute: https://www.youtube.com/playlist?list=PLc WX6wY4JtlbjLuLHDZhlx78sTDm

e.g. https://youtu.be/LD0ivjyuqMA?t=3032

Acknowledgments

I am grateful for the support of UCA/CTEL, whose artist residency research program has allowed to hold these experiments, and for the support of PRISM-CNRS.