Dominique Fober
email: fober@grame.frorlarey@grame.frletz@grame.fr

Yann Orlarey

Stéphane Letz

Representation of Musical Computer Processes

The paper presents a study about the representation of musical computer processes within a music score. The idea is to provide performers with information that could be useful especially in the context of interactive music. The paper starts with a characterization of a musical computer process in order to define the values to be represented. Next it proposes an approach to time representation suitable to asynchronous processes representation.

INTRODUCTION

Throughout the last decades, the development of electronic and digital technologies, including sound production, gesture captation, has led to a musical revolution and to the emergence of new genres firmly established in the culture and musical imagination. From these evolutions, new needs have emerged, notably in terms of music notation and representation [START_REF] Freeman | Bringing instrumental musicians into interactive music systems through notation[END_REF]. Raised by artistic forms like interactive music or live coding, questions related to dynamic notation or musical process representation are taking a significant importance. With interactive music, the cohabitation between static scores and interaction processes is to be organized. With live coding, the border between programmation and score tends to be blurry [START_REF] Magnusson | Algorithms as scores: Coding live music[END_REF]. Today, the object of the musical notation needs to be redefined to take account of the new dimensions of the current artistic practices.

This paper addresses the dynamic representation of processes and their states in the single graphical space of the musical score.

The musical context gives a specific dimension to the problematic of a process representation, because we are interrested in its current state, but also in a set of time ordered states, including futur possible states. Usability constraints require these state to be readable in real-time, i.e. in the time of the score performance, and to be a guide for the interaction choices of the performer.

The objective of the study is to integrate the representation of processes to INScore [START_REF] Fober | Inscore -an environment for the design of live music scores[END_REF][START_REF] Fober | Programming interactive music scores with inscore[END_REF], an environment for the design of augmented interactive music score. We will first present existing approaches in the domain of process representation, with a viewpoint centered on musical applications. We will next define the elements to characterize an interaction process. We will finally propose a model of process representation, not in the graphic space -that remains the score domain and the composer responsibility -but in terms of communication between the tools involved in a piece realization.

EXISTING APPROACHES

Generally provided with operating systems, there are tools to visualize the system state in terms of resources usage: CPU, memory, network bandwith, etc. (figure 1). In the musical domain, tools for music computation are proposing similar representations: the system state may include both the application resources usage and the system parameters, generaly reflecting audio processes. In the musical domain, PureData [START_REF] Puckette | Pure data: another integrated computer music environment[END_REF] and Max/MSP [START_REF]Combining Event and Signal Processing in the MAX Graphical Programming Environment[END_REF] are among the applications frequently used for interactive music design. PureData provides rough tools to represent the system state (figure 2b). Max/MSP includes an audio resources monitor (figure 2a) that gives also the value of the audio setup, like sampling rate, I/O vector size, etc.

In the context of interactive music, the current position in the score is part of the system state but the notion of score is missing win tools like Max/MSP or PureData. Extensions like Bach [START_REF] Agostini | Bach: An environment for computer-aided composition in max[END_REF] or MAXScore [START_REF] Didkovsky | Maxscore: Music notation in max/msp[END_REF] allow to introduce the music notation in Max/MSP, but without taking account of the musical processes representation.

With environments like Open Music [START_REF] Assayag | Computer-assisted composition at ircam: From patchwork to openmusic[END_REF] or i-score [START_REF] Allombert | Iscore: a system for writing interaction[END_REF], a process is viewed as an opaque box. Its temporal dimension is taken into account and a cursor locates its current time position, but without more information about the process activity. That's in the domain of live coding and interactive systems that the approaches look the more advanced. Thor Magnusson is developping a thought and tools that tend to narrow programming and music score [START_REF] Magnusson | The ixiquarks: Merging code and gui in one creative space[END_REF][START_REF] Magnusson | Algorithms as scores: Coding live music[END_REF]. Ge Wang proposes also an approach based on code [START_REF] Wang | On-the-fly programming: Using code as an expressive musical instrument[END_REF] as factor of instrumental expression for the live coding, with the programming language Chuck [START_REF]Chuck: a concurrent, on-the-fly audio programming language[END_REF]. This idea is notably implemented in the audio programming environment The Audicle [START_REF]The audicle: a contextsensitive, on-the-fly audio programming environ/mentality[END_REF] where different visualizations of the code and of the system activity are proposed in an original and extensible approach(figure 3). Another approach of the visualization is centered on the listener, aiming at improving the perception using graphical information in a live coding context [START_REF] Mclean | Visualisation of live code[END_REF] or to make the mechanism of electronic instruments perceptible [START_REF] Berthaut | Rouages: Revealing the mechanisms of digital musical instruments to the audience[END_REF] (figure 4).

The main limitation of all these approaches is to be partial and centered on the applications that propose them. In addition, there is no emergent general model and the interoperability between the tools is not taken into account.

CHARACTERIZATION OF A MUSICAL COMPUT PROCESS

We will consider music computer processes as resulting from the music composition and thus, as being part of the music score. From this viewpoint, a process takes place in time: it can be passed, present or futur. We will talk of active process for a process that is present, and of inactive process for a past or futur process.

The temporal status will determine different ways to characterize a process: the properties of a process will be different whether active or inactive. Similarly, properties of a past process will differ from those of a futur process, which date and duration may be undefined, relative to an external event.

The characterization of a musical computer process we will be based on three types of information, classified according to their change rate:

• a static state: represent the information that doesn't change (e.g. the parent process) or that may change at a low rate (e.g. the sampling freq.)

• a dynamic state: represents the information that changes over time and depend on the process execution (e.g. the CPU usage)

• a temporal state: the process start date and its end date or duration. These properties may be undefined in an interactive context.

Static state

A process static state is made of the set of information that are invariant or that change at a low rate. This state is completely defined for an active process. It reflects the last active state for a passed process. For a futur process, it may indicate the first state to activate when it is known in advance.

For a generic process, the state may include:

-its parent process -its status (active -inactive) -its computation mode (vectorization, parallélization,...) -...

For an audio process, the state may include:

-the I/O buffers size -the sampling rate -the I/O devices -the driver -...

For any process, the state may include the value of the control parameters. For example: delay and feedback values of an echo.

The set of static properties is process dependant.

Dynamic state

The dynamic state of a process is closely linked to its activity and characterizes an active process.

For a generic process, the state may include:

-the CPU usage -the threads count -the memory usage -...

The result of a process computation may also reflect its state, e.g. the values of a process that computes a signal.

The dynamic state may include indications on the current computation, e.g. an index of confidence for a process that is doing pitch recognition or that is doing score following.

The set of dynamic properties is process dependant.

Temporal state

In a musical context and especially for interactive music [START_REF] Rowe | Interactive Music Systems: Machine Listening and Composing[END_REF], the temporal state of process (i.e. its start date, its end date or its duration) may be partially or totally undefined.

In case of an active process, the start date is known but its end may depend on another process and/or an external event. For a future process, both the start and end dates may be undefined.

In addition, the way to represent dates and durations may lead to undefined results, e.g. when expressed in a time relative to a tempo that is undefined.

In a musical context, this temporal information is critical for the performance of a piece and thus, require to be represented whatever its status.

The temporal properties are common to all the processes.

REPRESENTING THE STATE OF A MUSICAL COMPUTER PROCESS

We propose a format to represent the state of a musical process, suitable to inter-applications collaboration, allowing to dissociate a process involved in a piece computation and its graphic representation, that could be viewed as part of the music score. We define the state of a process as a set of values that characterise this process at a given time. We'll talk of property to refer to one of these values. To represent a process state, this process should be able to describe its properties set.

Definition of a property

A process property is the association of an identifier and a value that may change over time. The value may be a number or a vector of values: e.g. a space position is defined by 3 values.

The value of a property may be bounded by an interval. Finally, the frequency of a property variation may also be defined.

The type of the values is in:

int : for an integer number float : for a floating point number bool : for a boolean probability : the value is a probability. By convention, it will be expressed by floating point numbers in the interval [0, 1].

Values of a vector type are defined by a list of types. By convention, the frequency defines the rate of a value variation: it indicates a number of changes by second. The value 0 denotes constant values. When the variation rate is unknown, the frequency should be omitted.

Declaration of a properties set

A process state is defined by a list of properties (figure 6). In order to represent this state, a process must be able to communicate a description of its state. JSON [START_REF] Crockford | The application/json media type for javascript object notation (JSON)[END_REF] could be used to declare the properties of a process. The figure 7 gives the example of a FAUST [START_REF] Orlarey | FAUST : an Efficient Functional Approach to DSP Programming[END_REF] process which state include the value of the control parameters as well as the computed signal.

A property state

The state of a property will be send as a pair associating an identifier and its value. { "process": "karplus", "states": [ { "ident": "excitation", "type" : "float", "range": [{"min": 2},{"max": 512}] }, { "ident": "play", "type" : "bool" }, { "ident": "level", "type" : "float", "range": [{"min": 0},{"max": 1}] }, { "ident": "attenuation", "type" : "float", "range": [{"min": 0},{"max": 1}] }, { "ident": "duration", "type" : "float", "range": [{"min": 2},{"max": 512}] }, { "ident": "signal", "type" : "float", "range": [{"min": -1},{"max": 1}], "freq" : 44100 } ] } When a list of values is associated to an identifier (figure 9), they could be interpreted in the context of the property frequency: when a frequency is defined, the set of values takes the corresponding duration.

{ "signal" : [ -0.2, 0.1, 0.23, -0.05, -0.01, 0.8, 0.8, 0.02, -0.5 ] } Figure 9. Values of the signal computed by the process karplus OSC [START_REF] Wright | Open Sound Control 1.0 Specification[END_REF] may be used to a state. In this case, the OSC address could be used as identifier (figure 10)

Temporal state representation

All the processes have two common properties: a start date and a duration (or end date). These properties may be undefined (e.g. for a futur process), or partially defined (e.g. for an active process which end date is undefined).

In the context of interactive music, an undefined date corresponds to an external event. For example, -a process start or end may correspond to the start or end of an improvisation sequence, /karplus/signal -0.2 0.1 0.23 -0.05 -0.01 0.8 0.8 0.02 -0.5

Figure 10. Values of the signal transmitted via OSC -a process mey be conditionally triggered, e.g. when a specific note sequence is played or in case of silence.

More generally, the scheduling of such processes may be described in terms of Allen relations [START_REF] Allen | Maintaining knowledge about temporal intervals[END_REF], relatively to the events which they depend on. We will talk of event based date or duration to refer to these undefined dates or durations.

The properties of a process temporal state are described by a start and an end or duration (figure 11). The values can be expressed as time or under event form. Even when a process has an event based date, we would like to represent it, at least in an approximative manner. In order to provide support for such representation, an event based date is defined as a triplet T = (t lef t , t, t right ), associated to a confidence level P and followed by an optionnal label (figure 12).

T is such that t lef t t t right . T defines a realisation interval [t lef t , t right ] and a possible realisation date t. The confidence level [t lef t , t right ] represents the realisation likelihood of the event at the date t. It is expressed as a floating point value in the interval 0, 1. expressed in terms of Allen relations (figure 13) could be expressed in a semi-instanciated way as follows:

{ "process": "A", "start": "0/1", "dur": [ "1/4", "1/2", "1/1", 0.7 ] } { "process": "B", "start": [ "1/4", "1/2", "1/1", 0.7 ], "dur": "1/4" } { "process": "C", "start": [ "1/4", "1/2", "1/1", 0.7 ], "dur": [ "1/2", "3/4", "1/1", 0.7 ] } { "process": "D", "start": [ "0/1", "1/2", "1/2", 0.8 ], "end": [ "3/4", "5/4", "3/2", 0.5 ] } Note that the start date of the process D expresses a constraint on the duration, that should be greater or equal to a whole note.

One of the possibilities for the representation may consist to use a color gradient to account for uncertainties, as illustrated in figure 14, that is based on the example above.

CONCLUSION

propose a simple description of a musical process state. This description is disconnected from any representation format or communication protocol. However, examples using JSON or OSC are given because their simplicity of implementation was consistent with the proposed description of musical processes.

The critical problem of the representation of event based time is treated using a probabilistic semi-instantiated approach. This solution is less general than a description in terms or Allen relations, but it avoids solving the corresponding constraints to visualization applications, while realization systems have already to do it.

  Copyright: c 2014 D. Fober et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 3.0 Unported License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Figure 1 .

 1 Figure 1. Visualization of the system state under Windows

Figure 2 .

 2 Figure 2. Visualization of the system state with Max/MSP and Pure Data

Figure 3 .

 3 Figure 3. Visualization of the system state in the Audicle : a) waveform and spectrum, b) temporal information per thread, c) threads activity in a graphic form, d) in a detailled textual form.

Figure 4 .

 4 Figure 4. Visualization electronic instruments mechanism.

Figure 5 .

 5 Figure 5. A property definition

Figure 6 .

 6 Figure 6. Description of a process state

Figure 7 .Figure 8 .

 78 Figure 7. A FAUST process described using JSON

Figure 11 .

 11 Figure 11. Properties of a process temporal state

Figure 12 .

 12 Figure 12. Approximation of event based time.This kind of representation is intermediate between the description of Allen relations and classical dating. As an example, the following relation A m (B si (C fi D))expressed in terms of Allen relations (figure13) could be expressed in a semi-instanciated way as follows:

Figure 13 .

 13 Figure 13. Relations between 4 processes: B and C start with the end of A, D ends with C.

Figure 14 .

 14 Figure 14. Representation of the processes A, B, C, D, using color gradients to account for uncertainties.

Acknowledgments

This research has been conducted in the framework of the INEDIT project that is funded by the French National Research Agency [ANR-12-CORD-009].