Jaime Arias

Myriam Desainte-Catherine

Camilo Rueda

FORMAL MODELS TO COMPOSE AND EXECUTE INTERACTIVE MULTIMEDIA SCORES IN REAL-TIME

INTRODUCTION CONCLUSIONS

Nowadays, the design of interactive multimedia systems based on a written scenario is a challenge that requires to handle dynamic and static events as well as dynamic and static data. Interactive scores (IS) [START_REF] Desainte-Catherine | Towards a Hybrid Temporal Paradigm for Musical Composition and Performance: The Case of Musical Interpretation[END_REF] propose a model to write and execute interactive scenarios comprised of several multimedia processes. In IS, multimedia processes are temporal objects, represented as boxes, whose temporal organization is defined by asserting temporal relations those objects should obey. IS may also contain interactive points. These are used as triggers to modify temporal relations during execution. An important requirement of this model is that the temporal constraints defined by the composer must be preserved during the execution of the scenario.

Currently, the software i-score implements the model described above in a visual environment. Its execution model is based on Hierarchical Time Stream Petri Nets (HTSPNs) [START_REF] Sénac | Hierarchical Time Stream Petri Net: A Model for Hypermedia Systems[END_REF]. Places and transitions of Petri nets model temporal aspects of the scenario (i.e., they define a partial order between static and dynamic events) defined during its composition. Such implementation provides an efficient and safe execution, but implies some limitations: (1) it presents a quite static structure and doest not support dynamic creation of processes; (2) it does not support conditionals and loops; (3) it cannot handle complex data; and (4) the visual language complicates the representation of some abstract programming concepts such as control structures, recursion and variables, and also limits the ability of expression in the temporal organization of scores. In this poster, we present some approaches to overcome the above limitations in the execution model of i-score.

We presented several approaches to overcome the current limitations of the execution model of interactive scores. Moreover, we extended the model with some features such as the notion of asynchronous functional composition and a real-time visualization system. Finally, we introduced a complete framework that allows to compose, verify and execute (on a physical platform) open scenarios.

We extend the current execution model of interactive scores with the capability to handle complex data [START_REF] Arias | Modelling Data Processing for Interactive Scores Using Coloured Petri Nets[END_REF]. For that, we used Colored Petri Nets (CPNs) [START_REF] Jensen | Coloured Petri Nets. Modelling and Validation of Concurrent Systems[END_REF] to model both complex data and the dynamic aspect of the functional composition of processes.

Multimedia streams are often cut into temporal frames to be carried from one process to another. Thus, we model frames as colored tokens that are handled by processes. Our approach provides the notion of asynchronous functional composition. This corresponds to the case where the composed processes are not executed at the same time. Then, it requires to buffer the output data stream of processes in order to hold data until another process read them.

We used CPN tools for prototyping and simulating our model and we implemented modules for reading, appending and reversing audio files.

COLORED PETRI NETS

The graphical environment of i-score provides composers with a visual environment to design, create and modify scenarios in an intuitive way. However, in visual programming the representation of some abstract programming concepts such as data structures, control structures, functions, recursion and variables is very hard or sometimes impossible [START_REF] Erwig | Heterogeneous Visual Languages-Integrating Visual and Textual Programming[END_REF]. In addition, in the case of i-score, it also limits the ability of expression in the temporal organization of scores [START_REF] De La Hogue | OSSIA : Open Scenario System for Interactive Applications[END_REF].

Therefore, we introduced a new programming language, called ReactiveIS, that allows to take advantage and extend the full capacity of temporal organization during the composition and execution of interactive scores. For that, we present a clear and extensive syntax, and also the operational semantics of the programming language.

The operational semantics is defined using tree structures and simple operations on them, which provides a practical and intuitive formalization of the behavioral and the hierarchical aspects of interactive scores. Moreover, the definition of a formal semantics will allow a static and dynamic analysis of the written scores.

DOMAIN-SPECIFIC PROGRAMMING LANGUAGE

The current implementation of i-score (version 0.2) includes conditional relationships between boxes. However, this new feature lacks of a well-defined formal semantics. Timed Automata (TA) [START_REF] Alur | A Theory of Timed Automata[END_REF] have been previously used in music applications showing to be a powerful model for describing both the logical ordering of events and also the durations between them [START_REF] Echeveste | Operational Semantics of a Domain Specific Language for Real Time Musician-Computer Interaction[END_REF]. Thus, TA are well suitable for expressing the timing constraints appearing in interactive scores. In this new approach, we define interactive scores equipped with conditionals as a network of timed automata. Moreover, we have preliminary results on modeling loops. Unlike CPN, TA have a variety of powerful multi-platform tools (e.g., UPPAAL) to design, simulate and verify real-time systems modeled as TA.

Our aim is to develop a framework that start with the composition of the interactive score in i-score, followed by the translation of the scenario into a TA in order to verify some important properties such as playability and the maximum number of boxes executed simultaneously. Finally, the verified model is translated into an executable code that enables a physical implementation (i.e., a chip). We pay special attention to the true parallelism offered by a Field Programmable Gate Array (FPGA). Compared to the operating system based platforms, the FPGA platform is able to achieve a much faster sampling frequency, and it is not affected by the rather complex behavior of the operating system services.

TIMED AUTOMATA

A disadvantage of the model based on Petri Nets is that the model is very static and modeling new features would require a complete redesign of the network or sometimes they cannot be expressed. Additionally, a problem of i-score is that it does not provide a good visual feedback in real-time of the execution of the scenario. Therefore, we explored a new way to define and execute interactive scores aiming at a more dynamic model [START_REF] Arias | Executing Hierarchical Interactive Scores in Re-activeML[END_REF].

For this purpose, we used ReactiveML [START_REF] Mandel | ReactiveML: a Reactive Extension to ML[END_REF], a functional programming language for implementing interactive systems (e.g., video games and graphical user interfaces). This language is based on the synchronous reactive model of Boussinot [START_REF] Boussinot | The SL Synchronous Language[END_REF], then it provides a global discrete model of time, clear semantics, and unlike Petri nets, synchronous and deterministic parallel composition and features such as dynamic creation of processes.

We took advantage of the synchronous model of ReactiveML and developed a visualization system in INScore [START_REF] Fober | An Environment for the Design of Live Music Scores[END_REF] that behaves like a synchronous observer of our interpreter [START_REF] Halbwachs | Synchronous Observers and the Verification of Reactive Systems[END_REF]. Therefore, it listens the events emitted by the interpreter and changes the temporal organization of the score in the graphical interface.

REACTIVE PROGRAMMING LANGUAGE

Declarations

  colset TIME = time; colset UNIT = unit; colset INT = int; colset BOOL = bool; colset DATA = INT; colset DURATION = TIME timed; colset FILE = product INT*DATA; var f_dur : TIME; var e : BOOL; var n_max : INT; var n : INT; var f : DATA; (A, 5, 5) ^ WaitFromEnd (B, 3, 3) stop: WaitFromEnd (C.D, 1, ∞) ^ WaitFromEnd (C.E, 1, ∞) (C, 1, 1) stop: WaitFromStart (C.E, 2, 2) Messages start: /sound/2 on stop: /sound/2 off E Conditions start: WaitFromStart (C, 2, 5) ^ WaitEvent (/mouse 1) stop: WaitFromStart (C.D, 1, 1) Messages start: /sound/1 on stop: /sound/1 off interactive score with five process boxes and two hierarchical boxes (Box. 5 and the whole scenario). Box.3 has an interaction point at the end whereas Box.5 has two interaction points. CPN module for reading an audio file. Reading audio files consists in acquiring audio frames from a file with a determined frequency (i.e., duration between frames). Graphical interface in INScore. Colored boxes represent the boxes that are currently executing. Tree representation of a scenario wrote in ReactiveIS. Nodes with messages represent process boxes while the others represent hierarchical boxes. Each node has both a condition to start and a condition to stop. Framework flow: composition, verification and real-time execution.

ACKNOWLEDGMENTS

The authors would like to thank Carlos Olarte and Sylvain Salvati who helped us a lot in the development of these ideas. Also, we would like to thank Louis Mandel for his valuable remarks about the implementation in ReactiveML. This work has been supported by the OSSIA (ANR-12-CORD-0024) project and SCRIME.


Alexis Pierrard

Embedding sociophonetics in grammar: aspects of structural complexity in Southern Bolivian Quechua

Language background

Southern Bolivian Quechua (SBQ) is a very homogeneous dialectal network, phylogenetically linked to the QIIC Quichuan languages subgroup and largely mutually intelligible, despite some noticeable differences, e.g. with the other cuzqueño-bolivianos dialects [START_REF] Cerrón-Palomino | Lingüística Quechua[END_REF]. SBQ consonantism is characterized by the absence of voicing correlation and by the presence of an autosegmental glottal trait involved in the realization of ejectives and aspirated stops in five places of articulation (labial, dental, palatal, velar, uvular). Nevertheless, ejectives in SBQ are strongly constrained, as they are contact-induced, and therefore, embedded in specific lexical sets. SBQ proceed by concatenation of derivational and flexional suffixes on mono-or disyllabic (more rarely trisyllabic) lexical roots, whose template has the shape CV(C)CV(C)-. SBQ shows a very strong phonological and morphological homogeneity, which may explain the absence of systematic dialectological previous works and the limited place given to variability in existing grammars, dictionaries and textbooks (notes available in [START_REF] Albó | El quechua a su alcance[END_REF][START_REF] Bills | An Introduction to Spoken Bolivian Quechua[END_REF][START_REF] Plaza | Qallarinapaq. Curso básico de quechua boliviano[END_REF][START_REF] Laime | Diccionario Bilingüe, Iskay Simipi Yuyayk'ancha: Quechua-Castellano Castellano-Quechua[END_REF].

Allophonic microvariation and sociolinguistic complexity

Nonetheless, the diversity of territories, local and regional identities, along with deep socio-economic changes in Bolivian society, do have an impact on the complexity of social and sociolinguistic profiles of the individuals and the language communities, since the second half of the 20th century (Pierrard forthcoming). This ecological and sociological intricacy seem propitious to the presence of sociophonetic variations, actually only available through a finegrained observation of allophonic microvariations, embedded in grammar. In this work, we are wondering how these variations represent a challenge for the description of SBQ, and we make an attempt for the understanding of patterns of structural stability and change, through sociophonetic variation, from the standpoint of linguistic ecology.

We will only focus here on the observation of uvular stops. Typologically rare, uvular stops have the advantage of being present in all syllabic position and of being absent in Spanish, the dominant language in Bolivia and mostly spoken by Quechua speakers. Existing descriptions mention the existence of unvoiced uvular stops in onset position and of a fricative allophone in coda position, phenomenon described by: q→ χ/_# ; _C.

This idealized pattern seems to be actually limited to the peripheral (with respect to the Quechua dialectal network) region of the Altiplano.

Qualitative observation by listening and spectrogram reading of native speakers recordings from the whole Quechua speaking area show a great richness of micro variations implying phenomena such as voicing, affrication, spirantization, approximantization, even elision or pharyngealization. These lenition phenomena show us an instability in the realization of the uvular phoneme that allows us to draw different idio-, geo-or sociolects on the basis of a single variable. In order to catch the complexity of the coexisting grammars and the interactions with social parameters we have to take into account different dimensions of phonetic or phonological properties such as syllabic position (lenition is not random but follow a strength scale), intrinsic intensity of the nucleus or consonantal correlation. This multidimensional approach is an attempt to embed sociophonetic variables as coexisting grammars in a single dialectal network.

Uvular realizations

Quechua speakers (main language) in relative and absolute values (own draft based on Census 2012) The table above shows the main coexisting allophonic grammars observed into the dialectal network of SBQ. This is the first necessary step before a sociophonetic analysis of the uvular stop phoneme. This analysis must be searching the dependent variables responsible not only of the distribution of two variants of a single phoneme, but of the whole set of lects or allophonic grammars.

[q] [q] [q] [χ] [χ] #_ C_ V_V Coda q χ q ʁ χ q χ ʁ χ G ʁ χ G χ χ Postconsonantal

  unvoiced stop <nisqayki> « I'll tell you » Postconsonantal voiced fricative <willasqayki> « I'll tell you » Initial unvoiced stop <qallu> « tongue » Initial voiced stop <qallu> « tongue » Different uvular allophonic grammars SBQ consonant inventory