Ph Depalle

G García

X Rodet

A Virtual Castrato (!?)

In this paper we present an original research work culminating in the high quality production of thirty-nine minutes of castrato voice, by the means of sound analysis, processing and synthesis. The goal of the project is the creation of a soundtrack for a film about a famous eighteenth century castrato. Two complementary voices -a coloratura-soprano and a counter-tenor -are used to cover the entire range and compensate for technical difficulties. Then their timbres are homogenised by using combined techniques such as phase-vocoding, additive analysis/synthesis and spectral envelope and pitch estimation.

Introduction

In this paper, we present an original work on the recreation of a castrato voice through sound analysis, processing and synthesis. It has been done to produce the soundtrack for a musical film [Corbiau,94] about Farinelli, the famous castrato of the eighteenth century. Farinelli, issued from the castrato's music schools of Porpora and Bernachi, and as a contemporary of Haëndel, was a celebrity of the Baroque era like "pop stars" are in the present [Dico,91]. The film, realised by Gerard Corbiau, brings back to life a repertoire which could not be sung anymore. The musical consultant of the film Marc David [David,94] has "recovered" unedited scores from the French National Library.

Thirty-nine minutes of high quality processed singing have been created for a CD of the soundtrack, produced by Audiovis [Audiovis,94].

In the following sections we explore some of the issues by describing the characteristics of castrati voices and the practical constraints of the film project. Then, we detail the chosen solution and the specific processing used to make such a voice.

Problematic

Castration has been forbidden in western countries since the last century, and the last occidental castrato died in 1922.

Castrati were generally well known for the special timbre of their voices : due to the surgical intervention they had undergone, their voices had not changed with puberty. Furthermore, with maturity, castrati lung capacity, chest's size, physical endurance and strength were generally greater than those of normal male [Sauvage et al.,84]. Consequently, they were able to sing very powerfully. Farinelli could sustain a note longer than one minute and he could sing long phrases of more than two hundred notes without seeming to take a breath. Their small and supple larynx along with their short vocal cords, allowed them to vocalise in a large range (up to three octaves and a half) and to sing with a great vocal flexibility (they could sing large intervals rapidly, cascading scales and trills). Furthermore, castrati were selected among the best child singers and trained very intensively. As castrati's specific repertoire takes their expert singing technique into account, this repertoire is extremely difficult to sing. Some pieces of music are interpreted today but they are chosen for their simplicity or are played at a lower tempo.

Another difficulty in recreating a castrato voice is the lack of recorded references. The last occidental castrato has recorded less than one hour of singing on wax cylinders between 1902 and 1904 [Moreschi,88]. This historical recording has very little technical utility due to its extremely poor quality.

Nevertheless, we can take into account the physical characteristics of the whole vocal production system of the castrati, the global aesthetic of the historical recording and descriptions found in the literature. On the other hand, the design of the final processed voice is conditioned by the wishes of the film and music producers. The definitive choice concerning the timbre of the voice is a compromise between the two preceding constraints.

Scores are difficult to interpret and parts of them can not be sung by contemporary singers anymore, due to the large range of the castrati. Thus it is necessary to use two complementary voices to cover the entire range and compensate for technical difficulties.

Recording and Editing of the Voice

Several constraints had to be respected to choose the two voices. Basically, the desired voice had to sound like an ambiguous juvenile male voice. A large casting was made by the film and music production to find a counter-tenor and a coloratura-soprano with similar and good baroque singing techniques (especially in terms of vibrato and articulation). Chosen singers are Derek Lee Ragin and Eva Godlevska.

The recording was made in the concert hall "L'Arsenal" in Metz, France with the orchestra "Les Talens Lyriques", conducted by Cristophe Rousset. Due to artistic constraints, sound engineer Jean Claude Gaberel recorded voice and orchestra simultaneously, despite the evident interest of a multitrack recording. One consequence is the presence of orchestra components at 20 to 30 dB under the mean average level of the singing voices. This drastic constraint imposed a certain robustness to our processing method. Technically, the recording was made on a Nagra IV-D machine with a precision of 20 bits.

The editing has been made by Jean Claude Gaberel on a Sonic solution machine. This remarkable work often reached the note-by-note editing level.

To conclude, we should notice several points inherent to the use of two singers: The perceived dynamic is different between them and one can sometimes hear in the middle of a phrase unnatural discontinuities which sound like phrase attacks.

Processing Introduction

The chosen strategy (Cf. figure 1) can be divided in two steps: First, since one of the artistic specifications was to make the finally processed voice sound close to the counter-tenor one, we modified the coloratura-soprano parts to match the counter-tenor timbre. This procedure, which we call voice morphing, constitutes the main and critical step of the scheme.

Secondly, we gave the voice a more juvenile aspect by using global modifications. For instance, we attenuated some high frequency bands to reduce the kind of breathiness found in Derek Lee Ragin's voice. We also made the voice sound brighter by modifying the spectral envelope.

Data Base Constitution

Due to the great predominance of vowels over consonants, we only processed the vowels. As vowel timbre is not only a function of phonemes but also of pitch and intensity, a specific processing has to be applied to each vowel note. Thus, a reference data base composed of all the combinations phoneme-pitch-intensity of the counter-tenor voice had to be set up. Phonemes are represented by spectral parameters which will be detailed further. Since song texts are written in italian language, we only use the five following phonemes /a/, / /, /i/, /o/, /u/. Pitch are chosen chromatically from 185 Hz to 987 Hz and intensity has only three levels: piano, mezzoforte and forte.

For practical reasons, the data base does not cover the entire intensity range. To complete it, we used intensity rules [Rodet et al.,89] to compute the missing fields.

Segmentation

Once the data base of the reference voice had been built, the musical phrases to be processed had to be segmented in order to label elementary portions in terms of singer, phoneme, pitch, power, begin and end time. Precise pitch estimation was made by the new frequential method described in [Doval,94]. A first segmentation pass was performed automatically on the fundamental frequency evolution [Cerveau,94]. Then, a second pass was performed by hand on the signal to adjust the begin and end time of the vowels and to give the singer and phoneme labels.

Reference Selection

The reference selection step associates each vowel segment found in the voice with a target phoneme in the data base. For example, every phoneme /a/ sung mezzoforte by the soprano with a pitch between 415 Hz and 520 Hz can be associated with the data base phoneme /a/ sung mezzoforte by the counter-tenor in the same range of pitch. In practice, this correspondence is not hard-coded but can be given in a parameter file. In this way, it makes the transformation process more flexible. For example, multiple versions of the data base elements coexist, corresponding to different recording conditions such as microphone distance, equalisation, etc. So, one of these versions can be selected by including its corresponding reference in the parameter file. Beyond standard voice morphing, this reference selection allows more dramatic transformations such as changing every phoneme /a/ into phoneme /o/.

Spectral Characteristics of the Voices

The basic idea of our voice morphing technique consists in modifying the spectral envelope of the soprano voice to match that of the counter-tenor voice. This is achieved by a frequency domain filtering through a phase-vocoder [Depalle et al.,91] (Cf. figure 1). But we use a more refined technique which is described now.

Since the scores we use are written for castrati, most of the songs are high-pitched, and it's a common fact that in this case the frequency response of the vocal tract is poorly estimated. This is particularly true in the low frequency range (below 2.5 kHz) where partials are widely spaced and formants are very narrow. The first consequence of bad estimation is that voice morphing does not reach the reference timbre; another consequence, which is specific to our context, is that transformation may emphasise some partials of the orchestra.

One possible solution to improve the estimation of the spectral envelope in the low frequency range is to use time evolution of frequency and amplitude of the partials, which scan the spectral envelope as can be seen in figure 2. Thus, a method which estimates a spectrum envelope model by minimising its distance to the set of frequency-amplitude points, such as the discrete cepstrum [Galas, 90] could be considered.

But in practice the spectral envelope is not always stable during a vowel note. First, the coloratura-soprano often changes continuously the shape of her vocal tract when singing a cascade of notes on the same vowel (Cf. figure 3).

In addition, tremolo correlated to the vibrato effect induces a variation on the amplitude of each partial, which superimposes on the scanning of the spectrum envelope (Cf. Figure 4).

In the middle (2.5 to 5 kHz) and upper frequency range (greater than 5 kHz), the estimation of the spectral envelope remains valid, because of the wider shape of the formants. But if the spectral envelope shape is still constant for a given vowel note, its global amplitude is modulated and fluctuates according to the tremolo. Moreover, this effect is emphasised by the loudness. Finally, in the upper frequency range, the average level is perceptually more important than the precise shape of the spectrum.

A (dB)

Frequency Response Building

Taking the preceding observations into account, we decided to build a filter frequency response which only acts on the vicinity of each voice partial. In this way, partials of the orchestra, present in between partials of the voice, are not emphasised. So, we designed frequency responses of rectangular form (Cf. figure 5). The rectangles are designed using additive synthesis parameters Depalle et al.,93] and spectral envelopes. For each partial, a rectangle, centred around its frequency is designed. Its level is computed to impose the same relative amplitudes between harmonics on the processed sound as those of the corresponding phoneme stored in the data base. The width of each active band is computed according to the size of the temporal window currently used by the phase vocoder and the frequency deviation due to the vibrato in this window.

In the medium and high frequency range, we evaluate the height of the rectangle by dividing the spectral envelope of the desired phoneme stored in the data base by the spectral envelope of the phoneme to be processed. In addition, frequency active bands are weighted by a coefficient which controls the breathiness of the result.

Conclusion

Our work constitutes one step towards the very difficult goal which consists in transforming one instrument into another. Futhermore thirty-nine minutes of concert quality singing voice have been produced. This work is all the more interesting because of real world production constraints implying recording conditions which were not optimal four our purposes. For instance, the presence of the orchestra imposed the development of very robust processings methods. Finally one of the most major interesting points of this works is that it now allows us to revive the whole musical repertoire of pieces originally written for castrato. Extracts of the pieces of music will be played during the conference.

Figure 1 :

 1 Figure 1: General Synopsis of the voice processing.

Figure 2 :Figure 3 :Figure 4 :

 234 Figure 2 : Time-trajectories of the first ten partials in the Frequency-Amplitude plane, for the phoneme / / sung by the coloratura-soprano. A (dB)

Figure 5 :

 5 Figure 5 : Frequency response building.

Acknowledgement

The authors would like to thank Digital Equipement Corporation France for the DEC alpha 600 and a JV 300 audio board used in this work.


Xavier Rodet

Adrien Lefevre

Macintosh Graphical Interface and Improvements to Generalized Diphone Control and Synthesis

Generalized diphone control is a powerful means of building a musical phrase from dictionaries of analysed sound units by concatenating and articulating them. The Diphone program, developed at IRCAM for diphone control, has been improved with fundamental frequency, noise components, spectral envelopes and parallel sequences. A new graphical user interface on Macintosh is presented. It communicates with Diphone through a simple command language. It allows for the managing dictionaries, the building and the articulation of parallel sequences of diphones, and for the editing of parameter values.

Introduction

Generalized diphone control has been presented in [Rodet 93]. A spoken sentence can be modelled as a succession of transient sounds (called diphones). Any sentence can be reconstructed from a dictionary of diphones. In order that duration and fundamental frequency can be modified, an analysis-synthesis model, such as source-filter [Rodet 88], is used. Therefore, it is not the diphone sound signal itself which is stored in the dictionary, but the corresponding control signals of the analysis-synthesis model. A sentence is obtained by concatenation of control signals of a sequence of diphones. In [Rodet 93] we have extended the concept of diphone control to musical sounds in general. A dictionary can include any segment of sound considered as an atom for the musical usage in view. Such an atom, in a representation like additive representation or source-filter representation, is called a segment. Diphone control does not rely on a particular synthesis technique. We have focused on the additive model, i.e. a sum of sinusoidal partials with time-varying frequencies and amplitudes defined at frame times. A segment is a data structure containing, in the additive case, the frame times and the associated parameters for a segment of sound. A musical phrase is obtained by concatenating basic segments to produce a new segment from which a sound signal is computed by the additive synthesizer (Fig. 2). In [Rodet 93], we have detailed the Diphone program written at Ircam for diphone control and synthesis. We now present recent improvements to diphone control and synthesis, and a Graphical User Interface with novel characteristics, both from the conceptual and the implementation point of view.

Improvements

A segment has to contain also its original (time-varying) fundamental frequency, say F0(t), since a desired fundamental frequency trajectory, say G0(t), is obtained by applying the transposition factor G0(t)/F0(t) to the sinusoidal partial frequencies (Fig. 2). For music written in terms of notes with well defined pitch, the segment paradigm applies nicely to each segment with constant written pitch. Since notes can be largely independent of phones or other timbre attributes, a musical phrase has to be defined by two sequences of segments, one for the pitch, the other for timbre attributes such as phones. We say that these two sequences are parallel (Fig. 3) since none of each should necessarily impose its metric structure on the other. On the contrary, vibrato tends to be synchronized on notes. In consequence, vibrato frequency and excursion can be defined by the same segments as pitch or by sub-segments of pitch segments. Finally, G0(t) is computed by applying the defined vibrato to pitch values given by pitch segments. Other articulations, such as portamento or loudness are implemented in the same way.

Random components of sounds, like flute noise or voice fricatives, are not correctly represented as sinusoids with parameters recorded in a segment, but can be represented as white noise filtered through a time-varying spectral envelope. To be used at the synthesis stage, the values of the noise spectral envelope at all frame times are also stored in a segment (Fig. 2). In a preliminary stage, dictionaries of diphones, i.e. of segments, have to be constituted (Fig. 1). First, an additive+noise analysis [Depalle 93] is performed on the sound recordings. Secondly, the analysis data are segmented according to the segment time limits chosen by the musician. Finally the segments are stored in dictionaries. Another improvement brought to the Diphone program is the use of spectral envelopes for sinusoidal partials as well. A source-filter model is well suited for certain classes of sounds such as the voice. In this case, sinusoidal partial amplitudes are determined by the value of the spectral envelope at the frequencies of the partials. At the stage of the synthesis of a segment, these amplitudes have to be recomputed when the fundamental frequency is modified. Therefore, the spectral envelopes of sinusoidal partials have to be stored also in a segment and used at the synthesis stage to compute the amplitudes of the partials (Fig. 2).

Fig. 2: Concatenation and synthesis of a sequence of segments

Finally, the Diphone concatenation and articulation program is given a textual interface in the form of a simple Command Language opening access to all its facilities. In this way, Diphone can easily be used an tested separately and is totally independent of any GUI which is usually platform dependent.

A Macintosh Graphical User Interface

A Macintosh Graphical User Interface (GUI), named MacDiph has been built to provide an easy access to the Diphone program. It is tested on PowerPC and Macintosh-68K platforms. It is written in C++, compiled with Metrowerks-CodeWarrior IDE 1.4, and built on the Metrowerks-PowerPlant set of classes. From these classes, two groups of classes have been derived. They are not specific to MacDiph but designed for general graphic programming. The first group is aimed at displaying and editing graphs and objects, such as diphone sequences. The second group of classes is aimed at displaying and editing tree structures, such as dictionaries of diphones and their constituents, i.e. instruments, composite segments and basic segments. Finally, a break-point function editor is being built for the control-signals contained in segments. The different tools provided by these classes are, as much as possible, compliant with the Macintosh Human Interface Guidelines (Inside Macintosh). In particular, they offer copy, cut and paste, as well as drag and drop facilities, and follow the Wysiwyg guidelines.

We have taken care to separate the GUI from the Diphone program itself. MacDiph communicates with Diphone by using the Command Language mentioned above. This permits also to have a version of Diphone running on a fast Unix platform and be driven, through the network, by MacDiph running on a relatively slow Macintosh. This connection is implemented with sockets. Segments also can be read and written by MacDiph for display and editing. Since segments can be huge data structures and, on the contrary of the Command Language, cannot be handled by users, this is done through binary streams.

MacDiph provides the usual functions of a data base for a set of diphone dictionaries, i.e. browsing through different dictionaries, displaying their content, modifying them, selecting instruments and segments, constituting new dictionaries and saving them (Fig. 3).

Fig. 3: Two parallel sequences and Fundamental frequency evolution

The drag-and-drop paradigm is used in order to move segments between dictionaries and to build various parallel sequences. A sequence can contain basic segments and sub-sequences. Segments and sequences are represented as graphical objects (Fig. 4) directly displaying their characteristics, i.e. duration, center, interpolation portion between successive segments, loudness and articulation speed. Click-and-drag allows easy change of these characteristics in a Wysiwig style. Parameter evolution, such as sinusoidal partial frequency or fundamental frequency evolution, as stored in segments or as computed from a sequence, can be displayed and edited in graphical windows (Fig. 5) placed under the sequence windows for easy visualisation of synchrony. Modify, cut, copy and paste are fully supported on sequences and on parameter evolution.

Conclusion

MacDiph and Diphone constitute a promising tool for musicians. On one hand, diphone control offers new possibilities for precise and powerful control of synthesis, which could not be obtained in another way. The ability to build and articulate complicated sequences of segments from diverse origins appears to be an attractive feature. On the other hand, MacDiph implements a new direct representation and handling of segments in terms of intuitive graphical objects. As opposite to discrete values, such as notes, the control of continuous quantities [Rodet 84] has always been a difficulty in computer music. MacDiph and Diphone should bring some help in that domain by establishing a close connection between discrete and continuous representations.

Fig. 1 :

 1 Fig. 1: Analysis data are segmented into segments stored in dictionaries

Fig. 4 :

 4 Fig. 4: Management of a dictionnary of instruments, segments and basic segments


François Déchelle

Maurizio De Cecco

Miller Puckette

David Zicarelli

The Ircam ``Real-Time Platform'': Evolution and Perspectives

This paper describes the current evolutions and perspectives of the FTS real-time executive as the colonne vertébrale of the Ircam ``Real-Time Platform'' project, which is a software oriented continuation of the ``Ircam Musical Workstation'' project.

Introduction

The most popular real-time DSP oriented software for the well-known Ircam Musical Workstation [START_REF] Lindemann | The Architecture of the IRCAM Musical Workstation[END_REF] is FTS [START_REF] Miller Puckette | FTS: A Real-Time Monitor for Multiprocessor Music Synthesis[END_REF], a distributed real-time executive. FTS is a message-oriented system, in which objects' activations are scheduled statically for DSP computations and dynamically for control computations. FTS has usually been used as a back end to the NeXTStep version of MAX.

The Open System approach

The Next Cube history proves once more that a long-lived technology must be as independent as possible of any particular hardware or software platform, and from vendor dependent choices. The current evolution of FTS is in the direction of a standard open system: FTS is becoming a hardware-independent system that can run on multiple hardware and software platforms, and interact with user applications in a well-defined framework.

FTS as an open server

The chosen approach is a client/server architecture, in which FTS is the real-time server of one or more client applications connected to it via a well-defined hardware independent protocol. The communication itself can use different transport layers: standard UNIX pipes, TCP connections or shared memory in specialized hardware architectures. The chosen transport is completely transparent with respect to the client application; the same application can use different transport layers in different sessions, depending on the configuration. The kind of server used, and the platform on which the FTS server runs, is also completely transparent to the application: the same client can connect to a standard Ircam Musical Workstation card, an Alpha workstation or a TMS320C40-based system in different sessions, without any difference from the client's point of view. Of course, different servers can include different features, like different sets of objects, different performance levels or a different audio I/O structure. A well-behaved client should not make assumptions as to the existence of a particular server feature, but the FTS protocol includes support to publish the main features of a server so that the client can actually know which features are supported by a particular server. One of the benefits of this approach is that the real-time architecture and graphical development environments are more fully decoupled, and can run on physically different machines connected through Ethernet or another network. A well-defined interface between the two components allows specialized applications, using FTS for their DSP computations, to be developed in programming languages other than Max. Currently, language bindings for the FTS library are provided in C, C++, Common Lisp and Dylan; other utilities developed include a shell command line interface to control the FTS server and FTS applications, and a Motif library to build specialized application interfaces connected to FTS. The support for multiple clients for the same server allows the use of Max as the development environment for the FTS part, while the final application interface is developed as an independent client by other means.

Multiple architectures

The immediate goal is the porting of FTS to commercially available hardware architectures, which are currently:

PC with add-on Texas Instruments TMS320C40 DSP boards providing the scalability required by large musical applications, Unix workstations with audio devices and high processing power, such as Silicon Graphics and DEC Alpha.

Evolution of FTS

A number of longer-term issues are also under consideration; a number of case studies with FTS users both in and outside of Ircam showed two areas of the system where improvements or added functionalities would be worthwhile. These are automatic partitioning and compilation of control algorithms.

Partitioning

Like money, computing power is never plentiful enough; most of the applications developed at Ircam with the Musical Workstation use the maximally expanded configuration (three cards, i.e. six i860 processors), and use the available computing resources, CPU and memory, to their very limits. The main consequence is that the partitioning of the different tasks on different processors is at best a critical process; in practice, once close to the machine limits, finding a good partitioning of the algorithm between the processors becomes the most time-consuming development activity. The other consequence is that working close to the limits implies that a number of stricly architectural and platform-dependent optimization choices are performed by the FTS programmer, and are hardcoded in the patch itself. In the anticipated multi-platform scenario, this is clearly a problem that will increase the cost of porting applications across platforms. In this scenario, a system support for application partitioning is seen as unavoidable. A development activity in this area has only just started; the goal is to reduce the development cost of distributed FTS applications: in the short term, we will give more accurate metering tools to monitor the load of the different parts of the system; in the medium term, we will develop semi-automated utilities for partitioning of FTS programs.

Compilation

The FTS DSP engine is currently highly optimized, at least at the single DSP operation level. However, the control algorithm is essentially interpreted. Investigation of finished works shows that the control part is not negligible and is sometimes the bottleneck, especially when the system reacts to external inputs. FTS developers often use workarounds, like extending the system audio FIFOs to cope with peak loads (which introduces audio delays that are sometimes difficult to handle), and splitting the control computation between different ticks by means of delay lines. In order to better optimize the control computation, the idea of compiling the control part of a patch instead of interpreting it, has been considered. Given the nature of the FTS programming language (simple data types, simple recursive control structure), compiling it directly to a low level byte code (or in theory to directly executable machine code), is rather straightforward; a feasibility analysis showed that for patches made of standard control objects (like int, float, trigger and select), using standard optimization techniques such as inlining, peep-hole optimization, and a bit of data-flow analysis, we achieve translation ratios around 1-2 machine instructions per FTS object, giving a theoretically possible improvement in control performance between one and two orders of magnitude over the existing system, depending on the target compilation language. The compiling algorithm is also compatible with an incremental implementation, something that is essential to avoid any impact on the ease of use of the Max programming environment. Compiling the control algorithm also allows for a better integration of the DSP computation (currently compiled to an intermediate language) and the control computation, permitting a more flexible semantic of the interaction between the two, more flexible scheduling strategies and also the application of global optimization to the DSP part (like elimination of sigobjects for instance). Using a compiled approach also allows the implementation of delivery-only versions of the FTS kernel, for use in embedded or dedicated applications.

Summary

We described in this paper the short-term and long-term evolution of the FTS real-time executive, making it an open server and adding new features such as partitioning tools for faster multiprocessor development and optimization of control computation.


Adrian Freed

Xavier Rodet

Philippe Depalle

Synthesis and Control of Hundreds of Sinusoidal Partials on a Desktop Computer without Custom Hardware

Introduction

The idea of synthesizing sounds by summing sinusoidal oscillations [Helmholtz 1863] has intrigued generations of musical instrument builders. Thaddeus Cahill's electromechanical implementations [Nicholl 93] illustrate graphically the basic challenge faced by these engineers--the creation of a large number of oscillators with accurate frequency control. Cahill used dynamos constructed from wheels of different sizes attached to rotating shafts ranging in length from 6 to 30 feet. The speed of each shaft was adjusted to obtain the required pitch. A total of 145 alternators were attached to the shafts. Since the vacuum tube and transistor (inventions of this century) were unavailable to Cahill, each rotating element had to produce nearly 12000 to 15000 watts of energy to deliver synthesized music to subscribers' homes.

In the late 1970's, the availability of single chip digital multipliers stimulated the construction of digital signal processors for musical applications [Allen 85]. Although these machines were capable of accurately synthesizing hundreds of sinusoids [DiGiugno 76], their prohibitive cost and limited programming tools precluded widespread use.

One hundred years after Cahill's work, despite rapid gains in computational accuracy and performance, the state of the art in affordable single chip real-time solutions to the problem of additive synthesis offers only 32 oscillators. Since hundreds of sinusoids are required for a single low pitched note of the piano, for example, current single chip solutions fall short by a factor of at least 20. This paper describes a new technique for additive synthesis, FFT -1 , that offers a performance improvement of this order. This technique also provides an efficient method for adding colored noise to sinusoidal partials, which is needed to successfully synthesize speech and the Japanese Shakuhachi flute, for example. Before describing the details of the FFT -1 method, additive synthesis will be compared to the popular synthesis methods: frequency modulation [Chowning 73] and digital sampling.

Additive synthesis, FM and Digital Sampling

Additive synthesis is a signal modeling approach and therefor inherits a rich history of signal processing techniques including methods to automatically analyze sounds in terms of sinusoidal partials and noise [START_REF] Garcia | Analyse des signaux sonores en termes de partiels et de bruit. Extraction automatique des trajets fréquentiels par des Modèles de Markov Cachés[END_REF], McAulay & Quatieri 1986[START_REF] Serra | A system for sound analysis/transformation/synthesis based on a deterministic plus stochastic decomposition[END_REF]]. The image below illustrates the result of such an analysis. It shows analyzed partial trajectories superimposed over successive short term spectra of a tenor voice recording.

Because independent control of every partial is available in additive synthesis, it is possible to implement models of perceptually significant features of sound such as inharmonicity, brightness loudness, and roughness. In commercial digital samplers, sound amplitude and pitch are readily controlled, but no fine control over the sound spectrum is possible for timbre manipulations. Recent products allow for coarse timbral manipulations by voice blending and filtering. However, without fine control over individual elements of the spectrum it is impossible to implement, for example, continuous changes in harmonicity.

Another important advantage of additive synthesis over digital sampling is that it allows the pitch and length of sounds to be varied independently [Quatieri&McAulay 92]. The sample rate conversion technique [Smith&Gossett 84] used in digital samplers achieves pitch alterations by changing the rate at which sounds are read from memory. This results in changes to the durations of those sounds.

Another important aspect of additive synthesis is the simplicity of the mapping of frequency and amplitude parameters into the human perceptual space. These parameters are meaningful and easily understood by musicians. This is in contrast to the parameter space of the frequency modulation synthesis method which maps awkwardly to the spectral domain through Bessel functions [START_REF] Corrington | Variation of Bandwidth with Modulation Index in Frequency Modulation[END_REF]].

Additive Synthesis by the Oscillator Method

Additive synthesis is usually done with a bank of sinusoidal oscillators. Let J be the number of partials of the signal to be computed at a certain time, namely a sample number n. Let the frequency, the amplitude and the phase of the j th partial, l<=j<=J, be named respectively fj, aj, and j. More precisely, since they are functions of time, i.e. of n, they are written fj[n], aj[n], and j[n]. Usually, fj[n] and aj [n] are obtained at each sample by linear interpolation of the breakpoint functions which describe the evolution of fj and aj. The phase, commonly ignored, requires careful treatment as discussed later. The j th partial is defined by: cj

[n] = aj[n] . cos( j[n]) j[n] = j[n-1] + F(2 fj[n],SR)
where SR is the sampling rate and the signal to be computed is:

J s[n] = cj[n] j=1
In the digital oscillator method, commonly used in frequency synthesizers [START_REF] Tierney | A digital frequency synthesizer[END_REF], the instantaneous frequency and amplitude are calculated first by interpolation. Then the phase j[n] is computed. A table lookup is used to obtain the sinusoidal value of this phase and the sinusoidal value is multiplied by aj [n]. Finally cj[n] is added to the values of the j-l previously computed partials The computation cost Cadd(J) of the oscillator method is of the form J per sample. Assuming linear amplitude and frequency interpolation, is the cost of at least 4 additions, 1 table lookup, 1 modulo 2 n , and 1 multiplication. The factor varies considerably between processors according to available parallelism. It is interesting to note that with careful pipelining all of the operations involved in the computation of a sample for each oscillator may be performed in parallel and that in current VLSI technology the table lookup operation has the longest latency. Oscillators based on higher order recursions [Smith&Cook 1992] and CORDIC operations [START_REF] Hen | CORDIC-Based VLSI Architectures for Digital Signal Processing[END_REF]] have been proposed, but implementations have not yet demonstrated significant performance advantages over the direct table lookup oscillator. This is because the cost saving associated with avoiding a table lookup is offset by the additional cost of maintaining and accessing additional state variables, the interaction between frequency and amplitude controls [Gordon&Smith 85], and managing the effects of finite wordlength .

Additive Synthesis by Inverse Fast Fourier Transform

In the FFT -1 method [Depalle&Rodet 90], computation of partials is not done by a bank of oscillators but by an Inverse Fast Fourier Transform, a transformation of a short term spectrum (STS) Sl Any signal can be exactly constructed in this manner, provided that exact successive spectra Sl[k] are computed and given to the inverse transform. For reasons of efficiency however some approximations are required. First, as shown in the STS magnitude below, a partial is represented in a spectrum by a few points of significant magnitude, typically K=9. This means that to build the contribution of a given partial in the STS Sl[k], K spectral values need to be computed and added to Sl [k]. If N is the size of a frame (typically N=256), this results in a gain in computation efficiency roughly proportional to N.d/K = 14. A second approximation is that if the parameters of a partial are slowly varying they may be considered constant during one frame, i.e. a pure sinusoid during the small interval of one frame. The advantage of this optional approximation is that the time-domain function is symmetric implying asa real spectrum. As explained later, this replaces a complex multiplication (implemented as 4 real multiply and 2 real add instructions) by 2 real ones. However, this advantage is balanced by some drawbacks. Since the window w has to be symmetrical, fast changes in amplitude or frequency require short windows, thereby decreasing the computational saving of the method. This tradeoff can be adjusted according to the application, and the processor in use. Also a dynamic adjustment according to the rates of changes in amplitude or frequency is conceivable.

Details of Construction of a STS

Let fj, aj and j be the mean values of fj[n], aj[n], and j[n] on the frame l. Let W be the Fourier Transform of the window w. Then the contribution of the partial j in the STS Sl[k] is aj . e i j. W[fj/SR-k], which means that W is shifted in order to be centered around fj and multiplied by the complex amplitude aj . e i j . Naturally this complex contribution is added in Sl[k] to the contributions of the other partials. Note that, as mentioned in the previous section, a complex-complex or real-complex multiply is required between aj . e i j and W[fj/SR-k] according to whether W is complex or real.

Optimizations

The computation of the inverse FFT has a fixed cost, independent of the number of spectral components. It amounts to a small fraction of the performance of modern processors. For example at a sample rate of 44.1Khz the computation of a 256 point real inverse FFT for every 128 output samples accounts for only 10% of the computational capability of the MIPS R4000 processor of the SGI Indigo workstation. In contrast to the FFT, the cost of the construction of a STS is proportional to the number J of components and must be minimized. This cost decreases with the number K most significant values in the spectrum W of the window. For this reason a good window has few significant values. But for an exact construction of signals, the sum of two overlapped windows is required to be exactly 1 in the overlapped region. Moreover, in this region the amplitudes and the frequencies of the partials should be interpolated by the windows from the value in one frame to the value in the next frame. This second, amplitude interpolation requirement suggests a triangular window which is not compatible with the first requirement. There is a simple way out of this dilemma: two windows --one in the frequency domain to satisfy the first requirement and the other in the time domain. The first window wl has as few significant values as possible and is used to compute the contribution of each partial to the STS Sl fulfilling the second requirement of a triangular window on the time-domain signal. Note that this procedure is not costly since it is applied globally for all the partials after the inverse FFT. But it insures that the overlap-add uses frames of signal with a triangular window, s[m].tr [m]. The advantage of this procedure is that amplitudes of partials are linearly interpolated as in the classical oscillator method. Moreover, by an adequate phase adjustment (discussed in the next section), this procedure allows for removal of a great deal of the amplitude modulation distortion which may appear when the frequency of a partial varies rapidly from one frame to the next.

Phase adjustment

To avoid confusion, superscripts l and l+1 will be used for to index successive frames: let f l , a l and f l+l , a l+1 be the frequency and amplitude of a partial in these frames. When f l != f l+l some amplitude modulation can occur by phase cancellation in the overlapped region as shown below.

This modulation is maximal at the point where the two triangular windowed overlapped sinusoids are of equal amplitude. The corresponding instant is easy to find because the window w2 is triangular and hence linear. The modulation can be significantly reduced if the phase function in each frame is chosen so as to be equal at time . Naturally the chosen value is the phase determined by a higher level control structure implementing a model of time varying frequency and amplitude.

Noise components

With the FFT -l synthesis method, noise components can be precisely introduced in any narrow or wide frequency band and with any amplitude. Bands of STS's of wl windowed noise are added to the STS under construction. This is simple and inexpensive when the noise STFT is precomputed and stored in a table before the beginning of synthesis.

Implementation and Performance

The FFT -1 algorithm has been implemented using a signal processing vector library UDI [Depalle & Rodet, 1990] and in real time on the array processor of the SUN-Mercury Workstation [START_REF] Eckel | A SUN-Mercury Workstation[END_REF] and on the MIPS RISC processor of the SGI Indigo [START_REF] Freed | Tools for Rapid Prototyping of Music Sound Synthesis Algorithms and Control Strategies[END_REF]]. In terms of cost, one of the critical elements is the construction of a STS Sl. Each partial requires K iterations of the form:

aj . e i j. W[fj/SR-k] for k [ , ] = j-F(k-1,2), = j+F(k-1,2) with j = F(fj,SR.q)
where q is the oversampling factor For efficiency, W is oversampled and tabulated. Let Wj,k represent a real value from the table W.

The complex value aj . e i j is computed outside the loop K as aj . e i j = x + i y. The instructions in the loop are then: * read Wj,k from the next location in the table W * compute x*Wj,k and y*Wj,k * add x*Wj,k and y*Wj,k to the next locations of the complex STS Sl.

By careful coding, many of the performance enhancing features of modern processors [Hennessey & Patterson 90, Lee 88] may be used to efficiently implement this critical inner loop. The SGI Indigo implementation takes advantage of the ability of the R4000 to overlap the execution of integer address operations, floating point additions and multiplications, delayed writes and multiple operand fetches into cache lines. It is interesting that the table for the oversampled window W is small enough to fit into onchip data caches of modern processors. This is not the case for the larger sinusoid table required in standard oscillator based additive synthesis. The following graph compares the performance of additive synthesis by oscillator and by FFT -1 .

Control by Spectral Envelopes

Amplitude Spectral Envelope of a STS In common implementations of additive synthesis, fj[n] and aj[n] are obtained at each sample by linear interpolation of breakpoint functions of time which describe the evolution of fj and aj versus time. When the number of partials is large, control by the user of each individual breakpoint function becomes impractical. Another argument against such breakpoint functions follows. For voice and certain musical instruments, a source-filter model [Rodet et al. 87] captures most of the important behavior of the partials. In these models the amplitude of a component is a function of its frequency, i.e. the transfer function of the filter [Rodet et al. 86]. That is, the amplitude aj can be obtained automatically by evaluating a spectral function called the spectral envelope e(fj) at the frequency fj of the partial j. The amplitude variations induced by frequency variations may be very large [Rodet et al. 86]. In consequence, a breakpoint function of time cannot neglect these amplitude variations and therefor needs many breakpoints. Moreover, the amplitude of a partial is then not an intrinsic property of the sound independent of other characteristics such as the fundamental frequency. Encoding amplitude as breakpoint functions of time precludes modifications of fundamental frequency or vibrato.

On the other hand, a spectral envelope can be described as an analytical function of a few parameters, independent of the number of partials. It can vary with some of its parameters for effects such as transitions between formants and spectral tilt or spectral centroid changes known to be related to perceptual parameters such as loudness and brightness [Wessel 79] and vocal effort [ Sundberg 87]. Also, spectral envelopes are effective for the description of noise components. Spectral envelopes can be obtained automatically by different methods, for example, Linear Prediction analysis [Rodet&Depalle 86].

If the amplitudes and frequencies of the partials are already known from sinusoidal analysis, the Generalized Discrete Cepstral analysis [ Galas&Rodet 90,91] provides reliable envelopes, the smoothness of which can be adjusted according to an order parameter as in classical cepstral analysis. The previous arguments motivate the decision to represent amplitudes of partials by use of successive spectral envelopes defined at specific instants, for example the beginning and the end of the attack, sustain and decay of a note, etc.... [Rodet et al. 88]. Then at any instant the spectral envelope to be used is obtained by interpolation between two successive envelopes. These specific instants may be found automatically for the optimal reconstruction of the sound by linear interpolation between two successive envelopes [Montacié].

Frequencies and phases also can be described by similar generalized spectral envelopes. As an example, sustained sounds have partials with frequencies very close to harmonic values of the fundamental frequency. It is therefore computationally advantageous to record only the deviation from harmonicity as a function, called the frequency deviation envelope, of the harmonic number or of the harmonic frequency. Then, at the synthesis stage, it is easy to compute the frequency of a partial, retaining control of the amount of inharmonicity in the resulting sound. The same procedure can be applied for the phases of the partials.

Conclusion

The new method of additive synthesis by FFT -1 brings a solution to the three main difficulties of classical additive synthesis: efficiency, noise, and control.

Processing time (calculation cost) is an order of magnitude less than that required for oscillator-based additive synthesis. Apart from one inverse FFT -the cost of which is independent of the number of voices and of sinusoid and noise components -the calculation is reduced to the construction of the Short Term Spectrum. By optimization in time and frequency domains this construction may be reduced, for each sinusoid, to simple table lookups and multiplications of a small number of values. As in the case of oscillator-based additive synthesis, this procedure can build sinusoids with rapidly-varying amplitude and frequency, and with phase control if required.

Noise of arbitrary amplitude and bandwidth can be precisely and easily added to the short term spectrum. Automatic methods for analyzing sounds in terms of partials and noise can be easily applied to FFT -l synthesis.

Control of additive synthesis is made easier by use of spectral envelopes instead of the time-domain breakpoint functions traditionally used for additive synthesis. Properties of desired sounds are expressed in terms of features such as the amplitude envelope of partials, fundamental frequency, deviation from harmonicity, noise components, etc. These spectral envelopes need only be defined at a small number of specific times such as the beginning of an attack, the end of attack, the end of sustain, etc.... Then at any time, the synthesis algorithm uses envelopes interpolated between the defined envelopes. This procedure yields economical and efficient user control.

Several years of experimentation with the FFT -1 method demonstrate that the method provides good control over a wide range of sounds of high quality. Experience with implementations on affordable desktop workstations suggest that a low-cost real-time multi-timbral instrument based on FFT -l is within reach. It would have all the possibilities of present day synthesizers, plus many others such as the precise modifications of recorded sounds, speech and singing voice synthesis.

  [k] into the corresponding time-domain signal swl[n]. To better explain the method, the STFT analysis method [Nawab&Quatieri 1988] is reviewed in which a signal s[n] is first partitioned into successive overlapping frames sl[m]. Each frame is multiplied by a window signal w[m]. Note that with an appropriate choice of w and of the overlapping factor d (typically d=0.5), s[n] can be exactly reconstructed from the windowed frames swl[m] by the overlap-add method. The complex STS of each windowed frame is then computed by FFT, leading to a succession of spectra S0[k], Sl[k], S2[k], ... etc. The FFT -l method is simply the inverse process. First the spectra Sl[k] are constructed, their Inverse Fourier Transforms are computed to obtain the swl[m] and finally the swl[m] are windowed and added with overlap to create the time-domain signal s[n].

  [k] as explained earlier. Then the result of the inverse transform of Sl[k] is the time domain windowed frame wl[m].s[m] where s[m] is the desired output signal. Then a second window w2 is applied. It is defined by w2[m]=tr[m]/wl[m], where tr[m] is the triangular symmetric window. The result of this application is: w1[m].s[m].tr[m]/w1[m] = s[m] .tr[m],

Acknowledgments

The authors gratefully acknowledge the support of Gibson, Silicon Graphics and Zeta music. Alan Peevers provided the 3 dimensional analysis plot.


Gerhard Eckel

A Signal Editor for the IRCAM Musical Workstation

The SignalEditor is an extensible set of general purpose tools for viewing, analyzing, editing, and processing sampled signals on the IRCAM Musical Workstation (IMW). Its main intention is to provide the composer and the scientist with a sophisticated interface to handle digital audio signals naturally. This involves a high quality graphics interface, a flexible signal analysis system, an efficient sound file system, and a powerful signal processing environment. In this paper we discuss a prototype implementation of the spectrogram editing facilities, some aspects of the editor's extension language and the lowest layer of the sound file system.

Introduction

The SignalEditor is an application running on the NeXT computer system under NextStep and MACH. A subset of the SignalEditor, without graphics, runs on several UNIX platforms. The SignalEditor is designed with Interface Builder using a set of Objective-C classes built for signal management, processing, viewing and editing. These classes are supplied in the form of the SignalEditor Kit, which is well integrated into the NeXT Application Kit. Other applications of the IMW (like ANIMAL [1] or the Universal Recorder [START_REF] Smith | A Universal Recorder for the IRCAM Musical Workstation[END_REF]) will use the SignalEditor Kit directly or use the SignalEditor application via MACH messages. The SignalEditor comprises an extension language based on Scheme [START_REF]Revised3 Report on the Algorithmic Language Scheme[END_REF] which provides a generic interface to the facilities of the IMW's i860 coprocessors via CPOS and FTS [START_REF] Viara | A real-time operating system for computer music[END_REF]. This paper describes the ideas which led to the design of the SignalEditor and it discusses some features which have already been realized. A general survey of the SignalEditor's features will be given followed by the description of its extension language. In particular we will discuss a prototype implementation of the spectrogram editing feature that has been realized in the form of the program SpecDraw, which will be presented in a demonstration session of this conference.

The SignalEditor

The SignalEditor is a tool for viewing and editing signals in various graphical representations. These are time domain (e.g. waveform) and frequency domain (e.g. amplitude spectrum) representations as well as combinations of the two (e.g. spectrogram). The SignalEditor eases the auditive and analytical evaluation and comparison of signal segments. This is accomplished by efficient and flexible signal access both on the level of graphical representation and sound playback. Therefore it allows selection and management of signal segments and related information. Signal segments are defined in the time domain, the frequency domain, or as a region of arbitrary shape in a time-frequency representation (e.g. a formant region). Segments can be named and referred to by name in subsequent operations. A set of segment management operations allows to deal efficiently with the segmented sound material.

In all signal representations one can select, copy, cut, and paste signal segments in order to change the sequence of samples in the signal. Editing of sample magnitudes in the time domain (e.g. envelope editing) as well as filtering operations based on frequency domain representation (e.g. spectral envelope editing) are supported. Another aspect of the SignalEditor is its ability to extract features from signals using advanced signal analysis methods. The extracted features might guide segmentation, editing, or any other operation. Furthermore the SignalEditor supports a generic interface to signal processing facilities of the IMW. This allows different production styles to be realized in the same environment. The SignalEditor can be seen as the uniform interface for any signal used within the context of the IRCAM Musical Workstation.

The main intention of the design of the SignalEditor was to integrate known techniques of signal representation and editing into a tool which is flexible enough to allow new methods to be integrated easily. Several signal representations showing different aspects of a signal are visible simultaneously, allowing the observation of the consequences of a change made in one representation.

The Extension Language

In order to make the SignalEditor an extensible and flexible tool it needs to be programmable. This allows the user to program when he needs and wants to but does not oblige the use of a programming language to run the SignalEditor. That feature is well known from the text editor Emacs [START_REF] Stallman | GNU Emacs Manual[END_REF] which is one of the best examples of an apphcation that can be used as easily by a novice user as by an expert user. In addition, such kinds of applications allow smooth transition from the novice to the experienced user. We believe that this strategy can also be used with success for applications based on a graphics user interface like the SignalEditor.

Since the use of extension languages for non-trivial applications becomes more and morc common in software development the number of language implementations suitable for that purpose is increasing. The choice of the extension language for the SignalEditor was guided by three major constraints: the language used should be a standardized, interpreted, high level language; furthermore it should comprise a small but powerful set of concepts that allow different programming styles and it should be easily available in form of C implementations. The language Scheme fits the first two constraints ideally. After having compared different Scheme interpreters we found the very careful C implementation in the form of ELK 61 to be a good basis for the extensions language of the SignalEdilor. ELK allows the easy inclusion of Scheme primitives written in C via its dynamic loading mechanism This feature is used to add the functionalities needed by the SignalEditor to ELK. We will now briefly discuss three sets of primitives added so far: the Objective-C interface, the signal processing interface and the Yirtual file interface.

The Objective-C interface TELL allows access to all classes known by the SignalEditor. These are the classes defined in the various class libraries coming with the NeXT system and the classes which form the SignalEditor Kit. TELL defines a data type to handle class and object pointers and it supplies a function to send messages. This enables the extension language to intervene on all levels of the application. Thus TELL guarantees the desired flexibility of the extension language which has already been proven to be very useful for debugging.

The signal processing interface SPEX [START_REF] Eckel | SPEX 0.9, Scheme Signal Processing Extensions, Reference Manual[END_REF] is based on the UDI library [START_REF] Depalle | Interface for sound signal analysis and synthesis[END_REF] developed a IRCAM. UDI comprises a large number of signal processing routines and it runs on many different platforms. SPEX introduces data types to ELK which allow storage, I/O and processing of signals using UDI routines, which are made available in the form of Scheme primitives. SPEX currently runs on the NeXT, Sun3, and DEC3100 workstations. When used by the SignalEditor on the NeXT the SPEX functions will run on the i860 of the IMW via CPOS and FTS.

The virtual file interface VFILE is based on a library of C functions which constitute a virtual file scheme. These functions duplicate the functionality of the UNIX standard buffered input/output package (stdio). Additionally they allow one to insert, move, and delete data at arbitrary positions in the file. This feature, which has been implemented very efficiently, was desired for editing large sound files. A virtual file is represented on disk as a directory containing a chain of fragment files which make up the virtual file. The order of the fragment files is established by the lexicographical sequence of their names. The file names also encode other information like the byte offset and size of the fragment. This scheme appears to be the most robust in keeping virtual files consistent because standard UNIX file system features such as directories and hard links are used. Since real time access of virtual files can be guaranteed the design of the new sound file system will be based on this layer.

SpecDraw

SpecDraw is a prototype implementation of the spectrogram editing feature of the SignalEditor. It represents the signal in the form of a spectrogram in which arbitrary regions may be selected and used in further processing. Two modes of processing are currently available. Either the selected regions are erased from the signal or all other parts but the selected regions are erased. The final implementation will generalize this approach and will allow any desired operation on the selected regions. This kind of signal interface allows relating characteristics readable in the signal representation to their auditive appearance, which is one of the central tasks we are confronted with when editing and processing sound signals. It is essential that the signal representation is adapted to the charactenstics of auditive perception. Therefore the SignalEditor will support perception-relevant scales like the Bark scale and it will take into account certain phenomena of auditive perception such as the masking effect [START_REF] Eckel | Ein Modell der Mehrfachverdeckung für die Analyse musikalischer Schallsignale[END_REF].

The editing of time-frequency distribution based signal representations is the main novelty of the SignalEditor. SpecDraw serves mainly for validation of the ideas concerning this new type of interface. The program has been presented to a number of composers and scientists and the positive feedback it provoked brought many new ideas which will be taken into account in the final implementation of the SignalEditor.


Philippe Dérogis

René Causse

René Caussé

Computation and Modeling of The Sound Radiation of An Upright Piano Using Modal Formalism and Integral Equations

  L'archive ouverte pluridisciplinaire

Introduction

Modal parameters extraction

The response at the point r to a sinusoidal excitation at the point s can be expressed using the eigen values and the eigen vectors of equation ( 1) :

where :

Computation of the displacement resulting from any excitation

The displacement x(t) resulting from an excitation can be computed using the formula :

where :

Mesasurements and Results

Figure 1: Modes resulting from the curve-fiting of measurements

Sound Radiation

Sound radiation of a baffled plate

The sound pressure p(r) and the acoustic velocity resulting from the vibrations of a baffled plate moving with normal acceleration are given by :

Sound pressure radiated by the first mode of the soundboard Active Intensity

Normal Active Intensity

The active intensity is the power radiated per unit surface.

It is given by the formula :

It is intersting to calculate the z componant of the active intensity near the soundboard in order to know which regions produce the acoustic power, in particular :

If it is positive, the energy travels from the soundboard to the acoustic field If it is negative, the energy travels from the acoustic field to the sounboard.

Computation of the normal Active Intensity at 5 centimeters of the soundboard

  eigen values and the eigen vectors of equation(1)

An example of a loop of active intensity

Active intensity field in a plane orthogonal to the plate for the mode 1 : 122Hz

Radiation efficiency

Power radiated

The total power radiated by a source can be computed using :

Power radiated by the soundboard versus frequency for several driving point

Computation of the displacement of the soundboard using modal formalism Computation of the acoustic field Computation of the radiated power

Results

Radiated power for several points excited by a 1N force

Radiation efficiency

Radiation efficiency of the soundboard

Estimated radiation efficiency

Radiation efficiency of a soundboard having eigen values (frequency and damping) a factor 1.5 higher.

Conclusion

The first modes of the soundboard look like the ones of an isotropic suported plate The pressure radiated by the modes depends strongly of the position of the observer The frequencies of the first modes are well below the coincidence frequency of these modes There are loops of active Intesity The acoustic power depends on the location of the excitation point The radiation efficiency is about 15-20%


Xavier Rodet

Nonlinear Oscillator Models of Musical Instrument Excitation

Introduction

Many physical models of sustained musical instruments have been proposed (strings, brass, reeds, flutes and voice [ACROE 90]). Mathematically, these models are dynamic systems [Khalil 92] which can be described by autonomous retarded functional difference and differential equations such as: (1), x'(t)=f(x(t), x(t-T)), or (2), x(t)=f(h x(t-T)), where f is a nonlinear function, T some delay, h a convolution kernel and the convolution operator. As an example, the system x' = v and v' = av-bv 3 -kx has been proposed by Lord Rayleigh [Abraham 82] as a model of clarinet oscillation but we will see below that another model is now preferred.

A standard model of speech production [Ishizaka 72] uses a two-mass model of vocal cords consisting of two coupled Nonlinear Oscillators (NLO) representing a vibrating vocal cord. A one mass model can be described by a nonlinear system: x' = v, v' = g(x,u,v) and u' = (c+ax)[f(x,u,v)-h u], which has the form of the equations (1) and (2) above without the x(t-T) delayed term. But for strings, reed-woodwinds or brass, the delay term plays an essential role. In some clarinet models the instrument itself is represented by a delay line and the non-linear excitation is represented by a time-varying pressure-or velocity-controlled reflection coefficient. Similarly, a very accurate model of clarinet excitation has been designed at IRCAM. The reed is an NLO driven by mouth pressure and bore pressure. In violin models also (e.g. at CCRMA, ACROE and IRCAM) the string is set into oscillation by the bow and the combination is an NLO. We have also implemented a model for the lips of the trumpet player [Rodet in ACROE 90], where lips are represented by an NLO with two degrees of freedom, moved by pressure from the mouth and mouth piece.

The delay found in the previous instrumental models comes from the instrument itself which is relatively easy to measure or estimate and can be modeled rather accurately as a linear system, by using, for instance, a state-space representation [Depalle 92]. One of the other key points for music synthesis is modeling the excitation process. This is why we present here research on non linear oscillators coupled to passive linear systems as a general model of a large class of musical instruments [McIntyre 83].

A clarinet-like basic model

As an example, we examine the reed of a clarinet-like instrument coupled to the bore. Following [McIntyre 83], let us call qo and qi the outgoing and incoming pressure waves in the bore respectively, p the pressure in the player's mouth and z the characteristic impedance of the bore. The system can be described in a simplified way by the equations: , , where h(t-T)=r(t) is the reflection function of the bore. The most important assumption here is that the reed has no mass, leading to a memoryless nonlinearity F. In the case where this system has a unique solution, then: qo = (h qo(t-T)). This is a very simple model to explain the basic oscillatory behavior of the reed in a clarinet-like instrument (Figure 1). To better understand this behavior, [McIntyre 83] and [Magenza 85] note that if h(t) is simplified into a dirac dt (the sign inversion is included in ) then: qo(t)= (qo(t-T)), and similarly for qi. The signal value qo(t) depends only on the value at t-T. If qo(t)=Q0 is constant on [-T, 0] then it is constant on any interval [(n-1)T, nT] with a value: Qn= (Qn-1).

We now examine this iterated map as the basic model of a clarinet like instrument. If p=0, then qo should stay zero. Thus the origin O is a fixed point of the map. In order for the system to oscillate around O, as we expect a musical instrument to do, the slope s1 of a smooth map about O has to be less than -1 (Figure 2). In order for the signal not to grow to infinity, the slope of has to become greater than -1 at some distance from O. Let us first choose as two segments with slopes s1<-1 and s2>-1 for Q>0 and symmetric around O (Figure 2).

Roles of the slopes s1 and s2

It can easily be seen that |s1| controls the transient onset velocity, the greater |s1|, the faster the onset. We have here a clear control parameter for the onset behavior of our instrument. If h(t)=d, the signal is a square wave. If h(t) is a low pass kernel, then the signal is rounded. This can be controlled by |s2|: the closer |s2| to 1, the less high frequencies in qo. This can also be viewed as follows: in the square wave case the system uses only two points of the map and in the rounded case it uses more points spread more regularly on the map.

As a first result, we have found that two important characteristics of the sound, transient onset velocity and richness, are controlled by the slopes s1 and s2. We have been lead to adopt the s1-s2 map by control considerations of the basic clarinet-like instrument. Remarkably, it happens to be the same map as in the Chua's circuit, described for instance in [Matsumoto 85] where different behaviors of this system are examined.

If we want a continuous map, we may choose a simple cubic (x)= ax 3 +s1x (Figure 3), where a is determined again according to the slopes s1 in O and s2 at the point of abscissa x0 such that: -x0 = ax0 3 +s1x0.

Periodicity

For musical purposes we expect to have control of the period of the waveform. In the continuous case, [Chow 85] has studied some similar equations in the form: x(t) = f(h(t) x(t-T)). It is shown that under some fairly general conditions on the map f and the kernel h, the period 2 (corresponding to the first mode in a clarinet, i.e. to a period of 2T) is asymptotically stable. This means that we can expect to play and keep some steady tone from the instrument. It is also shown that, if f is symmetrical around O the signal x(t) has the symmetry x(t+T)=-x(t). Then the signal is composed of odd harmonics only. This is an essential characteristic of the clarinet sound. Comparison with other instruments such as trumpet where the nonlinearity is with memory, is under way. In some conditions, periods having durations which are integer fractions of 2T are also possible.

If the slope s2 is greater than 1, the fixed point of the map f 2 =fdeg.f becomes unstable. We observe period doubling and, for greater values, chaotic behavior. From the sound synthesis point of view, this is very interesting. Period doubling corresponds to subharmonics. In other cases, the signal sounds like noise added to the periodic tone of the instrument but with some relationship between partials and noise. Due to the lack of space, details are differed until the conference.

Conclusion

Using HTM [Freed 92], we have implemented nonlinear oscillators in real time on a SGI Indigo with a Motif-C ++ graphical-user interface allowing for easy experimentations with their properties and behaviors. We have shown how the excitation of musical instruments can be studied from the point of view of nonlinear oscillators. Among the important results, we look for precise conditions for oscillation, stability, good control parameters and good properties of the generated waveforms. Other basic models of instruments, brass [Rodet 92], voice, flute, and string are studied in the same way as the clarinet, and are


Gérard Assayag

Visual Programming in Music

PatchWork is a lisp-based visual programming environment for music. Propositions are made in this paper for the extension of PatchWork visual scheme to full object-oriented programming.

Introduction.

The PatchWork (Laurson 90, Malt 93) experience at IRCAM has proven the great interest shown by composers in visual programming. PatchWork has been distributed, through the IRCAM user's group, to over a hundred musicians and researchers in musicology. Many specialized libraries have been written at IRCAM and elsewhere adressing a wide variety of problems including new models for rhythm quantification (Agon 94), and musical constraints propagation schemes (Assayag 93). Composers that did not know a word in computer science have realized in PatchWork visual programs (patches) of an amazing complexity, due to PatchWork original mixture of visual functional programming and graphical data editing.

PatchWork runs on top of the Apple (now Digitool) Macintosh Common Lisp / CLOS (Common Lisp object System) environment (Steel 90). The visual scheme of PatchWork is purely functional, i.e. a graphical patch is an exact image of an embedded lisp function call where boxes are functions and links between boxes connect actual arguments to function parameters. Boxes may have a local state, which is in the simplest case the value computed by the function. This local state may be transformed by hand through a wide palette of editors (e.g. music notation editors).

We are now working on the extension of the PatchWork visual scheme to object oriented programming, i.e. we would like a graphical patch to be an image of a CLOS program. This paper will describe some propositions for visual extensions allowing the user to define classes, inheritance hierarchy, instances of classes, generic or polymorphic patches. In the last part of the paper, propositions are made for improving graphical control in higher level organization of musical substructures, using a new editor above the patch level.

The Icon interface.

A class is symbolized by an icon (fig. 1). Opening a class lets you visualize the class internal structure inside a new window. Icons in that window symbolize the classes of the instance variables (slots). New slots can be added by dragging and dropping icons from the current reservoir of classes. Icons can be opened in turn, giving access to the content of their associated class, in view-only mode.

figure 1

Class hierarchies are designed by drawing links between icons (fig. 2). When opening a class icon that has been included in a hierarchy, the class window shows the parent classes instance variables in different layers, stacked from bottom to top in inheritance order. Only the bottom layer, i.e. slots belonging to the opened class, is editable. Instances of a class are derived graphically from the class icon, by a mouse operation (e.g. commad-drag). A new icon is then created for the instance. Opening an instance results in a new window whose internal structure is isomorphic to the window associated with the instance's class, except that icons here symbolize values or instances associated to slots rather than their class. Opening a slot's icon either gives access to it's substructure or to a specialized editor (e.g. music notation editor). Icons representing instances, just as constant boxes, can be provided as inputs to a PatchWork box.

The generic function interface.

Currently, PatchWork boxes are functions. There are two cases : either a lisp function (interpreted or compiled) is linked to the box, or a set of other interconnected boxes, implementing the body of a function, is associated with the box (in that case, it is called an abstraction). In order to be able to handle generic or polymorphic functions, i.e. functions that accept arguments of different classes then switch automatically to appropriate methods, the patch visual concept has to be extended. We shall call generic boxes boxes that dispatch on different methods depending on the type of their inputs, and method boxes boxes that materialize the methods themselves.

In the case where a generic lisp function and its methods are associated with a generic box, there is no particular interface problem. This is due to the fact that generic functions are smoothly integrated in the Common Lisp -CLOS environment and are considered externally as normal functions. Thus the usual mechanism will hold.

In the case where the generic box and its methods are programmed visually, we refer to the abstraction visual scheme. When a box is an abstraction, opening it results in the display of a patch window that contains a patch defining the abstraction. It is a normal patch except that it contains two kinds of special boxes : absin (abstraction in) and absout (abstraction out). absin boxes are the formal parameters of the abstraction. You will get as many rectangular inputs in the abstraction box than there are absin boxes in the abstraction patch. absout is the output box : the value it computes is the value of the abstraction itself.

In the example in fig. 3, the "transform" box is a generic box. It performs a musical transformation with two inputs : a musical structure on the left side, and the functional transformation on the right side. Suppose "transform" is defined so that the musical structure can be specified either as a chord object (the output of a Patchwork box "chord") or a BPF object (the output of a BPF --break point function --box), and the transformation either as a lisp-function name ("MyFunction") or a patch. We introduce a supplementary level of box integration as can be seen in the second part of the picture. The boxes appearing in a new window when "transform" is opened are method boxes defining the different methods for transform. They are regular abstractions, which means they have associated patch windows containing a patch with absin and absout boxes. For each of these method boxes, one must specify the formal parameters classes. This is achieved by dragging class-icons into the window and connecting them to the inputs.

The top-left method in the sample window is defined to be called when the first argument to "transform" is an object of class chord and the second one an object of class patch. The top-right method has chord and lisp-function as input classes. The bottom-right is defined for the class editor and the class everything (the class T in CLOS). The bottom-left method is defined for everything on both sides. This last method is labeled after, which, in the CLOS terminology means it should be called after every other eligible method has been called. before and around methods could have been defined as well (Steele 90).

figure 3

Suppose "transform" is evaluated with the BPF box and "MyFunction" connected to it. As there is no specific method defined for the BPF class, and as both chord and BPF inherit from the editor super-class (fig. 4), the bottom-right method will be called, then the after method.

If a method wants to call the next available method in the hierarchy, it uses the call-next-m box (fig. 5). If this box appears inside the patch associated with the top-left method, it will act as an abstraction box for the patch associated with the bottom-right method, as chord inherits from editor and patch inherits from everything.

To end with, if a patch wants to access to a particular slot in an object that flows throught it, it may use a "slot" box. This box can be opened in order to show the internal structure of the associated class, if this may be determined at edit-time. Then the user selects a slot, or a set of slots, with the mouse (fig. 6).

As one can see, the patch visual concept can easily be extended to encompass many interesting features of CLOS. Above the patches.

Using PatchWork, one can easily build a musical process modelled by a graphical patch, then connect the output to a musical editor module. Musical sub-structures are built that way. But it is still unpractical to combine sub-structures into higher level ones using concatenation and superposition . A new editor under design will let the musician create, display and manipulate graphical forms -currently simple rectangleson a surface where time is one of the dimensions (fig. 7).

The idea is to associate a patch to each of these forms (1). The patch is responsible for computing the music sequence that will give a meaning to the form, through a special output box identical to the absout box in an abstraction. Thus, by controlling the spatial distribution of forms on the surface, the user controls concatenation and superposition of computed musical entities. As in Animal (Lindemann 90), the vertical and horizontal extents of the rectangles can be mapped to parameters into the associated patch. Obviously, the horizontal dimension will be interpreted as a time factor and eventually converted into a tempo in the case when it means something. Other graphical attributes like texture or curves drawn inside the rectangles will be mapped by the user to arbitrary control parameters in the patch. Selecting and double-clicking one or more forms will trigger the evaluation of the patches, combine the partial results, and open a music editor in order to display the sequence (fig. 8).

In order to set up relations between sub-stuctures, e.g. connect several of them into a single voice, links between forms have to be handled (2). These links are graphical objects associated to "glue" patches that tell how to perform the connexion. As the forms and their associated patches can be belong to different subclasses, glue patches have to be generic patches, with different methods for different classes of arguments. The user can then override the standard behaviour of these links by designing gaphically his own methods.

Another kind of link is a simple binary constraint that relate a variable in a patch with a variable in another patch, through a transformation function (3). This function is itself implemented as a patch. By using this feature, changing the value of a variable in a musical substructure may cause significant transformations in other parts of the score.

  figure 4

  figure 7


René Caussé

J F Bresciani

O Warusfel

Radiation of Musical Instruments and Control of Reproduction with Loudspeakers

Commonly used types of loudspeaker systems have directional patterns that bear no relationship to those of the sound sources which they purport to simulate.

The purpose of this project is to derive some parameters linked to the radiation and relevant to our perception.

To address this problem, a series of three separate experiments have been conducted: collect data on sound power spectra and directional characteristics of musical instruments to be simulated. build of a multiple drivers reproduction system with signal processing control. investigate the physical and psychophysical properties of this sound source when the directional programming is varied. The influence of the coupling of the sound source and the room on the perception of the simulated sound sources is also examined.

Introduction:

This project is motivated by the observation that commonly used types of loudspeaker systems have directional patterns that bear no relationship to those of the sound sources which they purport to simulate. This lack of relationship is particularly troublesome when natural sounds are mixed with electro-acoustic sounds (recorded or amplified sounds), "...because the loudspeaker anonymises the actual source..."; "...the instruments, we might say, are sent to the rolling mill of amplification and lose part of their individuality" [START_REF] Boulez | L'acoustique et la musique contemporaine[END_REF].

Similar effects attract our attention when sounds from synthesis are reproduce with loudspeakers, specially sounds obtained from Physical Model Synthesis [START_REF] Adrien | The Missing link, Modal Synthesis[END_REF].

In contrast with traditional synthesis method, this technique provides multi-channel physically significant sound outputs (radiating holes of the clarinet for example) and for which a simple stereo mixing is irrelevant.

Our present psychophysical knowledges make it clear that the human observer is sensitive to such deviations, however it is still not possible to characterize the radiation of sound sources, pertinent to the perception. The purpose of this study is to derive some parameters linked to the radiation and relevant to our perception. These parameters will be also function of the acoustic surrounding and of the distance between the source and the listener (differents for the listener at the back of the hall and for the conductor on the stage).

Preliminary studies:

Preliminary studies done by one of the author [START_REF] Warusfel | rapport interne IRCAM[END_REF] have permit to verify the importance of the objective differences of directivity of sources and their noticeable influence on the perception, as well as the problems related to the positioning of the microphone and the informations carry in sound recording. They have shown that an appreciable amelioration is obtain by adding an omnidirectional diffusing system to the principal loudspeaker, which excite the reverberant field of the hall as would do the real instrument, provide that a proper input signal is fed to this complementary device. At this point this work is articulated around four axes of studies in the domain of physical characterization and modelisation of radiation of instruments, perception, sound recording and control of the electroacoustic system of reproduction.

The auditory image of a source perceived by the listener is conditioned by the acoustic surrounding, therefore the approach chosen is related to the knowledge acquired in the domain of perception in room acoustic [START_REF] Warusfel | Etude des paramètres liés à la prise de son pour les applications d'acoustique virtuelle[END_REF]. The psycho-acoustic studies have shown that our perception is structured by a finite number of factors related to room acoustic quality. In the hypothesis of a listener located far from the source and that the room and the source are homogeneous enough; it is possible to consider the level of first reflections as negligible compare to the level of the reverberation field; then the control of the latter and the direct sound provides a good approximation to reproduce the source.

Measurement of the power spectra radiated:

A good knowledge of the power spectra of the instrument is necessary in order to obtain a good reproduction by the artificial source. The method consists in the measurement of the sound power spectra on twenty points around the source [5], then, after the calculation of the average power, the level (versus frequency) compared with this average and for every microphone is traced.

On figure 1, four measurements are shown for two instruments with discrete sources ( 1a for the trombone, 1b for the clarinet holes closed and 1c holes open) and an instrumentbehaving like a wide-spread source (for exemple the violin; 1d).

For the clarinet and the trombone it is possible to approximate the measured power spectra by one microphone position among the twenty positions. This is not the case for the violin where strong irregularities are present related to favoured directions. Addition of a simple form like a mute in the bell of the trombone lead to strong irregularities also [START_REF] Caussé | Sourdine et timbre des instruments à vent (Cuivres) dans Le timbre métaphore pour la composition[END_REF].

Positionning of the microphones:

To minimize the influence of the room on the sound recording of the source (and also the signals from adjoining instruments) and to avoid the electroacoustic feedback (Larsen effect) it is necessary to place the microphone close to the instrument, which can also cause interferences. Then we see that the positionning of the microphones require a work of optimisation which is necessarely related to an elaborate knowledge of the radiation of the instruments based on the knowledge of their physical fonctionning. This next step consists first of a measurement of directivity patterns (sinusoïdal excitation and analysis in a cutting plane of the instrument) and second of a modelisation of the source by a spherical harmonics series representation [START_REF] Weinreich | Method for measuring acoustic radiation fields[END_REF], [START_REF] Filippi | Noise source modeling and identification[END_REF].

Reproduction system:

To mimic certain types of radiation pattern as a function of frequency, we have build an array of twelve independently programmable loudspeakers mounted on a polyhedral structure (dodecahedron). This reproduction system, equiped with digital signal processing both for piloting and filtering the different channels, constitute the tool to begin this study.

The physical characteristics have been measured with the help of the directivity index, which is a relevant criterion for a reproduction system based on the balance between latter and direct sound. At mid frequencies the directivity index varies between 0 and 10 dB [START_REF] Chemla | Etude et réalisation d'une source sonore à directivité variable[END_REF].

The last step consist on the investigation of the physical and psychophysical properties of this sound source when the directional programming is varied. Particularly is to evaluate the importance from the perception point of view of remarkable behaviour of a vibrating structure like the violin (transition between a monopolar radiation and a dipolar radiation observed on this instrument).

Plans for the future:

continuation of collecting data and modelisation for different family of instruments. improvement of the electroacoustic source related to the psychoacoustic tests. Integration of the necessary fonctions to simulate the main effects to which the ear is sensitive. study of the case where the first refections are not negligible compare to the level of reverberation process.