Keywords: Automatic Music Transcription, Statistical modeling and learning, Musical Acoustics knowledge, Non-eurogenetic music, Computational Ethnomusicology, Audio Signal Processing Transcription Automatique de Musique, Modélisation acoustique et statistique, Acoustique Musicale, Musique non-eurogénétique, Analyse musicologique informatisée Motivations, Contributions & Plan Motivations Musical Production Signal Acquisition Signal Analysis

Mes premiers mots de remerciement iront spontanément à Olivier, mentor de Master puis directeur de thèse, et j'espère collaborateur et ami pour de nombreuses années à venir. Je tiens ensuite à remercier Marc Chemillier, brillant co-directeur de ma thèse, qui a été à l'origine de ce projet de recherche passionant, amenant ainsi dans ma vie la cithare marovany et la musique malgache.

Ce projet de thèse a fait l'objet de nombreuses collaborations fructueuses et excitantes. Avec tout d'abord la Team : GuiGui, la Juju et Wang, merci à vous les gars pour votre boulot sur ce projet. J'espère pouvoir continuer à travailler avec vous dans les années à venir. L'équipe des ingés-sons "métalleux", menée par John Mary. Gregory Nuel, redoutable professeur de statistiques, que j'ai eu de la chance de rencontrer sur mes derniers mois de thèse. Toute l'équipe du LAM, notamment Laurent Quartier, pour son énorme travail à mes côtés sur les capteurs et le bricolage d'instruments, et bien sûr Hugues Genevois, pour m'avoir accueilli dans son labo et souvent conseillé sur l'aspect capteurs de ma thèse.

Il me faut maintenant remercier les nombreux musiciens que j'ai eu la chance de rencontrer durant ce projet. Dont l'incroyable "Papa" François Essindi alias Abakuya, pour la foi et la force qu'il porte dans des causes difficiles. Kilema, pour ta joie de vivre et l'enthousiasme que tu as mis systématiquement dans nos quelques séances de travail. Charles Kely, pour t'être rendu disponible pour mon projet, et tous les autres.

Enfin, je tiens à remercier mes proches pour leur soutien journalier. Mes parents, pour m'avoir supporté durant ces trois années. Caroline, pour son aide précieuse dans la fastidieuse tâche de recherche et de gestion bibliographique.

oun), the N'Goni (Mali). These systems use multiple string-dependent sensors to retrieve discriminatingly some physical features of their vibrations. For the AMT task, such a system has an obvious advantage in this application, as it allows breaking down a polyphonic musical signal into the sum of monophonic signals respective to each string. Then, we come back to a monophonic transcription problem, which is considered as practically solved. For sake of flexibility and robustness, various sensor types (optical, piezoelectric and electromagnetic) have been comparatively tested. After experimentation, piezoelectric sensors, although quite invasive, prove to provide the best signal-to-noise ratio and multichannel separability. The development of this technology has allowed the constitution of a new sound dataset dedicated to AMT evaluation for plucked-string instrument repertoires. We gathered in these datasets audio recordings, MIDI-like transcripts and sound samples over the instrument pitch ranges. We also performed field recordings in Madagascar with local musicians during two missions (in July 2013 and 2014), using our multi-sensor retrieval systems. Such systems were also explored towards applications of human-machine interaction (through the project ImproteK) and musical creativity (through "MIDIfication" of traditional acoustic instruments).

Our second axis of research tackles the AMT task on audio recordings with more fundamental investigations on the use of prior knowledge in transcription performance, in regards to different plucked-string instrument repertoires. AMT is often divided into different processing stages, generally a multi-pitch estimation stage, followed by note segmentation and post-processing stages. In this PhD thesis, we mainly build our AMT framework from two methods, namely Probabilistic Latent Component Analysis (PLCA) for multi-pitch estimation and Hidden Markov Models (HMMs) for note segmentation and sequential post-processing. PLCA belongs to a spectrogram-factorization class of methods which is based on the modeling of a signal as a sum of basic elements. HMMs are a ubiquitous statistical tool to model time series data. We then develop different configurations of these methods to provide a powerful probabilistic framework covering the time-frequency domain on different time-scales, in which we develop original integration methods of prior knowledge. Timbre prior knowledge class is used to constrain generic signal models with acoustics-based information. A pitch-dependent sparsity prior has then been developed by modifying the EM update rules of PLCA, which is informed by the phenomenon of sympathetic resonances characterized acoustically. The use of pitch-wise multi-templates corresponding to different playing modes (e.g. dynamics, plucking techniques) have also been investigated, as well as the characterization of pitch activation profiles, which can be informed by specific temporal envelop modulations (e.g. intermodulation). For its part, musicological knowledge concerns more temporal musical structure and can be integrated conveniently into the HMM framework to build informed relations between frame-wise estimations. We develop original first-and second-order HMMs to model musicological polyphonic harmonic transitions between note mixtures, as well as higher-order HMMs including note duration modeling applied to note segmentation. This research axis has first allowed achieving successful transcription enhancements in the marovany repertoires, by optimizing selectively the integration of prior knowledge. As an illustrative result, the traditional repertoire of the marovany benefits mostly from timbre-related prior knowledge, namely spectral inharmonicity and energy modulation, intermodulation and sympathetic resonances. These features have been highlighted by our acoustic characterization preceding the task of AMT, used to guide the design of the repertoire-specific priors. Also, we provide a complete framework to have a better understanding about the issues arising from the explicit inclusion of specific prior knowledge in specific instrument repertoires, and to investigate whether it is really valuable when targeting tasks such as AMT. In that Développement d'un système de transcription automatique de musique dédié aux répertoires de la cithare marovany Résumé L'ethnomusicologie est l'étude de la musique en mettant l'accent sur les aspects culturels, sociaux, matérielles, cognitives et/ou biologiques. Ce sujet de thèse, motivé par Pr. Marc Chemillier, ethnomusicologue au laboratoire CAMS-EHESS, traite du développement d'un système automatique de transcription dédié aux répertoires de musique de la cithare marovany de Madagascar. Ces répertoires sont transmis oralement, résultant d'un processus de mémorisation/transformation de motifs musicaux de base. Ces motifs sont un patrimoine culturel important du pays, et évoluent en permanence sous l'influence d'autres pratiques et genres musicaux. Les études ethnomusicologiques actuelles visent à comprendre l'évolution du répertoire traditionnel, et de préserver ce patrimoine. Pour servir cette cause, notre travail consiste à fournir des outils informatiques d'analyse musicale pour organiser et structurer des enregistrements audio de cet instrument.

La transcription automatique de musique consiste à estimer les notes d'un enregistrement à travers les trois attributs : temps de début, hauteur et durée de note. Notre travail sur cette thématique repose sur l'incorporation de connaissances musicales a priori dans les systèmes informatiques. Une première étape de cette thèse fût donc de générer cette connaissance et de la formaliser en vue de cette incorporation. Cette connaissance explorer les caractéristiques multi-modales du signal musical, incluant le timbre, le langage musical et les techniques de jeu. La recherche effectée dans cette thèse se distingue en deux axes : un premier plus appliqué, consistant à développer un système de transcription de musique dédié à la marovany, et un second plus fondamental, consistant à fournir une analyse plus approfondie des contributions de la connaissance dans la transcription automatique de musique.

Notre premier axe de recherche requiert une précision de transcription très bonne (c.a.d une F-measure supérieure à 95 % avec des tolérances d'erreur standardes) pour faire office de supports analytiques dans des études musicologiques. Pour cela, nous utilisons une technologie de captation multicanale appliquée aux instruments à cordes pincées. Les systèmes développés à partir de cette technologie utilisent un capteur par corde, permettant de décomposer un signal polyphonique en une somme de signaux monophoniques respectifs à chaque corde, ce qui simplifie grandement la tâche de transcription. Différents types de capteurs (optiques, piézoélectriques, électromagnétiques) ont été testés. Après expérimentation, les capteurs piézoélectriques, bien qu'invasifs, se sont avérés avoir les meilleurs rapport signal-sur-bruit et séparabilité inter-capteurs. Cette technologie a aussi permis le développement d'une base de donnée dite "ground truth" (vérité de terrain), indispensable pour l'évaluation quantitative des systèmes de transcription de musique.

Notre second axe de recherche propose des investigations plus approfondies concernant l'incorporation de connaissance a priori dans les systèmes automatiques de transcription de musique. Deux méthodes statistiques ont été utilisées comme socle théorique, à savoir le PLCA (Probabilistic Latent Component Analysis) pour l'estimation multi-pitch et le HMM (Hidden Markov Models) pour le post-traitement des estimations et la segmentation de notes. La méthode PLCA appartient à la famille des méthodes de factorisation non-négative de spectrogrammes, basé sur une modélisation du signal musical comme une somme de noyaux harmoniques, dites bases spectrales. HMM permet une modélisation probabiliste du processus génératif de séries temporelles. Nous avons développé différentes extensions de ces méthodes de base, pour constuire un cadre probabiliste général couvrant les domaines fréquentielles et temporelles à différentes échelles, dans lequel nous avons pu incorporer différentes composantes de connaissance a prior. Les composantes de connaissance sur le timbre sont utilisées pour contraindre des modèles de signaux en paramétrant leurs paramètres acoustiques. Un a priori de parcimonie spécifique à chaque base spectrale a été défini en modifiant les règles d'update de l'algorithme EM, et qui est informé par de l'information sur le phénomène de résonance par sympathie. L'utilisation de templates multiples spécifique à chaque base spectrale correspondant à différents modes de jeu (par exemple, dynamique de jeu, techniques de pincement) ont été développés. Nous avons aussi proposé une caractérisation acoustique des profils d'activation temporels, basé sur la connaissance acoustique de l'enveloppe temporelle des notes. Les composantes de connaissance sur le langage musical sont utilisées pour modéliser la structure temporelle d'une pièce musicale, et les transitions harmoniques entre aggrégats de notes, par différentes méthodes basées sur des HMM. Cet axe de recherche a permis d'accomplir des gains en transcription significatifs sur des répertoires de marovany, en optimisant sélectivement les composantes de connaissance. Un résultat illustratif est que le répertoire de Velonjoro bénéficie principalement des composantes timbrales, telles que l'inharmonicité spectrale, la modulation énergétique de l'enveloppe, l'intermodulation et la résonance par sympathie. Nous avons aussi développé un cadre d'analyse plus général pour la compréhension de l'impact des connaissances a priori dans les systèmes de transcription automatique de musique, en relation avec différents répertoires et instruments de musique.

Notations

All notations of terms are listed by order of appearance in the thesis. [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF] RWC Real World Computing [START_REF] Goto | Rwc music database: Popular, classical, and jazz music databases[END_REF] List of method-specific variables MCSS parameters x(t) desired sensor signal

List of acronyms

HMM parameters

S i state i, S i ∈ S, S = {S 1 , S 2 , ..., S Ns }, with N s the number of states q t state at frame t, q ∈ S Introduction Background

Musical signal

As a starting point, let's consider the musical chain production illustrated in figure 1. Musical Knowledge (Step 1) conditions any musical production. Musical signals have very rich temporal and spectral structures, and it is natural to think of them as being organized in a hierarchical way. At the lowest level of this organization, two universal processes in temporal organization of auditory sequences have been identified [START_REF] Drake | Psychological processes involved in the temporal organization of complex auditory sequences: Universal and acquired processes[END_REF] : (1) segmentation sequence into groups of events; and ( 2) the extraction of an underlying pulse. The first, based on changes in pitch duration and salience, is already present in early infancy [START_REF] Krumhansl | ), chap. A key-finding algorithm based on tonal hierarchies[END_REF]. The second involves the extraction of temporal regularities and likewise appears early in infants [START_REF] Baruch | Tempo discrimination in infants[END_REF], and is assumed to be culture transcending [START_REF] Carterette | The Psychology of Music[END_REF]. At the highest "symbolic" level of this organization, we have "prescribed rules" for music execution, which may be the score of a piece, as intended by a composer, or more generally standard codes characterizing musical practices of orally transmitted repertoires. Then, on the basis of these rules, the performers add their interpretation to music (Step 2), and render the score into a collection of "control signals", which are both characteristics of their instrument timbre and playing style. Let's now move towards the "signal" aspect of music, which should first be captured by some recording device and digitalized (Step 3). For example, most music recordings from CDs are recorded with a microphone, and digitalized using the PCM sampling method, with a sampling frequency F s of 44.1 kHz and 16-bit resolution. Musical signal can then be observed for analysis through specific computational representation methods (Step 4), the most popular one being the FFTbased time-frequency plane called spectrogram, where time flows from left to right and different F 0 are arranged in an ascending order on the vertical axis.

As illustrated in figure 2, spectrograms allow for a direct observation of the hierarchical aspect of music. In the time domain, tempo and beat specify the range of likely note transition times, drawing abrupt step-like vertical lines in the spectrogram at the time locations of these transitions. In the frequency domain, two levels of structure can be considered. First, each note is composed of a fundamental frequency (related to the pitch of the note), and partials whose relative amplitudes determine locally the timbre of the note (which is in reality a complex non-linear time-domain system [START_REF] Fletcher | The Physics of Musical Instruments[END_REF]). The frequencies of the partials are approximately integer multiples of the fundamental frequency, although this clearly does not apply for instruments such as bells and tuned percussion. Second, as audio signals are both additive and oscillatory (i.e. musical objects superimpose and not conceal each other), several notes played at the same time due to polyphony will merge their respective spectral structures. Eventually, when performing musical analysis, a multiple-F 0 estimator produces horizontal lines which indicate the probabilities of different notes to be active as a function of time. It is visually clear from the spectra of a note that it will be quite easy to estimate the pitch from singlenote data that is well segmented in time (so that there is not significant overlap between more than one separate musical note within any single segment). Metrical analysis, in turn, produces a framework of vertical "grid lines" which can be used to segment the note activation curves into discrete note events and to quantize their timing.

Low-level musical parameters & Piano-roll representation

Low-level representation

In describing music, we are usually interested in an abstract representation synthesizing music information, which abstracts us away from the signal details. A common approach for this is to characterize the notes played in a musical piece by the four basic parameters of Onset Location, Pitch, Duration and Loudness, through an operation called low-level transcription, and represent them graphically into a time-pitch plane. Such low-level information is sufficient to be compiled into a MIDI 1 file. In this format standard, the time-pitch plane is called a piano-roll (an illustrative example is given in figure 3), in which continuous physical values of note parameters are discretized into different scales, as detailed in table 1. Such such discrete scales are of course way too restrictive for general music representation, which for example does not follow the assumption of equal temperament tuning. However, the format of MIDI files can still be used with physical continuous values on Onset Location and Duration, and extensions of the MIDI standard exist with finer scales for pitch (e.g. the MIDI tuning standard [START_REF] Midiwebpage | Midi tuning standard[END_REF]). Furthermore, MIDI files also present the precious advantage of being readable and editable on any audio sequencer and score edition program. For these reasons, this format will be adopted to format our transcription results. We now describe in more details these four note parameters, both from the points of view of psychoacoustics and signal processing.

Note parameters Discrete MIDI scale Onset location

Integer multiple of T a Pitch From A 0 (27 Hz) to C 8 (4186 Hz) Duration

Integer multiple of T a Velocity Arbitrary scale of 127 levels Table 1 -The discrete MIDI scales of the four different note parameters. For onset location and duration, we defined the Tatum T a (time quantum) as the smallest metrical unit [START_REF] Klapuri | Analysis of the meter of acoustic musical signals[END_REF]. This notion refers to the shortest durational values in music that are still more than incidentally encountered. For pitch, this discrete scale corresponds to the semi-tone scale using an equal temperament tuning.

Onset

Perceptually, human ears are able to distinguish between two onsets as close as 10 ms apart [START_REF] Moore | An Introduction to the Psychology of Hearing[END_REF]. This acuity make them play an important role in our perception of music. Studies showed that onsets play a pivotal role in the perception of timbre, as it is much more difficult to recognize the timbre of tones with removed onsets [START_REF] Handel | Hearing[END_REF][START_REF] Martin | Sound-source recognition: A theory and computational model[END_REF]. Additionally, music cognition experiments have shown that features related to the beginning of music notes can help humans to discriminate different instrument notes. There is also some evidence that the onset coding plays an important role in direction finding through representation of interaural time and level differences [START_REF] Rouiller | The Central Auditory System[END_REF]. And more globally, onsets also make it easier to detect new information in music; we can detect tones with pronounced onsets well before we can determine their pitch [START_REF] Newton | A neurally inspired musical instrument classification system based upon the sound onset[END_REF]. At a larger scale, onset succession build the musical measure pulse, which is usually related to the harmonic change rate.

As a signal parameter to be analysed, an onset is usually defined as the exact time a note or instrument starts sounding after being played. However, this timing is hard to determine, and thus it is impossible to annotate the real onset timing in complex audio recordings with multiple instruments, voices, and effects. Thus, the most commonly used method for onset annotation is marking the earliest time point at which a sound is audible by humans. This instant cannot be defined in pure terms (e.g., minimum increase of volume or sound pressure), but is a rather complex mixture of various factors.

Pitch

Perceptually, perceived pitch is a complex function of the fundamental and all partials. Only a few cycles are needed to identify a pitched note [START_REF] Robinson | The duration required to identify the instrument, the octave, or the pitch chroma of a musical note[END_REF]. Actually, the human auditory system tries to assign a pitch to almost all kinds of acoustic signals [START_REF] Meddis | Virtual pitch and phase sensitivity of a computer model of the auditory periphery. i: Pitch identification[END_REF].

As a signal parameter, we define the concept of fundamental frequency, labelled F 0 , which has to be limited to periodic or nearly periodic sounds.

Duration

Duration in music refers to how long or short notes are. Durations, and their beginnings and endings, can then be described as long, short, or taking a specific amount of time. A tone may be sustained for varying lengths of time. It is often cited as one of the fundamental aspects of music, encompassing rhythm, form, and even pitch. Durational patterns are the foreground details projected against a background metric structure, which includes meter, tempo, and all rhythmic aspects which produce temporal regularity or structure. Duration patterns may be divided into rhythmic units and rhythmic gestures.

As a signal parameter, the definition of a note offset has been a long-time ill-posed problem. It is often stated that for "decay instruments", the offset time is only important in a limited period of time, because after some point the sound energy will be below the threshold of hearing [START_REF] Zwicker | Psychoacoustics: facts and models[END_REF], even if from the musician point of view the note is still playing.

Amplitude

Perceptually, the note amplitude is correlated to the sensation of loudness. This perception is strongly non-linear and frequency-dependent.

As a signal parameter, the strong non-linear aspect of intensity perception and the absence of a precise normalization in MIDI make this parameter very hard to define. Adhoc relative measures, such as the root mean square value (abbreviated RMS or rms, and defined as the statistical measure defined as the square root of the mean of the squares of a sample), are generally proposed.

Automatic Music Transcription

In this section, we propose a global overview of the computational task of Automatic Music Transcription (AMT), along with more specific issues related to this research field.

Background

Due to the widespread use of the Information Technologies in the distribution and consumption of music, the topic of automatic description/annotation of audio recordings has become a research topic with many practical applications. This research is being carried out within the field that is known as Music Information Retrieval (MIR). This research field is part of a larger research area of multimedia information retrieval. Researchers working in this area focus on retrieving information from different types of media content: images, video, and sounds. Although these types of content differ from each other, separate disciplines of multimedia information retrieval share techniques like pattern recognition and learning techniques. This research field was born in the 80's, and initially focused on computer vision [START_REF] Lew | Content-based multimedia information retrieval: state of the art and challenges[END_REF]. The first research works on audio signal analysis started with automatic speech recognition and discriminating music from speech content [START_REF] Typke | A survey of music information retrieval systems[END_REF].

MIR is the interdisciplinary science (bridging musicology, signal processing, psychology) of retrieving any meaningful information from complex audio, which may then be used in a variety of user scenarios such as searching and organizing music collections. Some of the problems that the MIR community attempts to solve include classification and organization of music, recommendation systems and everything up to and including complex analysis of large musical databases by musical experts. Such problems are dealt with in disciplines called [START_REF] Mirex | Music information retrieval evaluation exchange (mirex)[END_REF]: audio identification, beat detection, prominent melody extraction, genre identification, cover song detection, or query by humming. Many of these problems have very tangible commercial premise, but most are related to the simple desire to understand basically how music functions by utilizing large databases and the power of computer processing. As in definitive, by increasing the quantity and quality of what the computer can "listen to", we can develop more effective tools for indexing and manipulating large audio collections as well as improve musician-computer interactions.

Moving now to the task of AMT, [START_REF] Piszczalski | Automatic music transcription[END_REF] proposed the first monophonic music transcription system which is limited to certain types of instruments with strong fundamental frequencies. This frequency domain approach simply chooses the most pronounced spectral peak as the fundamental frequency. In the same year, [START_REF] Moorer | On the transcription of musical sound by computer[END_REF] proposed the first polyphonic music transcription system, focusing on duets with a limitation to two monophonic instruments playing at the same time.

Goal & Applications

AMT is the process according to which a structured symbolic representation is inferred from a musical signal. This process can be seen as reverse-engineering the source code of a music signal (Klapuri, 2004a). As illustrated in figure 1, the process of AMT can be thought as the one closing the loop of the musical chain production. Conventionally, such symbolic representation generally results from highly time-consuming manual works, which also require a certain degree of musical education. Also, information retrieval for music transcription can be classed into two levels (Klapuri, 2004a) : the low-level (with the four parameters defined above), and the high-level (tonality, instrument recognition), which asks for more global and complex notions. In a traditional sense, transcribing a piece of music implies a number of these high-level tasks, such as estimating the tempo, metre and key of each section of the piece, identifying and labeling ornamentations and timbre information, recognising the instruments being played, and segregating "voices" according to the instrument that played them and to their function, i.e. melody, accompaniment, etc. However, performing such a high-level operation is totally out-of-reach of today AMT technology [START_REF] Benetos | Automatic music transcription: Challenges and future directions[END_REF], and most proposed algorithms reduce the issue of transcribing music to identifying the four basic parameters of a note event.

For what concerns applications, in addition to the transcription application itself, computational music transcription systems, even in its low-level encoding, can be further used for a wide range of high-level applications including automatic search and annotation (e.g. retrieval of musically-similar recordings) from large audio databases, real-time interaction between musicians and computers, analysis of recordings of the same piece by different performers and audio to score alignment (also used in automatic music tutors, as it aims to match the performance of the user with the original notation in order to help the music student to align her/his performance visually) [START_REF] Mayor | Performance analysis and scoring of the singing voice[END_REF].

Another AMT application of particular interest in this PhD concerns computational musicology. The term Computational Musicology comes from the research tradition of musicology, field that has focused on the study of the symbolic representations of music (scores) of the classical western music tradition. This research perspective takes advantage of the availability of scores in machine-readable format. Music theoretical models, like the one by [START_REF] Lerdahl | A generative theory of tonal music[END_REF], are very much followed and current research focuses on the understanding and modeling of different musical facets such as melody, harmony, or structure, of western classical music. Eventually, AMT systems would also aim to help amateur musicians without proper music education, or musicians whose countries do not possess a culture of music writing, such as orally transmitted traditional repertoires of Madagascar, to write down their musical compositions [START_REF] Wang | A robust singing melody tracker using adaptive round semitones (ars)[END_REF]. Although this second category of music repertoires will require much more research works before automatizing the process of transcribing them, as fundamental questions should be addressed beforehand about the identification/representation of their musical codes, which may present strong singularities with standard euro-genetic2 music language.

Challenges & perspectives

In order to fully solve the AMT problem and have a system that provides an output that is equivalent to conventional sheet music, additional issues need to be addressed, such as metre induction, rhythm parsing, key finding, note spelling, dynamics, fingering, expression, articulation and typesetting. Then, the transcription of real-world music with an unknown number of coinciding notes, arbitrary instrumentation, various musical genre and tempi, or percussive accompaniment suffers from many unsolved problems. As a con-sequence, most AMT systems adopt simplifying processing scenarios, such as restricting their musical corpus to certain musical genres or even to only one specific class of instruments [START_REF] Bello | Automatic piano transcription using frequency and time-domain information[END_REF][START_REF] Gainza | Onset detection and music transcription for the irish tin whistle[END_REF]. In particular, piano music attracted a lot of attention due to its comparatively limited spectral and temporal variations [START_REF] Marolt | Neural networks for note onset detection in piano music[END_REF][START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF][START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF]. Most current AMT systems are also limited to obtain only a partial transcription of complex musical signals, e.g. restricting the transcription to the dominant melody or bass lines [START_REF] Ryynanen | Automatic transcription of melody, bass line, and chords in polyphonic music[END_REF], or to pitch estimation of several concurrent sounds over short frames of a recording. This last sub-task of AMT, called Multi-Pitch Estimation (MPE), is at the core of AMT and is often used as a front-end representation to retrieve higher-level note parameters needed for AMT. Beyond these practical problems of retrieving physical features from a complex signal, the main challenge of current automatic transcription of music would be to capture all components which make the musicality of a piece, which cannot simply reduce to signal variations of pitch and amplitude. For example, it is often stated by musicologists that musical breathing guides segmentation of music, but how can such a concept be objectively captured by mathematical descriptors ? In the following, we describe two specific challenges for AMT methods which particularly interest this PhD project.

Facing music diversity

The meeting between Music Information Retrieval and Ethnomusicology3 has given way to a new scientific discipline called Computational Ethnomusicology (CE) [START_REF] Tzanetakis | Computational ethnomusicology[END_REF], which aims to adapt MIR tools and develop specific ones to the corpus of ethnic music. Following the conferences of the International Society of Music Information Retrieval, we can see this progress and identify the current trends [START_REF] Downie | Ten years of ismir: reflections on challenges and opportunities[END_REF]. The community is quite conscious of the limitations of the current approaches [START_REF] Lidy | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections[END_REF] and advancements are being explored by increasing the sizes of the audio collections and the variety of the data types used. Also, the expansion of the Folk Music Analysis (FMA) workshop (with a 5 th edition organized by our team at the University of Pierre and Marie Curie in Paris) also largely confirms this tendency.

On the contrary to MIR, which focuses more on the fundamental development of generic computational tools, quite regardless of the evaluation sound datasets, CE projects rather start from musical corpus to choose/adapt/develop their computational tools. Several PhD projects have already been completed in this field, we can mention [START_REF] Gomez | Tonal description of music audio signals[END_REF], who worked on methods of tonality induction, including in her test corpus a great variety of musical genres which extend the "usual" classical sound dataset. [START_REF] Kranenburg | A computational approach to content-based retrieval of folk song melodies[END_REF] developed content-based retrieval systems for a vast corpus of folk song melodies. [START_REF] Gedik | Automatic transcription of tradional turkish art music recordings : a computational ethnomusicology approach[END_REF] was interested in automatic transcription of traditional Turkish art music. More major research projects have also been initiated, such as the CompMusic (2011CompMusic ( -2017) ) and DIADEMS (2012DIADEMS ( -2015) ) projects.

This research field of CE has already known the pitfall of opposing too radically between "western" and "non-western" (terms used in [START_REF] Gomez | Comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction[END_REF]) repertoires in their computational studies (see interesting review of [START_REF] Lartillot | Commentary on "comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction" by emilia gomez, and perfecto herrera[END_REF] about [START_REF] Gomez | Comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction[END_REF]'s paper), without addressing explicitly the fundamental question of knowing to what extent and regarding which features the computational analysis, especially AMT, of non-eurogenetic music could be seen as more complex than euro-genetic repertoires. Several papers have raised such concerns in those terms [START_REF] Moelants | Problems and opportunities of applying data-& audiomining techniques to ethnic music[END_REF](Moelants et al., , 2007;;[START_REF] Lidy | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections[END_REF].

Incorporating musical knowledge

To overcome current limitations of AMT systems, a practical engineering solution was to use computational techniques from statistics and digital signal processing allowing the insertion of prior knowledge from cognitive science, musicology and musical acoustics (Engelmore and Morgan, 1988;[START_REF] Ellis | Prediction-driven computational auditory scene analysis[END_REF]. This approach is close to human experience, in which the perception of sounds is embedded with prior knowledge, using a collection of global properties such as musical genre, tempo, and orchestration, as well as more specific properties, such as the timbre of a particular instrument.

An important aspect of current AMT programs in regards of musical knowledge is that they are strongly oriented towards eurogenetic instrument repertoires. While the classical solo piano is indeed the most represented instrument in AMT studies [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF], most of them also follow a series of assumptions based on cultural concepts. These assumptions apply to structural aspects (e.g. equal temperament, tonal key, assumption of octave equivalence, instrumentation), social organisation of the music (e.g. composers, performers, audience) and technical aspects (e.g. record company, release date). To address this gap between current AMT technologies and actual diversity of musical repertoires around the world, the Music Information Retrieval (MIR) and Ethnomusicology communities have met to give way to a new scientific discipline called Computational Ethnomusicology (CE) [START_REF] Tzanetakis | Computational ethnomusicology[END_REF]. This new discipline aims to adapt MIR tools to the multiple corpus of non-eurogenetic music. As a result, this global tendency of AMT systems has raised many concerns [START_REF] Tzanetakis | Computational ethnomusicology[END_REF]Moelants et al., 2007;[START_REF] Lidy | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections[END_REF] about the appropriateness of current AMT systems to efficiently transcribe repertoires from different musical cultures. Investigations on the subject have already been initiated, either in a broad perspective with a radical eurogenetic / non-eurogenetic opposition [START_REF] Gomez | Comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction[END_REF][START_REF] Lidy | On the suitability of state-of-the-art music information retrieval methods for analyzing, categorizing and accessing non-western and ethnic music collections[END_REF], or through a single traditional instrument repertoire (e.g. [START_REF] Benetos | Automatic transcription of turkish makam music[END_REF] in Turkish Makam music). In this PhD project, we will focus on the instrument repertoires of the marovany zither from Madagascar.

Context & Objectives

Within this PhD project, we will deal with the task of automatic transcription of polyphonic music from solo instruments 4 . All instruments study in this PhD, including the marovany zither as main instrument of investigation, are decay instruments, i.e. producing sounds characterized by a sudden transient attack with a rapid rise to its peak amplitude. This attack is followed by a long decay envelop. For plucked string instruments and piano, these two successive phases respectively result from the sudden excitation of a string, and then from the free-resonating behaviour of the soundboard.

The two underlying objectives of this PhD project are to 1. develop efficient automatic music transcription systems dedicated to different repertoires of pluck string instruments, with a case study on the marovany zither from Madagascar, and 2. understand the contributions of musical knowledge in the optimization of these systems to the different repertoires. These two goals call for both "practical research" with short-term developments of operational tools, and "fundamental research" providing deeper insight into the functioning of these tools.

Our long-term objectives would serve ongoing ethnomusicological studies by providing computational tools able to generate automatically ready-to-use robust transcriptions, so they can be used as analytical supports for ethnomusicologs. They would further help in organizing and structuring audio recordings of this instrument, and thus in better understanding the evolution of the traditional repertoire.

Methodological approach

Our methodological framework is at the crossroads of different research fields. The science of instrumental acoustics (1) study the mechanisms that produce the sound using precise physical modeling of the vibrations and couplings which take place in musical instruments when subject to a player excitation [START_REF] Fletcher | The Physics of Musical Instruments[END_REF][START_REF] Rossing | The Science of String Instruments[END_REF]. This physics-based approach allows relating the signal characteristics of a tone to the physical properties of the sound producing components of an instrument (e.g. the materials and the geometry of strings and soundboard, the string-sound board coupling, etc.). In our PhD project, we will not perform complex physical modeling and modal analysis of our instruments, but rather use generic physics formula and principles to guide and validate them through our signal analysis. This approach actually consists in mixing instrumental acoustics with audio signal processing [START_REF]5 Illustrative example explaining the computation of the state-conditional likelihood probability[END_REF], which rather aims at modeling musical sounds according to their "morphological" attributes, without necessarily paying due regard to the particular instruments that are played. Eventually, the science of statistics learning and modeling (3), such as sparse coding [START_REF] Mallat | Matching pursuit with time-frequency dictionaries[END_REF], Bayesian modeling [START_REF] Gelman | Bayesian Data Analysis[END_REF], or rank reduction methods [START_REF] Cichocki | Nonnegative Matrix and Tensor Factorizations. Applications to Exploratory Multi-way Data Analysis and Blind Source Separation[END_REF], is also used, which allows building probabilistic signal modeling with the incorporation of instrument-specific musical knowledge. The marovany is a tall zither in the form of a rectangular box built from recycled wood products (the box is commonly made in plywoods, and the easels in rosewoods), as illustrated with four different models of this instrument in figure 5. Table 2 shows the physical dimensions of our different marovany models. The metallic strings, measuring up to 1 m 20, and mostly coming from brake cables type motorcycle, are stretched on each side of the box. They are nailed at each end on an easel, made of wood or metal, and are raised by battens whose places along a string determines its pitch. Then, during a song, to each string corresponds a single pitch. However, musicians may switch pitches and tuning of their marovany between different songs. Wood type, sizing and number of strings of a zither are not fixed. One can indeed find zithers made of light or heavy weight wood, measuring from 1 to 2 meter in length, possessing from 10 to 12 strings on each side, with battens also ranging from 2 cm to 0.5 cm in height (it is known that bringing strings closer to the soundboard produce more powerful sounds, according to an effect called mafo be, i.e. "louder"). Each set of strings of the marovany forms, like the famous tubular zither valiha, an alternating diatonic scale. The minimal harmonic interval of this instrument is then the semi-tone, although tuning deviations from the well-tempered scale are often observed, as described later.

Originally, the marovany is a traditional instrument of Madagascar, whose musicians often take part to trance rituals called tromba. The marovany repertoires are mostly orally transmitted, resulting from a process of memorization/transformation of original base musical motives. These motives represent an important culture patrimony. Nowadays, it has been exported outside the country, mainly by native musicians participating to World Music festivals, subject to the influences of different musical genres and practices. Although the valiha zither has already been subjected by past research studies (Razafindrakoto, 1999;[START_REF] Domenichini | Stanley Dani[END_REF], there is currently no large-scale systematic analysis and classification of the marovany repertoires, based on precise musical (rhythmic, modal, structural properties) criteria. Especially, the total absence of manuscript support for the traditional repertoire of the instrument has further impeded the development of such a work. The most common feature of CE studies so far is indeed the use of audio recordings instead of symbolic data (as many traditional repertoires are transmitted orally, without well-established musical codes), which calls for the need of transcribing sampled recordings of music into a piano-roll type representation. The relevance of this endeavour to the CE community is straightforward: piano-roll provides a cartoon-like representation of musical data that is extremely compact when compared to the sampled audio data, yet retains the necessary information for many content based analyses and queries. The automation of this process is made imperative as manual transcriptions are cumbersome and time-consuming. Also, the complexity of this transcription (due to speed of playing, polyphonic characteristics, noisy environment) implies a great variability in hand-made results, making them prone to errors, without systematic estimation of transcription accuracy. Musical Creativity, 1. with the integration of traditional instrument in the ImproteK project of humanmachine interaction (Book 1) ;

2. using the creative potential of our retrieval system in musical performances, through the collaboration with three musicians playing different instruments, each one having his own personal project on how using this "new" instrument ;

Despite a few shortcomings, we think the general methodology described in this thesis is of high interest, and is undoubtedly useful as a brief tutorial on the use of computational methods in comparative music research with large corpora. As such, the thesis is rather technical, although we tried adding more explanation, examples, and illustrations to increase its accessibility to empirical musicologists.

Plan

This PhD thesis is organized as follows. Chapter 1 presents the different Knowledge Components of Musical Acoustics (KCMA) used to help in transcription, and develops the methods used to identify and extract these components. Chapter 2 details the different baseline statistical methods used for transcription, which also constitute the statistical background for KCMA incorporation, with the methods detailed in Chapter 3. Chapter 4 presents the methods we developed to create our ground truth sound database, which is mainly based on the Multi-Channel Sensory System (MCSS) technology. We eventually present results in Chapter 5 and discuss some specific questions around the notion of KCMA in AMT systems.

Introduction

Musically, the occurrence of simultaneous notes can result either from "acoustic polyphony", or from "musical polyphony". "Acoustics polyphony" is strongly related to the timbre of the instrument, and more precisely to the physical phenomena of mutual resonances and note persistence. Although this type of polyphony is an integral part of instrument timbre, it represents a noise signal added to the actual played note from the point of view of music transcription. For what concerns "musical polyphony", it corresponds to the note combinations played by the musician and intended by a composer with a proper polyphonic writing. It directly provides useful information about which notes are commonly played simultaneously in a musical piece. In this PhD thesis, we will formalize all knowledge one can collect on these two types of music polyphony through the notion of Knowledge Components of Musical Acoustics.

Sources of KCMA

We will begin this chapter by presenting the different data sources used in this PhD project to generate our different Knowledge Components in Musical Acoustics (KCMA). These components cover the multi-facets of musical signal, from timbre to music language. Based on their different sources, the KCMA extracted can be roughly categorized into three types of musical acoustics knowledge, as described in table 1.1. Although these different categories have been designed for sake of clarity, they should not be understood as exclusive categories, as they are of course strongly correlated in musical practice. In the following, we briefly present the different data sources of our KCMA.

Theoretical musical codes

Two different sciences have been studied to compose this first data source of KCMA. From the science of pyschoacoustics, many theories have been put forward about the degree of acceptance and pleasure when listening to certain acoustic events. For example, studies show that concurrent harmonic sounds (sharing harmonic partials in simple algebraic relation) are preferred, in comparison to noisy or inharmonic sounds. Specific knowledge about psychoacoustic similarities between chords can also be deduced from these theoretical concepts. From the science of musicology, euro-genetic is well-known to have a long history of theorization, from which many different musicological concepts have been created, such as the notions of tonality and equal temperament.

Isolated note samples

This second data source of KCMA consist of instrument isolated sound samples. Musical knowledge extracted from these samples allows extracting various components of musical acoustics knowledge. It basically allows learning timbre features, which can be used to constrain generic signal models with acoustics-based information in different instrumentdedicated AMT systems (e.g. [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF] and [START_REF] Rigaud | Does inharmonicity improve an nmf-based piano transcription model ?[END_REF] using inharmonicity measures for the piano instrument, [START_REF] Benetos | Automatic transcription of turkish makam music[END_REF] using pitch-wise spectral acoustic signatures for the ney instrument Turkish makam music). But it can also allows to extract the precise tuning of an instrument, which is more of a musicological information, and also to identify the impact of specific playing techniques on instrumental note acoustics.

Transcripts from playing

This fourth data source is commonly obtained from a technology called Multichannel Capturing Sensory System, which allows a physical measure of the musical playing directly on the instrument being played. Such a technology will be developed in Sec. 4 of this PhD thesis. Such transcripts encompass both the "prescribed" musical features, as found in written scores, as well as the "interpreted" musical features, i.e. to the particular playing style of a musician, and include all knowledge informing the question on how an instrument is played (e.g. the use of the soft pedal in piano, or palm mute in guitar), which modifies intrinsic acoustic properties of the instrument, as characterized by isolated note samples.

Written Scores

Musical knowledge extracted from our third data source of KCMA, namely written scores, is rather related to music language itself. This language encompasses "prescribed' features, which refer more to standard codes characteristic of a musical piece, and possibly to its belonging musical genre. "Prescribed" features include note-to-note harmonic transitions, tonality, rhythmical figures, base musical motives.

Index

Preliminary discussions of notions

Temporal scaling: frames / note events Most MPE algorithms do not focus on note objects and their features but generally refer to a data granularity of a lower abstraction level: frames (labelled t ∈ T). A frame is a short chunk (typically on the order of 10 ms) of audio, from which both time and frequency domain features can be computed. Consecutive frames are usually considered with some overlap for smoother analyses. Furthermore, note segmentation is commonly done by a simple energy-based aggregation of successive frames (see Sec. 2.3.3), without considering longer observation time scales. However, using notes as the basic analysis units, instead of pitched frames, enables a more profound melody modeling, since the musicological relationships between these units are well-defined when examining them as constituents of melodies and musical context.

Then, when formatting our KCMA, the question of their time scale is of first importance. Indeed, although frame-based knowledge is more easily fused with algorithm estimations, as sharing the same time scale, note event-based knowledge is much more pertinent for music language modeling than frame-based information, especially for time-dependent sequential knowledge (e.g. a same melody played at different tempi will provide different frame-based modeling on harmonic transitions). Furthermore, note event-based information can be directly interpreted and informed by musicologists, and allows inter-repertoire comparisons based on explicit musical features. In literature, the work of [START_REF] Ryynanen | Polyphonic music trancription usng note event modeling[END_REF] is one of the very few examples which adopted such a note event-based approach for music knowledge incorporation in their AMT system.

Analysis musical event: chords / note mixtures

Studies interested in chord segmentation and recognition (e.g. [START_REF] Bello | A robust mid-level representation for harmonic content in music signal[END_REF]; [START_REF] Papadopoulos | Large-scale study of chord estimation algorithms based on chroma representation and hmm[END_REF]; [START_REF] Lee | Acoustic chord transcription and key extraction from audio using key-dependent hmms trained on synthesized audio[END_REF]) naturally build their sequential musicological knowledge on the musical object of chords. The chord is a fundamental unit of music structure, and corresponds to attributes of Western chord notation such as "minor", "major", "diminished" ... It is determined both by its constituent pitch classes, and also by the syntactical context in which the chord occurs. Contextual significance is then a consequence both of the chord's position relative to other chords within a temporally organized sequence and also to its functional relationships to other chords and (ultimately) to an underlying "tonic". Furthermore, because the chord as a structure comprised essentially of pitches, its contextual identity (and the perceptibility thereof) is typically strong enough to shine through the addition or deletion of individual notes.

However the musical object of chords has limited applications in music retrieval tasks other than chord recognition, such as AMT, as it is not flexible enough to allow for a general modeling of music language. To answer this concern, we introduce in this PhD thesis the concept of the note mixture event (which will be defined mathematically in Sec. 2.4.5). This notion of mixture events refer to any grouping of notes simultaneous in time.

Definition of the matrix of inter-pitch influence ι

This 12-dimensional matrix ι quantifies the likelihood that the 12 pitch classes within an octave range either combine with each other, or transit from one to another. This matrix can easily be generalized to a size of I 2 , with I the number of pitches. The matrix ι is then defined as follows,

∀(i, j) ∈ {1, • • • , I} 2 , ι =    ι 11 • • • ι 1I . . . ι ij . . . ι I1 • • • ι II    (1.1)
where ι ij denotes the likelihood that pitch i is combined with or is followed by pitch j. It is noteworthy that simpler modeling of prior knowledge, such as a simple pitchdependent vector, can also take the form of a diagonal matrix ι of size I 2 , with the vector values put into this diagonal (the zero-coefficients of S provide an unitary prior value which does not affect particle weights).

Context and objectives

In this PhD, our different KCMA are listed in table 1.2. Although we have reviewed in our Introduction all data sources of KCMA for sake of completeness, in our context of orally transmitted music, i.e. without any written supports, a priori knowledge can only be extracted from the first three data sources from table 1.1.

Knowledge source I : from theoretical concepts

We propose in the following knowledge-based musicological modeling, where the output distribution parameters are fixed based on an expert's music theoretical knowledge. In typical tonal music, most note mixture progressions are repeated in a cyclic fashion as the piece unfolds. Also, notes comprising a note mixture act as central polarities for the choice of notes at the next note mixture in a musical piece. Furthermore, given that a particular temporal region in a musical piece is associated with a certain note mixture, notes comprising that mixture or sharing some harmonics with notes of that mixture are more likely to be present. These "universal" musical tendencies provide a strong prediction on note succession and superposition. Such a "universal" musical tendency will be modeled in the following with two different approaches, from pyschoacoustics and musicology. In our following section titles, we report the name of the KCMA (like e.g. (KCM A ∼ T heo1)), as listed in table 1.2, described in the given section.

By Psychoacoustics (KCM A ∼ T heo1)

In this first section, psychoacoustic considerations on acoustic properties of note mixtures will be used to compute a likelihood on note transition. This likelihood will quantify perceptual acoustic similarity in the transition from one mixture to another. This modeling starts with the assignment of a single timbre for each group of observed notes forming a note mixture [START_REF] Vassilakis | Chords as spectra, harmony as timbre[END_REF].

Parametric note model

The frequency content of an idealized musical note i is composed of a fundamental frequency F 0,i and integer multiples of that frequency. The amplitude of the h-th harmonic F h,i = h F 1,i of note i can be modeled with geometric decaying ρ h , with 0 < ρ < 1. A slightly more complex signal modeling will be used here, which includes a parameter of inharmonicity β in the relation between fundamental and harmonics. For real strings, the frequencies of the partials obey the formula

F h,i = hF 0,i 1 + β(h 2 -1) (1.2)
where F is the fundamental frequency, h the harmonic index (partial number ≥ 1), and β the inharmonicity factor [START_REF] Fletcher | The Physics of Musical Instruments[END_REF]. This inharmonicity phenomenon is due to the stiffness of real strings and causes the higher-order partials to be slightly shifted upwards in frequency. Furthermore, to take into account the spectral energy spreading around each harmonic location, a harmonic spectrum is modeled as the sum of Gaussians representing the partials, similarly to the HTC model [START_REF] Kameoka | A multipitch analyzer based on harmonic temporal structured clustering[END_REF]. Indeed, due to the convolutive nature of this kind of source-filter model, the harmonic partials can be designed independently of the pitch of a note and the spectral spreading of its partials.

Perceptual bin-wise loudness

We then map this harmonic model with frequency f to a discrete pitch scale with a 20-cents resolution1 , and define a perceived loudness of each spectral bin b f present in a note mixture M t k (this notion will be defined later in Sec. 2.4.5) as

l k (b f ) = max h∈N,i∈Mt k (ρ h |m(f h,i ) = b f ) (1.3)
The max function is used instead of a sum in order to account for the masking effect [START_REF] Moore | An Introduction to the Psychology of Hearing[END_REF]. For each note mixture M t k , we then have l k = {l 1 (i 1 ), • • • , l d (i d )} corresponding to the perceived strength of the harmonics related to every note i d of the well-tempered scale.

Perceptual loudness within an octave

We can eventually use octave invariance to give a measure v k of the relative strength of each spectral bin in a given note mixture. We then obtain 1200 20 = 60 bins b f o within an octave, i.e. 5 bins per pitch class, forming the note mixture representation v k = {v k (0), • • • , v k (59)}, whose elements are defined as

v k (b f o ) = (b f mod60)=i l(b f ) (1.4)
where i is the pitch. Perceptual similarity From the previous timbre-related note mixture modeling v k , we need to define a transition probability between two note mixtures. Perceptually similar mixtures tend also to be close in Euclidian distance [START_REF] Krumhansl | ), chap. A key-finding algorithm based on tonal hierarchies[END_REF], which motivate us to define our probability as

Ed = v i -v j 2 (1.5)
and from this timbre information, we can obtain a transition probability between two mixtures by computing T heo1 ∼ P (i, j) = e -Ed(i,j) j e -Ed(i,j)

(1.6)

By Musicology

Major and minor keys given the key (KCM A ∼ T heo2/T heo3)

Key is an architectonic system of contextual relationships, and has been foundational to Western music, both classical and popular, since the 17 th century. The term key (or tonality) is usually defined as the relationship between a set of pitches having a tonic as its main tone, after which the key is named [START_REF] Kennedy | The Concise Oxford Dictionary of Music (Encyclopedia.com[END_REF]. A key is then defined by both its tonic and its mode, and generates expectations favouring certain pitch sequences. The tonic identifies to one of the 12 semitones of the chromatic scale within an octave range. The mode is usually minor or major, depending on the used scale. The major and minor keys then rise to a total set of 24 different tonalities. Figure 1.3 represents the key signatures of the 24 major and minor keys. Euro-genetic music is mainly governed by the chord templates (e.g. see http://www.piano-keyboard-guide. com/major-chords.html)), defined as the theoretical chroma vectors corresponding to the 24 Major and minor triads. Knowing the key of a piece of course provides very valuable information about the chords as well, as in Euro-genetic tonal music, a key and chords are very closely related. For instance, if a musical piece is in the key of C major, then we can expect frequent appearances of chords such as C major, F major, and G major, which correspond to the tonic, subdominant, and dominant chord, respectively. On the other hand, minor or major chord do not appear, since neither has any harmonic function in a C major key. Based on the key information, we can then compute 24 ι matrices of harmonic transitions, one for each key, as illustrated in figures 1.4 for the keys A major and C sharp major. The amplitude of a note in the key is set to 1 if the note belongs to the considered key. These matrices can then be used as KCMA for both note combination, i.e. ∼ T heo2, and transition, i.e. ∼ T heo3.

Figure 1.4 -Transition probability matrices of musicological modeling. It gets its knowledge from the theoretical keys described in Sec. 1.2.2. One state transition matrix is created for each key, as here for example, are represented matrices associated with keys A major and C sharp major.

Major and minor keys (KCM A ∼ T heo4/T heo5)

When key information is not available, a more global modeling can be performed using knowledge of all major and minor keys, by summing and normalizing element-wise the 24 ι matrices respective to each key. As previously, this matrix can then be used as KCMA for both note combination, i.e. ∼ T heo4, and transition, i.e. ∼ T heo5.

The circle of fifths (KCM A ∼ T heo6)

Chords succeed to one another following certain rules. The transition probability between two chords can be derived from musical knowledge: their distance in the doublynested circle of fifths, as proposed by [START_REF] Bello | A robust mid-level representation for harmonic content in music signal[END_REF] (see this reference for details) and represented in figure 1.6. The doubly-nested circle of fifths depicts relationships among the 12 equal-tempered pitch classes comprising the chromatic scale. Although we do not know which state is going to follow another one, musical rules allow us to make hypotheses that are more probable than others. For instance, especially in popular western music, an A major chord is more likely to be followed by a F# minor or D major chord than by a G# Major chord. It is also assumed through this model that most music tends not to make large and quick harmonic shifts, but rather one might gradually wander from the C to the F#, but not immediately.

T heo6 is computed by forming a ι matrix obtained from this chord-based modeling by decomposing them into their constitutive notes, and report to the individual pitches their different likelihoods from the chords they belong.

Figure 1.6 -Doubly-nested circle of fifths, with the minor triads (lower case) staggered throughout the major triads (upper case). Triads closer to each other on the circle are more consonant, and thus receive higher initial transition probability mass than triads further away. From [START_REF] Bello | A robust mid-level representation for harmonic content in music signal[END_REF].

Krumhansl (1990)'s tone profiles (KCM A ∼ T heo7) [START_REF] Krumhansl | Quantification of the hierarchy of tonal functions within a diatonic context[END_REF] have shown that stable pitch distributions give rise to mental schemas that structure expectations and facilitate the processing of musical information, making individuals intuitively expect and perceive basic patterns of tonal organization, almost as if they were born with this understanding [START_REF] Krumhansl | ), chap. A key-finding algorithm based on tonal hierarchies[END_REF]. Using the now famous probe-tone method, [START_REF] Krumhansl | ), chap. A key-finding algorithm based on tonal hierarchies[END_REF] showed that listeners' ratings of the appropriateness of a test tone in relation to a tonal context is directly related to the relative prevalence of that pitch-class in a given key. The Krumhansl tone profiles for major and minor keys (Krumhansl, 1990, p. 67), represented in figure 1.7 have been widely used in key recognition research (e.g. [START_REF] Temperley | Music and Artificial Intelligence[END_REF]; [START_REF] Hu | A probabilistic topic model for unsupervised learning of musical key-profiles[END_REF]). T heo7 is directly identified to these tone profiles in order to predict most likely note combination according to this theoretical knowledge, given the key.

Knowledge source II : from isolated note samples 1.3.1 Signal samples (KCM A ∼ IsoN o0)

In our knowledge incorporation, we will use isolated note samples x(t) to extract probabilistic templates which directly fit the spectral characteristics of these notes. Figure 1.8 reveals that even from these simple representations, clear differences can be observed between notes with different pitches. The spectra of these note samples are classically computed using the short-time Fourier transform of the signal x(t), defined as

X(f, t) = +∞ -∞ [x(τ )w(t -τ )]e -j2πf τ dτ (1.7)
with w(t -τ ) a gliding hamming-window, f ∈ F the frequency bin and t ∈ T the time frame index. Furthermore, acoustic analysis of musical signals [START_REF] Fletcher | The Physics of Musical Instruments[END_REF] reveal that the timbre of an instrument varies over its pitch range, as well as over different playing dynamics. This dynamics refers to the volume of a note, whose variations, called nuances, are crucial for expressiveness in music. Also, in plucked string instruments the strings may be either plucked or rubbed at truly different distances from the bridge on which they pass, involving also a great spectral envelop variety for each different pitch. As a result, this greatly varies the intrinsic acoustic signature of an instrument inducing a local timbre alteration. As it will be explained later (see Sec. 4.4), the recording of isolated note samples allows including different instrument models and playing techniques, with their corresponding modifications in acoustic features.

Basic acoustic descriptors

Any instrument has a specific acoustic signature, which makes them recognizable among different plucked string instruments playing a same pitch 2 . It is well known that timbre is multidimensional, i.e. it is not correlated with a single acoustic property. Among these properties, attack and decay transients, inharmonicity and changes in the distribution (i.e. amplitude and shape) of spectral energy contribute to the perception of timbre [START_REF] Iverson | Isolating the dynamic attributes of musical timbre[END_REF][START_REF] Handel | Hearing[END_REF][START_REF] Godsmark | A blackboard architecture for computational auditory scene analysis[END_REF]. We propose in the following a low-modeling of this timbre through the computation of four acoustic descriptors on isolated note samples (see Annex A for computational details).

Note duration (KCM A ∼ IsoN o1)

Physical duration of a note is related to both the strength of the excitation and the vibratory properties of the resonating structure, e.g. shorter string with strong stiffness (i.e. higher pitch) tends to have a shorter duration [START_REF] Fletcher | The Physics of Musical Instruments[END_REF]. We use the descriptor Ef f Dur, detailed in Annex A, taken from [START_REF] Peeters | The timbre toolbox: Extracting audio descriptors from musical signals[END_REF].

Amplitude of Energy Modulation (KCM A ∼ IsoN o2)

Amplitude of Energy Modulation aims to quantify a signal modulation depth (i.e. peak-to-valley difference) in the temporal envelop of a sound. The phenomenon of intermodulation in musical sound is characterized by a strong amplitude of energy modulations with most often the formation of several valleys in the envelop. The marovany timbre presents a certain emphasis of this feature, depending on pitches and playing modes. We use the descriptor AmpM od, detailed in Annex A, taken from [START_REF] Peeters | The timbre toolbox: Extracting audio descriptors from musical signals[END_REF].

Spectrum Inharmonicity (KCM A ∼ IsoN o3)

Spectrum Inharmonicity quantifies small departure from exact harmonicity, particularly the overtone series is slightly stretched in the high-frequency band, as illustrated in figure 1.9. To quantify automatically this feature, we use the descriptor HD, detailed in Annex A. 2. Definition of the timbre [START_REF] Fletcher | The Physics of Musical Instruments[END_REF].

Spectrum Variability (KCM A ∼ IsoN o4)

Spectrum Variability quantifies frame-to-frame differences in spectral envelops. We use the descriptor SpecV ar, detailed in Annex A.

In certain instruments, this acoustic signature encompassed by these different acoustic descriptors can be very variable through the different pitches, as illustrated in figure 1.10 for the marovany. As we study decay instruments, with small variations on the onset part, the two KCMA ∼ IsoN o1 and IsoN o2 encompass most of the temporal timbre signature characteristic of each instrument. From top to bottom, these descriptors are the duration, the intermodulation, the spectrum inharmonicity and the spectrum variability.

Sympathetic resonances (KCM A ∼ IsoN o5)

Background

Mutual resonances (also called sympathetic resonance or sympathetic vibration) result from a harmonic phenomenon wherein a formerly passive string or vibratory body responds (or modes) to external vibrations to which it has a harmonic likeness [START_REF] Rossing | The Science of String Instruments[END_REF]. For strings on a bowed, plucked, or hammered instruments, mutual resonances result from sympathetic strings, which vibrate (and thereby sound a note) in sympathetic resonance with the note sounded near them by some other agent [START_REF] Kennedy | The Concise Oxford Dictionary of Music (Encyclopedia.com[END_REF]. Unison or octave will provoke the largest response as there is maximum likeness in vibratory motion.

Measurement protocol & Quantified measure

To generate knowledge for this prior, we quantify the acoustic residuals in pitches j due to mutual resonances from a played pitch i = j. It takes the form of the matrix ι. To compute each coefficient ι ij , we used two datasets of isolated notes, a first one composed of free-resonating notes, and a second one in which all strings were muted excepting the played one. For each note sample of these two datasets, the spectrum was computed with a FFT using a 4096-sample Hamming window after the onset, unitary normalized and labelled X (d,i) for pitch i and dataset d (equal to 1 or 2). We then used the algorithm 1 to get the different scores ι ij .

Algorithm 1 Computation of coefficients ι ij from sympathetic resonances.

1: for For each pitch i ∈ {1, • • • , N I } do 2: Xi = ||X (1,i) -X (2,i) || 3:
Binary thresholding of Xi , i.e.

Xi (f ) = 1 for f = arg( Xi ≥ 0.5) 0 otherwise

4:

for Each pitch j ∈ {1, • • • , N I }, j = i do 5:

ι ij = X (2,j) • Xi , with [•] the element-wise product 6:
end for 7: end for It is noteworthy that due to different acoustic behaviors of mutual resonances depending on the excited pitch, the matrix ι corresponding to KCM A ∼ IsoN o6 is not symmetrical. Figure 1.11 provides an example of such a matrix for the marovany, using the instrument model N 1 .

F3 G3 A3 B3 C4# D4# F4 G4 A4 B4 C5# D5# F5 G5 A5 B5 C6# F3 G3 A3

Non-tempered tuning (KCM A ∼ IsoN o6)

Background

In a "non-tempered" scale, harmonic intervals within each octave are not multiple of a common unity, the semi-tone. Tempered scale, as instaured in the MIDI scale, has been introduced in the XVIII century. Octave are correctly tuned in the sense of natural resonances, i.e. frequencies are in a rapport of 2, but smaller harmonic intervals are multiple of the semi-tone, which results in the equal temperament, where neither thirds and fifths are correctly tuned in the sense of natural resonances, i.e. their frequencies are not in a simple division of 3 or 5. In the XVIIe century, other musical temperament were used to tune harpsichord for example, such as bare fifths (Pythagore temperament) or bare thirds (Zarlino temperament).

Although theoretical pitch scale with equal temperament can be assigned to marovany repertoires for convenience of analysis and music transcription, as illustrated in figure 1.12, it appears from quantitative measurements that the players do not systematically adopt this standard tuning. Especially traditional marovany players, without any influences from eurogenetic music concepts, tend to tune their instrument with the "just" intervals for thirds, fifths and, minor sevenths, i.e. at an exact sub-multiple of the chord fundamental note frequency.

Measurement protocol & Quantified measure

According to our training data, this tuning deviation has a reasonably strong presence in Velonjoro repertoire in particular, with a variation from 5 cents to 45 cents depending on the pitch (i.e. almost up to half of a semi-tone, as this interval is equal to 100 cents), as shown in figure 1.13. These measures have been performed automatically using the YIN pitch tracker [START_REF] De Cheveigné | Yin, a fundamental frequency estimator for speech and music[END_REF], and subtracting these physical measures to the theoretical values of the equal temperament.

Knowledge source III : from playing-based transcripts 1.4.1 Data-based frequency counting of note mixture occurrences (KCM A ∼ T raP la1/T raP la2)

Let's first model which pitches are the most likely to be played together, which is done here by a frame-wise counting of pitches j played simultaneously to pitch i from a training dataset. This counting can then be unitary normalized and identified to the coefficients of matrix ι (eq. 1.1). As previously, this matrix is used as the KCMA ∼ T raP la1. Similarly, to compute the KCMA ∼ T raP la2 on note transitions, we perform a frequency counting of successive note mixture events M k and M k+1 . To train M k transitions, we count the number of occurrences of each M k transition in the training set. By anticipation on our AMT methods (see Chapter 2), this data-based approach allows a direct modeling of harmonic transitions on note mixtures, also learned from the training dataset. Then, in this KCMA, we keep the musical object of note mixtures as the modeling unit. Figure 1.15 provides an example of a dictionary of note mixtures, along with the matrix ι modeling their likely harmonic transitions.

Pitch-wise frame-to-frame transition (KCM A ∼ T raP la3)

Special playing techniques on an instrument repertoire, such as palm muting, pizzicato, staccato and pedal effects, mainly deviates the temporal waveform of a note from its free-resonating behaviour. These transitions are determined by sampling the training MIDI transcripts at the precise times corresponding to the analysis frames of the activation matrix, and just checking for the presence of a note in each frame.

Duration-informed pitch-wise frame-to-frame transition (KCM A ∼ T raP la4)

A more refined modeling of (KCM A ∼ T raP la3) is obtained by including a duration information in the previous pitch-wise frame-to-frame transition model. In contrast to the physical duration of each pitch, as already measured for KCM A ∼ IsoN o1 in Sec. 1.3.2, we extract a pitch-wise duration distribution from notes under musical playing, which are directly readable from our transcript data by counting successive frames corresponding to active notes.

Motive structure (KCM A ∼ T raP la5)

Background

In music theory, [START_REF] Schenker | Harmony[END_REF] asserted that repetition is what gives rise to the concept of the motive, which is defined as the smallest structural element within a musical piece. [START_REF] Ockelford | Repetition in Music: Theoretical and Metatheoretical Perspectives[END_REF] argued that repetition/imitation is what brings order to music, and order is what makes music aesthetically pleasing. Melodic progressions then constitute a fixed, non-dynamic structure in time and thus can be used to aid in describing long-term musical structure. [START_REF] Ruwet | Methods of analysis in musicology[END_REF] used repetition as a criterion for dividing music into small parts, revealing the syntax of the musical piece. A ground-truth example of the motive structure in a marovany musical piece is given in figure 1.16.

Constitution of a motif dictionary (KCM A ∼ T raP la4)

The learning phase generates a dictionary of the N max most recurrent of D-long note patterns, with d ∈ [D min : D max ]. This dictionary has a size of (D max -D min ).N max . The parameters (D min ; D max ; N max ) are set arbitrarily by the user, depending on the quantity of information he wants to integrate in the modeling. A maximal probability is assigned to each different pattern with a simple counting strategy as performed previously. Figure 1.17 illustrates a dictionary with (D min = 4; D max = 8; N max = 4).

Conclusion

In this chapter, we have developed several Knowledge Components of Musical Acoustics (KCMA) in order to characterize an instrument repertoire in view of its automatic transcription.

Chapter 2

Baseline statistical methods for AMT Abstract

This chapter presents the baseline statistical methods we used to perform Automatic Music Transcription (AMT), which are mainly based on Probabilistic Latent Component Analysis (PLCA) and Hidden Markov Models (HMMs). PLCA is a probabilistic subspace analysis method which can be used for decomposing audio spectrograms. HMMs belong to dynamic Bayesian networks, and are especially known for their applications in modeling and recognition of time series. These two statistical methods use Bayesian probabilistic frameworks, which allow powerful modeling with the incorporation of prior knowledge. PLCA is frame-based, performing the bottom level of analysis against individual spectral slices derived from short time frames. Since note events typically last for many frames, temporal continuity is introduced by some higher-level processing modeled by HMM. These two methods are then complementary by covering both the time and frequency domains of music signals.

Introduction

AMT methods in three families

Classically, the four processing steps of the AMT process are as follows 1. Multi-pitch estimation: within the MIR framework, the goal of Multi-Pitch Estimation (MPE) is to extract the fundamental frequencies of all (possibly concurrent) notes within a polyphonic musical piece.

2. Harmonic (vertical) prior / post-processing 3. Sequential (horizontal) prior / post-processing

Note segmentation

The methods proposed so far for AMT perform these steps in different ways, and can be roughly categorized into three main families, according to which signal representation or modeling methods they are based on.

Feature-based

Methods from this family mainly use signal feature properties in a blind way to extract information. [START_REF] Klapuri | Multiple fundamental frequency estimation based on harmonicity and spectral smoothness[END_REF] estimates the notes present in each frame using an algorithm that iteratively estimates and removes the fundamental frequencies of notes present. [START_REF] Bello | Automatic piano transcription using frequency and time-domain information[END_REF] report a novel method for multiple-F 0 estimation using both frequency and time domain information. The signal frames considered are a linearly weighted sum of waveforms in a database of individual piano notes. In [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF], the likelihood maximization principle is accounted for, assuming a sinusoidal model with coloured background noise and an autoregressive spectral envelope for the overtones. Based on this principle, a F 0 estimator is developed to find the fundamental frequency that maximally flattens both the noise spectrum and the sinusoidal spectrum. Klapuri (2008)'s system uses a computational model of the human auditory periphery, followed by a periodicity analysis mechanism where fundamental frequencies are iteratively detected and cancelled from the mixture signal. [START_REF] Yeh | Multiple fundamental frequency estimation of polyphonic music signals[END_REF] present a method for evaluating together multiple F 0 hypotheses based on three physical principles: harmonicity, spectral smoothness and synchronous amplitude evolution within a single source. Evaluation of each possible harmonic sound is performed using a score function, which formulates the guiding principles in a mathematical way. In [START_REF] Goto | A real-time music-scene-description system: Predominant-f0 estimation for detecting melody and bass lines in real-word audio signals[END_REF], Goto proposes a F 0 estimation method, called PreFEst, which obtains the most predominant F 0 by estimating the relative dominance of each possible F 0 (represented as a probability density function of the F 0 ). The method uses MAP (maximum a posteriori probability) estimation and accounts for temporal continuity by using a multiple-agent architecture.

Spectrogram factorization-based

As audio signals are both additive and oscillatory, it is not possible for example to look for energy and harmonic changes simply by differentiating the original signal in the time domain; this has to be done on an intermediate signal that reflects, in a simplified form, the local structure of the original. Hence, methods from ranking reduction and source separation methods have been employed. A powerful method for MPE is then to represent spectra as a linear combination of vectors from a dictionary. Such models take advantage of the inherent low-rank nature of magnitude spectrograms to provide compact and informative descriptions.

Musical signals are highly structured, and it is then possible to state the whole multiple-F 0 estimation problem in terms of a signal model, the parameters of which should be estimated. The parametric model paradigm [START_REF] Itoyama | Instrument equalizer for query-by-example retrieval: Improving sound source separation based on integrated harmonic and inharmonic models[END_REF][START_REF] Heittola | Musical instrument recognition in polyphonic audio using source-filter model for sound separation[END_REF][START_REF] Ewert | Estimating note intensities in music recordings[END_REF] allows to obtain a note-wise parametrization of the spectrogram. Eventdriven feature analysis has been shown to give more accurate musical feature extraction than more traditional approaches based on frames of equal length (Bello et al., 2005).

In [START_REF] Leveau | Instrument-specific harmonic atoms for mid-level music representation[END_REF], the audio signal is decomposed into a small number of sound atoms or molecules bearing explicit musical instrument labels. Each atom consists of a sum of windowed harmonic partials, whose relative amplitudes are specific to one instrument. Each molecule is composed of several atoms from the same instrument spanning successive time windows. The Specmurt technique is proposed by [START_REF] Saito | Specmurt analysis of polyphonic music signals[END_REF]. This technique consists in searching the fundamental frequency distribution by deconvolving the observed spectrum with an assumed common harmonic structure. In [START_REF] Canadas | Note-event detection in polyphonic musical signals based on harmonic matching pursuit and spectral smoothness[END_REF], harmonic decompositions are modified in order to maximize the spectral smoothness for those atom amplitudes that belong to the same harmonic structure.

Statistic and model-based

Here, expert systems incorporating models of sound characteristics or musical properties are employed. These experts allow for solving otherwise ambiguous situations and obtaining meaningful transcription results. [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF] treat transcription as a classification problem, using support vector machines to classify individual frames as to whether they contain particular notes or not. [START_REF] Kameoka | A multipitch analyzer based on harmonic temporal structured clustering[END_REF] have proposed the Harmonic Temporal structured Clustering (HTC) method. This method decomposes the energy patterns diffused in the time-frequency space into distinct clusters, grouping patterns from the same source. [START_REF] Ryynanen | Polyphonic music trancription usng note event modeling[END_REF] use two probabilistic models: a note event model, used to represent note candidates, and a musicological model, which controls the transitions between note candidates by using key estimation and computing the likelihoods of note sequences. In the note event model, a three-state HMM is allocated to each MIDI note number in each frame. The states in the model represent the temporal regions of note events, comprising namely an attack, a sustain and a noise state, and therefore taking into consideration the dynamic properties and peculiarities of musical performances. State observation likelihoods are determined with recourse to features such as the pitch difference between the measured F 0 and the nominal pitch of the modelled note, pitch salience and onset strength. The observation likelihood distributions are modelled with a four-component Gaussian Mixture Model (GMM) and the HMM parameters are calculated using the BaumâĂŞWelch algorithm. The note and the musicological models then constitute a probabilistic note network, which is used for the transcription of melodies by finding the most probable path through it using a token-passing algorithm. Tokens emitted out of a note model represent note boundaries.

Context and objectives

The objectives of this chapter are to expose theoretically our baseline methods for AMT, which are based on Probabilistic Latent Component Analysis (PLCA) method 2.2 and Hidden Markov Models (HMMs) 2.4. These baseline frameworks will allow to cover the first three processing steps of the AMT process. At the end of this chapter, we also present our baseline methods to perform note segmentation.

Probabilist Latent Component Analysis

Background

Probabilistic Latent Component Analysis (PLCA) belongs to a class of probabilistic models, known as latent class models, that attempt to explain the observed histograms as having been drawn from a set of latent classes, each with its own distribution. In other words, latent component models enable one to attribute the observations as being due to hidden or latent factors. The PLCA model is then probabilistic in the sense that the objective is to develop a structured representation for an empirically developed probability mass function characterizing the probability distribution for vibration amplitude "quanta" as a function of frequency f and time t. The main characteristic of these models is conditional independence, i.e. multivariate data are modeled as belonging to latent classes such that the random variables within a latent class are independent of one another.

Originally, PLCA is a straightforward extension of Probabilistic Latent Semantic Indexing [START_REF] Hofmann | Probabilistic latent semantic indexing[END_REF] which deals with an arbitrary number of dimensions and can exhibit various features such as sparsity or shift-invariance. PLCA can also be seen as a direct probabilistic extension of Non-Matrix Factorization (NMF) to obtain a better semantic interpretation, leading to enhanced modeling.

Unlike NMF which tries to characterize the observed data directly, latent class models characterize the underlying distribution P (x 1 , x 2 ). This subtle difference of interpretation preserves all the advantages of NMF, while overcoming some of its limitations by providing a framework that is easy to generalize, extend, and interpret. From past studies, the following advantages of PLCA have been highlighted, in particular from the NMF method:

1. It is efficient for modeling non-stationary sound since the learnt dictionary accommodates invariant characteristics of input sound, for this reason, linear combinations of its dictionary elements (basis) are sufficient for representing the timevarying patterns in audio signal ;

2. It can separate the non-negative components without assumptions about their orthogonality ;

3. Its probabilistic nature makes it possible to utilize additional a priori information, in particular to follow sparseness assumptions

General formulation

In more technical terms, PLCA decomposes a multi-dimensional distribution as a mixture of latent components where each component is given by the product of onedimensional marginal distributions. Recently, it has been shown that PLCA is numerically identical to NMF for two-dimensional input, and non-negative tensors for arbitrary dimensions [START_REF] Shashanka | Sparse overcomplete latent variable decomposition of counts data[END_REF]. The basic model is defined as

P (x) = z P (z) J j=1 P (x j |z) (2.1)
where P (x) is an J-dimensional distribution of the random variable x = (x 1 , . . . , x J ), z is a latent variable and the P (x j |z) are one dimensional distribution with j ∈ {1, . . . , J}.

Such a general model has been successfully applied to audio signal, with a theoretical framework developed by [START_REF] Smaragdis | A probabilistic latent variable model for acoustic modeling[END_REF]. PLCA method is then based on the assumption that a suitably normalized magnitude spectrogram, V, can be modeled as a joint distribution over time and frequency, P (f, t). This quantity can be factored into a frame probability P (t), which can be computed directly from the observed data, and a conditional distribution over frequency bins P (f |t); spectrogram frames are treated as repeated draws from an underlying random process characterized by P (f |t). We can model this distribution with a mixture of latent factors as follows:

P (f, t) = z P (z, t)P (f |z) (2.2)
where P (f |z) is a multinomial distribution of frequencies for latent component z. It can be viewed as a spectral vector from a dictionary. P (z, t) is a multinomial distribution of weights for the aforementioned dictionary elements at time t, i.e. time activations.

Using an asymmetric factorization, which treats f and t differently, we can decompose P (f, t) as a product of a spectral basis matrix and a component activity matrix, as follows

P (f, t) = P (t) z P (f |z)P (z|t) (2.3)
where z is the component index, P (t) is the energy of the input spectrogram (known quantity), P (f |z) is the spectral template that corresponds to the z th component, and P (z|t) is the activation of the z th component.

Note that when there is only a single latent variable z this is identical to NMF. The latent variable framework, however, as already said, has the advantage of a clear probabilistic interpretation which makes it easier to introduce additional parameters and constraints. It is worth emphasizing that the distributions in PLCA are all multinomials. This can be somewhat confusing as it may not be immediately apparent that they represent the probabilities of time and frequency bins rather than specific values; it is as if the spectrogram were formed by distributing a pile of energy quanta according to the combined multinomial distribution, then seeing at the end how much energy accumulates in each time-frequency bin. This subtle yet important distinction is at the heart of how and why these factorization-based algorithms work.

Formulation for Automatic Music Transcription

We now propose a reformulation of this generic method for our task of AMT of polyphonic solo instrument. The magnitude spectrogram of a sound source can be viewed as a histogram of "sound quanta" across time and frequency. With this view, probabilistic factorization, which is a type of non-negative factorization, has been used to model a magnitude spectrogram as a linear combination of spectral vectors from a dictionary.

The model then takes as input a log-frequency spectrogram X f,t and, as stated above, approximates it as a joint distribution over time and log-frequency P (f, t) (f is the logfrequency index and t the time index), that is

X N (f, t) ≈ P (f, t) (2.4) with X N = |X(f,t)| f,t |X(f,t)|
the normalized spectrogram making it a distribution of acoustic energy across the time-frequency plane. This quantity can be factored into a frame probability P (t), which can be computed directly from the observed data (i.e. energy spectrogram), and a conditional distribution over frequency bins P (f |t), as follows

P (f, t) = P (t)P (f |t) (2.5)
Spectrogram frames are then treated as repeated draws from an underlying random process characterized by P (f |t). Let's first define the latent variable z = {i, m}, representing respectively pitch and playing mode, with i ∈ I, I being the set of pitches, and m ∈ {1, • • • , M }, M being the number of playing modes considered. We can then model this distribution with a mixture of latent factors related to music transcription as follows:

P (f |t) = i,m P (f |i, m)P (t|i, m) (2.6) = i,m P (f |i, m)P (m|i, t)P (i|t) (2.7)
where P (f |i, m) are the spectral templates (also called kernel distribution) for pitch i and playing mode m. For the selected pitch i, the frequency f is selected in a probability distribution P (f |i, m). No orthogonality between the different spectral basis P (f |i) can be assumed as several of these basis are mixed into the same frequency f, or in other words, these basis overlap in frequency. P (m|i, t) is the playing mode activation, and P (i|t) is the pitch activation (i.e. the transcription, also called impulse distribution). The playing mode m will refer to different dynamics of instrument playing (i.e. note loudness). The constant-Q Transform (CQT) is usually used to compute the logarithm spectrogram. Annex A.3.2 provides details about this representation. When employed on a constant-Q transform of a mixture of notes, PLCA can be used to decompose the input data into a summation of convolutions of pitch spectra and the pitch track corresponding to their temporal activation.

Shift-Invariant PLCA

For what concerns its algorithmic implementation, we use the Shift-Invariant PLCA [START_REF] Shashanka | Sparse overcomplete latent variable decomposition of counts data[END_REF] extension of the PLCA, exploiting the fact that in a CQT, a change of fundamental frequency is traduced by a simple frequency translation of its partials, resulting in a shift invariance over log-frequency. SI-PLCA then performs a multi-pitch detection with a frequency resolution higher than MIDI scale. Shifting of templates are performed by re-writing eq. 2.7 as

P (f |t) = i,δ f ,m P (f -δ f |i, m)P (m|i, t)P (i|t)P (δ f |i, t) (2.8)
where δ f is the pitch shifting factor. To constrain δ f so that each sound state template is associated with a single pitch, the shifting occurs in a semitone range around the ideal position of each pitch. Thus because we are using in this paper a log-frequency representation with a spectral resolution of 60 bins/octave, i.e. a 20 cent resolution, we have

δ f ∈ [-2:2].

EM-based model parameter estimation

Classical EM-based estimation

To estimate the model parameters P (m|i, t) and P (i|t), since there is usually no closedform solution for the maximization of the log-likelihood or the posterior distributions, iterative update rules based on the Expectation-Maximization (EM) algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF] are employed. We then follow the likelihood principle [START_REF] Mclachlan | Mixture Models[END_REF], which leads us to maximize the log-likelihood function of eq. 2.5, i.e.

L = f ∈F t∈T n(f, t)log(P (f, t)) (2.9)
where n(f, t) denotes the term frequency, i.e., the number of times f occurred in t. As stated by [START_REF] Hofmann | Probabilistic latent semantic indexing[END_REF], "a standard procedure for maximum likelihood estimation in latent variable models is the EM-algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]". EM alternates two steps: (i) an expectation (E) step where posterior probabilities are computed for the latent variables z, based on the current estimates of the parameters, (ii) an maximization (M) step, where parameters are updated for given posterior probabilities computed in the previous E-step.

For the Expectation step, we compute the contribution of latent variables i, δ f and m over the complete model reconstruction using Bayes' theorem as follows

P (i, δ f , m|f, t) = P (f -δ f |i, m)P (δ f |i, t)P (i|t)P (m|i, t) i,δ f ,m P (f -δ f |i, m)P (δ f |i, t)P (i|t)P (m|i, t) (2.10)
For the Maximization step, we utilize the posterior of eq. 2.10 for maximizing the log-likelihood of eq. 2.9, resulting in the following update equations:

P (δ f |i, t) = f,m P (i, δ f , m|f, t)X N (f, t) δ f ,f,m P (i, δ f , m|f, t)X N (f, t) (2.11) P (m|i, t) = f,δ f P (i, δ f , m|f, t)X N (f, t) m,f,δ f P (i, δ f , m|f, t)X N (f, t) (2.
12)

P (i|t) = f,δ f ,m P (i, δ f , m|f, t)X N (f, t) i,f,δ f ,m P (i, δ f , m|f, t)X N (f, t) (2.13) P (f |m, i) = δ f ,t P (i, δ f , m|f + δ f , t)X N (f + δ f , t) f,t,δ f P (i, δ f , m|f + δ f , t)X N (f + δ f , t) (2.14)
Alternating eq. 2.10 with eq. 2.11-2.13 defines a convergent procedure that approaches a local maximum of the log-likelihood in eq. 2.9. Typically 15-20 iterations are sufficient. This convergence towards a fixed point can be monitored by calculating the log-likelihood (eq. 2.9) associated with the model for each iteration.

Although this set of EM equations can be commonly found in specialized literature [START_REF] Hofmann | Probabilistic latent semantic indexing[END_REF][START_REF] Shashanka | Sparse overcomplete latent variable decomposition of counts data[END_REF]Benetos et al., 2013a), this EM-solved optimization problem coming with the parameter estimation of PLCA-based models has never been properly posed and justified. We then propose in our Technical note 1 to re-define the theoretical framework of this problem, with the motivation of making it clearer to understand, and more admissible for further developments of PLCA-based computational systems. The pitch activity matrix P (i, t) is deduced from P (i|t) with the Baye's rule

P (i, t) = P (t)P (i|t) (2.15)

Limits of EM-based estimation

The major limitation of current PLCA models lies in the inherent problems of the EM algorithm. This algorithm was originally introduced by [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF] to overcome the difficulties in maximizing likelihoods of missing data models. The main advantage of that method is its easy implementation, consisting of initializing the parameters and iterating expectation and maximization likelihoods in a step-by-step process until convergence. Its major drawback, besides the requirement of convex likelihoods, lies in its sensitiveness to initialization, which increase the risks to local convergences [START_REF] Robert | Monte Carlo Statistical Methods[END_REF]. That issue is exacerbated in the case of multimodal likelihoods. Indeed, the increase of the likelihood function at each step of the algorithm ensures its convergence to the maximum likelihood estimator in the case of unimodal likelihoods, but implies a dependence on initial conditions for multimodal likelihoods. Alternative techniques have also been proposed to optimize the search of global maxima, such as running the algorithm a number of times with different, random starting points, or using variants from the basic EM algorithm such as Deterministic Annealing EM (DAEM) algorithm [START_REF] Ueda | Deterministic annealing em algorithm[END_REF]. These theoretical issues have reached research fields working on audio signals.

To tackle the problem of dependency to initialization, some authors [START_REF] Grindlay | A probabilistic subspace model for multiinstrument polyphonic transcripton[END_REF]Benetos and Dixon, 2013) perform a training of the instrument templates, which has proved to be an effective way to initialise the spectral bases. Indeed, dependency to initialization should be a problem in the basic PLCA model when both basis functions and activations are estimated from a given magnitude spectrogram. If, however, one of the two is fixed, the estimation of the other is effectively a gradient descent method with a convex cost function which should converge against a global optimum. In other words, by fixing them without data-driven updating, we obtain a stable output for the gain function, independent of its initialisation. However, when the model becomes more complex with for example the introduction of different instrument variables, performing robust initialization is more difficult. For what concerns the local convergence problem, some works [START_REF] Hoffman | Finding latent sources in recorded music with a shift-invariant hdp[END_REF][START_REF] Grindlay | A probabilistic subspace model for multiinstrument polyphonic transcripton[END_REF][START_REF] Cheng | A deterministic annealing algorithm for automatic music transcription[END_REF]) have used the DAEM algorithm based on a temperature parameter. This limitation becomes particularly critical when integrating priors into the PLCA framework. Generally speaking, this integration introduces generic problems in optimization convergence to global maxima, especially when the prior has a multi-modality form. Indeed, when a prior is injected, the maximization step becomes a maximum a posteriori step and the log posterior probability needs to have the good properties for maximization. [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF] used of a numerical fixed point algorithms to solve the modified EM equations with a sparsity prior, whose convergence is only theoretically supposed, but "observed in practice" (although the sensitivity of the algorithm convergence to the evaluation sound dataset is not detailed). Benetos and Dixon (2013) privileged the use of pre-defined templates, which allows them to skip computing the EM update equation of templates, and just to apply a sparsity constraint on the pitch activity matrix and the pitch-wise source contribution matrix. Also, the simultaneous use of several priors on a same model parameter leads to some difficulties in terms of mathematical calculation and increases convergence problems [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF].

To overcome such limitations, two estimation algorithms are now proposed as alternatives of the classical EM algorithm, namely the Deterministic Annealing EM (DAEM) and the Filtering Particle (FP) algorithms.

Deterministic Annealing EM (DAEM) -based estimation

Deterministic Annealing EM (DAEM) is an extension of the algorithm EM, proposed by [START_REF] Ueda | Deterministic annealing em algorithm[END_REF], and already applied to PLCA by [START_REF] Cheng | A deterministic annealing algorithm for automatic music transcription[END_REF]. EM equations are modified to integrate a temperature coefficient β as follows

P (i, δ f , m|f, t) = P (f -δ f |i, m)P (δ f |i, t)P (i|t)P (m|i, t) β i,δ f ,m P (f -δ f |i, m)P (δ f |i, t)P (i|t)P (m|i, t) β (2.16)

Filtering particle -based model parameter estimation

State space representation

The PLCA model can be expressed as :

P (x, t) = z P (z, t)P (x|z) = z 1 ,...,z K P (z 1 , . . . , z K , t) J j=1 P (x j |z 1 , . . . , z K ) = z 1 ,...,z K P (z K , t) K-1 k=1 P (z k |z k+1 , . . . , z K , t) J j=1 P (x j |z 1 , . . . , z K ) (2.17) with : -z ∈ Z 1 × . . . × Z K is a vector of K latent components (z 1 , . . . , z K ) associated to a finite subset Z k = {1, . . . , L k } -t ∈ {0, . . . , T } is the time variable -x ∈ X 1 × . . . × X J
is a vector of J features (x 1 , . . . , x J ) where X j = {1, . . . , F j } In this decomposition, P (z K , t) can be seen as the activation distribution of the latent variable z K , P (z k |z k+1 , . . . , z K , t) as the weight of the variable z k conditionally to (z k+1 , . . . , z K ) and P (x j |z 1 , . . . , z K ) as the J features basis.

To estimate the set of parameters p t = {P (z K , t), P (z k |z k+1 , . . . , z K , t)∀k ∈ {1, . . . , K -1}} at each time t ∈ {0, . . . , T }, the model can be rearranged as a state space process

p t ∼ f (p t |p t-1 )
(2.18)

y t ∼ g(y t |p t ) (2.19)
where f is the transition state density function for p t defined above and g the observation function of y t .

Transition and observation densities

Transition density Assuming that each latent variable z k ∈ Z k is i.i.d, each marginal vector P (z K , t) and P (z k |z k+1 , . . . , z K , t) can be independently estimated. Recalling that at a given time t, P (z K , t) and P (z k |z k+1 , . . . , z K , t) represent distributions, Dirichlet priors are injected to ensure that their elements belong to [0, 1], as follows,

∀(z 2 , . . . , z K ) ∈ Z 2 × . . . × Z K , ∀k ∈ {1, . . . , K -1} P (z K , t) ∼ Dir(θ 1 t , . . . , θ L K t ) (2.20) P (z k |z k+1 , . . . , z K , t) ∼ Dir(δ k (z k , . . . , z K-1 ) 1 t , . . . , δ k (z k , . . . , z K-1 ) L K t ) (2.21)
where θ and δ k are random variables representing the weight of each component of z K in P (z K , t) and P (z k |z k+1 , . . . , z K , t). That injection leads to the following hierarchical model

H t → H t+1 ↓ ↓ P t P t+1 ↓ ↓ Y t Y t+1 (2.22)
with H t = (Θ t , ∆ t ) the new states defined by

Θ t = {θ z K t , ∀z K ∈ Z K } (2.23) ∆ t = {δ k (z k , . . . , z K-1 ) z K t , ∀z K ∈ Z K } (2.24)
where we have defined

θ z K t+1 = θ z K t × α z K t , α z K t ∼ φ (2.25) δ k (z k , . . . , z K-1 ) z K t+1 = δ k (z k , . . . , z K-1 ) z K t × γ z K t , γ z K t ∼ ψ k (2.26)
with φ and ψ k are positive distributions.

Observation density y t has been defined as a representation of x at time t. In that state space approach, each component of y t is represented by the sum of the PLCA model and a white noise, ∀x ∈ X 1 × . . . × X J ,

y t (x) = P (x, t) + V t = z 1 ,...,z K P t (z 1 , . . . , z K ) J j=1 P t (x j |z 1 , . . . , z K ) + V t (2.27)
where V t ∼ N (0, σ 2 ). Denoting ŷt the vector of components P (x, t), the observation density g follows a normal distribution, i.e. g ∼ N ( ŷt , σ 2 ).

Definition of spectral templates P (f |i, m)

Fixed parametric note model

Here, the spectral templates P (f |i, m) are extracted from isolated note spectra of each pitch using a one component PLCA, and are not updated during parameter estimation. In our baseline PLCA system, labelled B 0 in the following, note spectra are built synthetically with parametric harmonic note models. We model a note spectrum as a weighted sum of fixed narrowband harmonic spectra, called kernels, convolved with a pitch impulse distribution. Amplitudes of kernels are fixed, and set to decrease as 1 over the square of the harmonic number. This acoustic model intends to be as generic as possible in modeling plucked-string instrument sounds.

EM-updated parametric note model

In the PLCA model developed by [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF], at a given time t, the spectrum of the m th polyphonic source is decomposed as a weighted sum of different harmonic notes, each one having its own fundamental frequency and spectral envelope. These authors then use the algorithm EM to adapt these parametric kernel amplitudes, i.e. spectral envelop, to the input audio data.

Pre-recorded note samples

Kernels can be learned from pre-recorded isolated notes using a one component PLCA model, as performed by Benetos et al. (2013a). This prior makes use of a single template per pitch, selected as the one with the highest amplitude. These templates can be adaptively updated with input data, or kept fixed.

Data-based template adaptation & Conservative transcription

As a last strategy to define PLCA templates, the concept of conservative transcription for data-based template adaptation has been lately proposed [START_REF] Tidhar | High precision frequency estimation for harpsichord tuning classification[END_REF]Benetos et al., 2014a), using the EM update eq. 2.14. When updating templates on unlabelled data, one has to avoid that one latent variable is scattered into P (i|t) and P (f |i, m) for multiple bases, which consequently degrades the accuracy of these magnitudes, leading to an incorrect solution. To answer this problem, some authors [START_REF] Tidhar | High precision frequency estimation for harpsichord tuning classification[END_REF]Benetos et al., 2014a) have then introduced the concept of conservative transcription. This particular transcription first consists of identifying only those detected note events for which we have a high degree of confidence (i.e. the system returns few false alarms but might miss several notes present in the recording), and omitting any unsure candidates. This is performed with a PLCA model using a generic fixed template dictionary.

Note events are then extracted by thresholding the event activation matrix P (i, t) = P (t)P (i|t), and select only the time intervals T p whose activity values exceed the threshold T hres f ix (see Sec. 2.3). We will now use these intervals to learn the template dictionaries, adaptively to the input data. As before, a one-component PLCA is used for each time interval, taking as input X(f, t k ) with t k ∈ T p. The output for each latent component z is a spectral template P (f |z) which can be used in order to expand the present dictionary. Following Benetos et al. (2014a), this template adaptation can be controled by a parameter ε through the following equation (2.28) with w theo = P (f |z) corresponding to some initial template dictionaries. P t (z|f ) is the posterior of the model (defined in Benetos et al. (2013a)). Higher values of the parameter ε, which will be set to 0.02 here, allows increasing template adaptation. This parameter controls the levels of precision/recall: a low threshold has a high recall and low precision; the opposite occurs with a high threshold, which is done for our conservative segmentation.

P (f |z) = t εP t (z|f )X(f, t) + (1 -ε)w theo f,t εP t (z|f )X(f, t) + (1 -ε)w theo
We can now use these intervals to learn the template dictionaries corresponding to the different pitches, adaptively to the input data.

Note segmentation

Notations

This note segmentation stage allows estimating a subset of played notes Ĩ ⊂ I, and the time intervals U (i)

t k = [t k , • • • , t k + length(U (i) t k )] on which a note is played, for each pitch i ∈ Ĩ. Each interval U (i)
t k is indexed by its starting time frame t k . Based on these different intervals, we can compute the binary piano-roll transcription output P (i, t) ∈ {0, 1}, defined as

P (i, t) = 1 for t ∈ U (i) t k 0 otherwise

Monophonic transcription

A first simple note segmentation is to choose the most likely pitch in each frame by using the MAP estimator, ∀t

P (i, t) = 1 for i = argmax i {P (i|t)} 0 otherwise

Simple thresholding

Eventually, as in most spectrogram factorization-based transcription or pitch tracking methods (Grindlay and Ellis, 2011;[START_REF] Mysore | Relative pitch estimation of multiple instruments[END_REF][START_REF] Dessein | Real-time polyphonic music transcription with nonnegative matrix factorization and beta-divergence[END_REF], we use a simple threshold-based detection of the note activations from the pitch activity matrix P (i, t), followed by a minimum duration pruning. The threshold on note activations, is labelled T hres f ix and set to 0.6 by default. The threshold for minimum duration for pruning was set to 130 ms as in [START_REF] Dessein | Real-time polyphonic music transcription with nonnegative matrix factorization and beta-divergence[END_REF]. In comparative studies of AMT systems, the use of this simple thresholding method allows one to better highlight the differences between these different systems, or brought by the incorporation of different KCMA. The binary matrix P (i, t) is given by

P (i, t) = 1 for {i, t} = {i, t|P (i, t) ≥ T hres f ix } 0 otherwise

Adpative thresholding

In this paper, we first extracted onsets from P (i, t) using a pitch-wise energy-based Onset Detection Function (ODF), which computes a spectral flux on each activation band to emphasize local energy changes, followed by a half-wave rectification (Bello et al., 2005) so that negative peaks marking the offset of a musical event are discarded and only positive values are taken into account. Most ODFs are post-processed with a dynamic thresholding T hres dyn (t) to take into account the loudness variations of a music piece, and in our case the corresponding activation probabilities, which can be computed (adapted from Bello et al. (2005)) as T hres dyn (t) = T hres f ix + median(max(P (:, t -M )), ..., max(P (:, t + M )))

(2.29) with T hres f ix an offset coefficient. All these peaks are then stored, and unitary normalized over all peak values. An onset activation score is itself defined as the likelihood averaged over four temporal frames just after the frame where the onset has been located. Based on this score, we define the offset time of each note as the time it takes to the onset activation score to decrease by 80 % of its value. The binary matrix P (i, t) is given by

P (i, t) = 1 if {i, t} = {i, t|P (i, t) ≥ T hres dyn (t)} 0 otherwise

Hidden Markov Models

Theoretical Background

The two main questions to be addressed when developing Hidden Markov Models (HMMs) are 1. What are your observations ? From which one will compute observation probabilities ;

2. What is the supposed generative process of these observations ? From which one will compute transition probabilities between different states to explain the observations.

Markov Chain

Consider a system that is described by a set of N distinct states S k , where S k ∈ S = {S 1 , S 2 , ..., S Ns }. The states of the system may change with time, and at the time instants t, they are denoted by q t , a discrete-time random variable taking value in the finite set S. The dynamics of the system is described by a homogeneous first-order Markov chain, that is, when at time t the system is in state S i , there is a fixed probability that at time t, it will be in state S j , where the probability depends only on the state at time t-1. This Markov chain is then defined as follows:

P (q t = S j |q t-1 = S i , q t-2 = S k , ...) = P (q t = S j |q t-1 = S i ) = a ij (2.30) where A = [a ij ] is an N s × N s transition matrix, with a ij ≥ 0 and, N j=1 a ij = 1
Training Markov chains (of any order) from data is fairly straightforward if state values are directly observable, as they are for symbolic data such as text and musical scores. To obtain the Markov chain under which the observed data are most likely to have occurred, one simply sets the transition probability vector from each state to match the relative frequencies of each observed transition.

Hidden Markov Chain

Although simple Markov chains lend themselves well to applications involving symbolic data, to model continuous data such as feature vector sequences describing audio we need to add another layer of complexity. In such modeling scenarios, the state sequence is not directly observable, but hidden from the observer. Then Hidden Markov Model (HMM) assumes that each state is associated with an emission probability density function that generates our observed data. A HMM can then be seen as a doubly stochastic generative process, with two components: a set of hidden variables that can not be observed directly from the data, and a Markov property that is usually related to some dynamical temporal behaviour of the hidden variables.

In mathematical terms, at every time instant t, the system generates an observation y t according to a probability distribution that depends on the underlying state q t . If the number of distinct observations is N o and the set of observation symbols is υ = {υ 1 , υ 2 , ..., υ No }, the probability distributions of observed symbols are given by an N s × N o matrix B, whose elements b jk are known as emission probabilities and are defined according to

b jk = P (y t = v k |q t = S j ), 1 ≤ j ≤ N s and, 1 ≤ k ≤ N o (2.31) with b jk = No k=1 b jk = 1 (2.32)
Finally, to complete the specification of the model, one needs to provide the initial state distribution defined by Π = (Π 1 . . . Π Ns ), where Π i = P (q 1 = S i ). The three probability distributions described by A, B and Π are, in short, denoted by λ = {A, B, Π}. Typically, a common assumption for an observed sequence y T = [y 1 , y 2 , ..., y T ] is that its joint probability mass function conditioned on the state sequence q T = [q 1 , q 2 , ..., q T ] and the parameters λ is given by

P (y|q, λ) = T t=1 P (y t |q t , λ) (2.33)
which means conditional independence of the observations.

Sigmoid projection

The following sigmoid function f is applied to this observation distribution, in order to project non-negative data into the range [0 : 1],

f (x) = 1 1 + exp -x-λ (2.34)
where the parameter λ controls the smoothing strength, i.e. the higher its value, the more low probability values will be discarded. The use of this sigmoid function has already been proposed in the literature (Benetos and Dixon, 2013). We propose in our Technical note 2 a theoretical development generalizing the use of this sigmoid function to format observation distributions before note segmentation, relating its parameters to explicit music characteristics. For parameter estimation, we employ a GEM algorithm.

HMM problems

There are three basic problems related to HMMs [START_REF] Rabiner | A tutorial on hidden markov models and selected applications in speech recognition[END_REF], and in order of increasing complexity, they are the following:

1. Given a set of observations y T = [y 1 , y 2 , ..., y T ] and the model parameters λ, the objective is to find the probability of the observed sequence y, P(y|λ) ; 2. Given a set of observations y T = [y 1 , y 2 , ..., y T ] and the model parameters λ, the objective is to find the corresponding state sequence q ;

3. Given a set of observations y T = [y 1 , y 2 , ..., y T ], the objective is to find the state sequence q as well as the model parameters λ

The solutions to these three problems are well known [START_REF] Rabiner | A tutorial on hidden markov models and selected applications in speech recognition[END_REF]. The first problem can be solved efficiently by the forward-backward procedure, the second, by the Viterbi algorithm, and the third by the iterative method of BaumâĂŞWelch.

Viterbi resolution

We develop here the well-known resolution of the problem 2 described above, using the Viterbi algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]. It is very complex and difficult to directly model the joint probability density function of the observation sequence. A reasonable approach is to group nearby observations of similar characteristics as being produced by the same state and then consider how do the states progress and how does a state sequence produce the observation sequence. Hence, the model is split into two parts, the first part considers the probability of a state sequence and the second part considers the observation sequence based on a state sequence, and the joint probability distribution given by the model is written as, given a sequence of measurements y 1 , ..., y T and assuming a certain sequence of hidden states q 1 , ..., q T , P (q 1 , ..., q T , y 1 , ..., y T ) = P (q 1 )P (y 1 |q 1 )

T t=2 P (y t |q t ) • P (q t |q t-1 ) (2.35)
From this equation 2.35, we define the score of the candidate optimal partial path that, at time t, reaches state i δ(i, t) = max q 1 ,...,qt ln(P (y 1 , ..., y T , q 1 , ..., q t = i))

(2.36)

The most likely state sequence Q is then given by

Q = arg max i,t {δ(i, t)} (2.37)
which can be estimated by the Viterbi algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]. Once we have the initial model parameters, this algorithm is used to decode the training data and obtain the new optimal state sequences. Then, we can re-estimate the model parameters based on the new optimal state sequences. The decoding and re-estimation procedure is iterated until the likelihood score of the data converges or a prescribed number of iterations are reached.

Higher-Order HMM

So far, we have made the assumption that the future state of a process depends only on its single most recent state, in terms of state transition and output observation. Consequently, the feature vectors of consecutive sound frames belonging to the same state are independent identically distributed, and trajectory modeling (i.e., frame correlation) in the frame space is not included. Such modeling characteristics are often unreasonable and reveal limits of first-order HMMs, especially in regards to state duration modeling as they violate the high correlations among successive frame-wise states. Indeed, the piecewise stationary assumption does not precisely match the non-stationary nature of the actual sound process.

In regards to such limitations, it sometimes makes sense to take more than one previous state into account using a higher-order Markov chain to get a higher prediction accuracy. HO-HMMs can be defined by making transition probabilities depend on the last k states in addition to the most recent state. [START_REF] Ching | Higher-order markov chain models for categorical data sequences[END_REF] and [START_REF] Ching | Higher-order multivariate markov chains and their applications[END_REF] proposed the high-order (k th -order) Markov models. To a k th -order (k > 1) Markov chain, the present state relies not only upon the last one state, but also on the k-1 previous states. We can define a k th -order Markov chain as follows (2.38) HO-HMMs have received a great deal of interest to model frame state duration [START_REF] Mari | Automatic word recognition based on second-order hidden markov models[END_REF][START_REF] Ching | Higher-order multivariate markov chains and their applications[END_REF]. By definition, one can actually build more "memory" into states by using a high-order Markov model with a trajectory modeling within state sequences.

P (q t = S j |q t-1 = S i , q t-2 = S k , ...) = P (q t = S j |q t-1 = S i , ..., q t-k = S l ) = a i...l
Mathematically, if we consider that state transition depends only on the previous O d states and output depends only on the previous O e states, then the model can be thought as an HO-HMM of orders (O d , O e ). The joint probability of state and observation sequences of eq. 2.35 can then be re-written as P (q 1 , ..., q T , y 1 , ..., y T ) = P (q 1 )P (y 1 |q 1 )

• P (q 2 |q 1 )P (y 2 |q 1 , q 2 )

• P (q t |q t-O d , ..., q t-1 )P (y t |q t-c+1 , ..., q t )

• P (q T |q T -O d , ..., q T -1 )P (y T |q T -c+1 , ..., q T ) (2.39)
From eq. 2.39, we can see that there is an enormous number of parameters to be estimated (about N O d transition probabilities and N Oe probability density functions for an N-state HO-HMM). The corresponding transition matrix A is an N O d × N matrix. Obviously, the total number of independent parameters is N O d × (N -1), which grows exponentially with the order O d , so that it's impossible to achieve effective parameters estimation perfectly.

It is noteworthy that some researchers [START_REF] Dubnov | Using machine-learning methods for musical style modeling[END_REF] have shown that at very low orders -such as the second order or so-called bigram Markov models -generate strings that do not recognizably resemble strings in the corpus, while at very high orders, the model simply replicates strings from the corpus.

General applications in audio

In audio recognition systems, especially in speech, based on HMMs are very popular not only because of its modeling power but also because there are powerful mathematical tools that can be used to efficiently estimate the model parameters, calculate the likelihood scores, and guide the design of systems. Applied to music, the HMM paradigm is used to solve three main tasks, namely classification, segmentation and learning, which can be related to these three HMM problems mentioned above (see Sec. 2.4.1).

Problem 1 Given several candidate HMM models representing different acoustic sources (musical instruments in our case), the classification problem computes the probability that the observations came from these models. The model that gives the highest probability is chosen as the likely source of the observation.

Problem 2

The segmentation problem means finding the most likely sequence of the hidden states given an observation o 1 , ..., o T . HMMs have been used to address musical segmentation problems by several researchers (e.g. [START_REF] Raphael | Automatic segmentation of acoustic musical signals using hidden markov models[END_REF]; [START_REF] Aucouturier | Segmentation of musical signals using hidden markov models[END_REF]). These works dealt with segmentation of a sound into large-scale entities such as characterizing repetitive patterns [START_REF] Logan | Music summarization using key phrases[END_REF], with the purpose of performing tasks such as score following or identification of texture changes in a musical piece. In the same vein, HMMs have also been used for music structure analysis at a smaller scale, with harmonic analysis [START_REF] Raphael | Le timbre dans le répertoire de la valiha, cithare tubulaire de madagascar[END_REF] and chord estimation [START_REF] Bello | A robust mid-level representation for harmonic content in music signal[END_REF][START_REF] Lee | Acoustic chord transcription and key extraction from audio using key-dependent hmms trained on synthesized audio[END_REF].

Problem 3 Learning is the first problem that needs to be solved in order to use a HMM model, unless the parameters of the model are externally specified. It means estimating the parameters of the models, usually iteratively done by the EM algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF].

For what concerns the task of AMT, the sequential structure that may be inferred from musical signals can be usefully integrated to systems with HMMs. Indeed, as approaches consisting of a Multi-Pitch Estimation stage, or even classification approach [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF], display the obvious fault of processing each frame independently of its neighbors: the inherent temporal structure of music is not exploited. HMMs have then been used for three main tasks: note modeling, note segmentation and sequential post-processing. All these tasks fall within the second HMM problem defined above, and are now developed in the following sections. In this PhD thesis, we put a special focus on the last two ones, which will be used in Chapter 3 for KCMA incorporation.

Acoustic modeling of note events

HMMs have made use of the inherent temporal structure of audio and have shown to be particularly powerful in modeling sounds in which temporal structure is important, such as speech [START_REF] Rabiner | A tutorial on hidden markov models and selected applications in speech recognition[END_REF]. Two different approaches can be distinguished here. A first one which integrates HMMs directly into MPE methods, such as NMF [START_REF] Nakano | Nonnegative matrix factorization with markov-chained bases for modeling time-varying patterns in music spectrograms[END_REF] and PLCA [START_REF] Mysore | Relative pitch estimation of multiple instruments[END_REF]Benetos and Dixon, 2013). A second one which uses HMMs to model note events from the salience function of the system during a post-processing stage. [START_REF] Ryynanen | Polyphonic music trancription usng note event modeling[END_REF] and [START_REF] Ryynanen | Automatic transcription of melody, bass line, and chords in polyphonic music[END_REF] define a left-to-right HMM topology to model acoustically note events. [START_REF] Grosche | Automatic transcription of recorded music[END_REF] use two models for segmenting the semitone bands. An on model that captures properties of sounding notes and an off model which captures regions where no note is active. The on model exhibits three states with a left-right topology. This reflects the assumption that a note consists of an attack, decay, and sustain part.

P (y t |q t = S 1 ) P (y t |q t = S 2 ) S 1 S 2 S 1 a 22 a 11 a 12 a 21 b 2 b 1 Figure 2.
2 -On the left, graphical representation of transition probabilities between three hidden states. On the right, probability density functions for the observed data given the underlying state of the model y t is the observation at time t, and q t is the underlying state label at time t.

HMM-based note segmentation

As a replacement of the simple thresholding (see Sec. 2.3.3), HMM can be used to perform pitch-wise note segmentation from a salience matrix. This function of HMMs consists in a time filtering of note detection decision and generates smoothed note boundaries. We propose two HMM models for this task, the first order two-state on/off HMMs, developed by [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF], and an original one which extends this model by including note duration modeling through a higher order. Here, the hidden stochastic process for HMMs in AMT is associated to the activation matrix P (i, t) computed during the multi-pitch estimation stage.

First-order two-state on/off HMM

This model has been introduced in [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF]. Each pitch is modelled as a two-state on/off HMM, i.e. S i ∈ {0, 1}, which denotes pitch activity/inactivity. The state dynamics, transition matrix, and state priors are estimated from our "directly observed" state sequences, i.e. the training MIDI transcripts, which are sampled at the precise times corresponding to the analysis frames of the activation matrix. The initial probability P (q 1 ) for each note is supposed to be 1 for the off state, because all notes are inactive at the beginning of a recording. There are only four transition probabilities P (q t |q t-1 ), which correspond to the following state transitions: on/on, on/off, off/on, off/off. These probabilities are strongly dependent on the tempo of the considered composition. P (y t |q t ) is the observation probability, whose values are extracted frame-wisely from the PLCA activation matrix. Figure 2.2 provides a graphical representation of transition probabilities between two hidden states with this model, along with their observation distribution.

O d -Order two-state on/off HMM

In this HMM model, as we only have two states, S i ∈ {0, 1}, the left-to-right HMM topology allows for a simple modeling of their temporal dynamic through the duration that the last state has stayed. Through this model, we no longer consider consecutive states belonging to a same state as independent identically distributed, but aggregate them in groups according to the duration of staying in a state. The knowledge memorized in this modeling is then about note duration, and allows for an efficient reduction of the HMM parameters to be estimated. Mathematically, eq. 2.39 can be reduced to P (q 1 , ..., q T , y 1 , ..., y T ) = P (q 1 )P (y 1 |q 1 , d 1 (q 1 ))

• P (q t |q t-1 , d t-1 (q t-1 ))P (y t |q t , d t (q t ))

• P (q T |q T -1 , d T -1 (q T -1 ))P (y T |q T , d T (q T )) (2.40)
where d t (s t ) ∈ {1, ..., O d }, represents the duration that state q t has stayed up to time t, and is equal to O d when it exceeds the maximum dependency order O d . Figure 2.3 provides a graphical representation of transition probabilities between three hidden states with this model, along with their observation distribution. We now need to define the score of the candidate optimal partial, which will depend on state i and staying duration d, at a given time t. Such a score can be defined as, for 0

≤ d ≤ O d , δ(i, d, t) = max q 1 ,...,q t-d-1 ln(P (q 1 , ..., q t-d-1 , q t-d = i -1, q t-d+1 = • • • = q t = i, y 1 , ..., y T )) (2.41) and for d=O d , δ(i, D, t) = max q 1 ,...,q t-O d -1 ln(P (q 1 , ..., q t-O d , q t-D+1 = • • • q t = i, y 1 , ..., y T )) (2.42)
This HO-HMM model is actually equivalent to O d N-state first order HMM and can then be recursively solved with a Viterbi algorithm. Algorithm 2 presents this resolution for two on/off states S i ∈ {0, 1}. In this algorithm, the probability of transition from state i to itself after staying for d frames is denoted by ã(i, d). Briefly, in the initialization step (step 1) of the algorithm, the first frame is constrained to be in state S 1 =0 (i.e. a sequence begins with silence). For d=1 (step 6), the first entrance of state S 1 is from state S 2 with a stay time of 1 through O d , and reciprocally. For d > 2 (step 8), the previous frame must be at the same state with stay time equal to d-1. For d =O d (step 10), the first entrance of state S 1 is from state S 2 with a stay time at state O d -1, and reciprocally. In the termination step (step 14), the last frame is constrained to be at state S 1 =0, and it should be at the end boundary state after the last frame.

One major advantage of this algorithm is that it allows the complex topology of equivalent first order HMMs to be automatically learned from the training data. Initially, the training data are uniformly segmented according to the state number. The observation vectors belonging to the same state and staying time are grouped. From the uniformly segmented state sequence, we can accumulate the state transition counts and then estimate state transition probabilities. Let C(i,d), d ≤ O d , denote the times that state i has stayed d frames and let C(i,O d +) denote the times that state i has stayed for more than O d frames. Then, the state transition probabilities can be estimated by

ã(i, d) = C(i, d + 1) C(i, d) , 1 ≤ d < O d (2.43) ã(i, O d ) = C(i, O d +) C(i, O d ) (2.44) 1-a(i,1) a(i,1) a(i,2) S j,1 1-a(i,2) a(i,3) 1-a(i,3) S i,2 S i,3 b i (O) S i,1
P (y t |q t = S i,1 ) P (y t |q t = S i, 2)

P (y t |q t = S i,3 ) b i (O) b i (O)
Figure 2.3 -On the left, graphical representation of transition probabilities between three hidden states. On the right, probability density functions for the observed data given the underlying state of the model y t is the observation at time t, and q t is the underlying state label at time t.

Algorithm 2 Viterbi Algorithm adapted for a O d -order two-state on/off HMM

1: Initialization (t=1) 2: δ(1, 1, 1) = ln(b 11 (y 1 )) 3: δ(1, i, d) = -inf, for i ∈ {1, 2}, 1 < d < O d 4: for 1<t<T do 5:
for i, j ∈ {1, 2}, with i = j do 6:

δ(t, i, 1) = max 1<τ <O d δ(t -1, i -1, τ ) + ln(ã(j, τ )) + ln(b i1 (y t )) 7: for 2 < d < O d do 8: δ(t, i, d) = δ(t -1, i, d -1) + ln(ã(i, d -1)) + ln(b id (y t )) 9:
end for 10:

δ(t, i, O d ) = max O d -1<τ <O d δ(t -1, j, τ ) + ln(ã(i, τ )) + ln(b iO d (y t ))
11:

end for 12: end for 13: Termination (t=T) :

14: δ opt = max 1<d<O d δ(T, 1, d) + ln(ã(2, d))

Note mixture-based HMM for sequential post-processing

In this section, we develop an original HMM-based post-processing method, which consists in rejecting unlikely successions of note mixtures based on prior knowledge of polyphonic harmonic transitions. Instead of keeping a list of binary notes as in P (i, t), we will define probabilistic note candidates and further weight their respective likelihoods using a note mixture-based HMM method.

Definition of the note mixture event

In the note segmentation stage (Subsec. 2.3), we have defined the time intervals

U (i) t k
on which a note is played, for each pitch i ∈ Ĩ. Then, each appearance of a new note event, occurring at the different starting time frames t k ∈ T, will create a note mixture M t k , also indexed by t k . Before defining these mixtures, in order to reduce the number of defined note mixtures, we aligned the intervals U (i) t k whose starting times are closer than 46 ms (i.e. 4 time frames, see Sec. 5.1.3 for details on numerical values).. This way, we aim to identify more conveniently symbolic music units by aggregating individual estimations, and only keep the most significant grouping of notes.

Over the duration of a musical sequence, and after note segmentation, we obtain a list

M = {M t 1 , • • • , M t k , • • • , M t K },
with K the number of starting points in a musical sequence, and where each note mixture is indexed in time by its starting time t k . Each note mixture M t k is then parametrized by the vectors of size N Ĩ , defined as

Υ t k = [Υ t k (1), • • • , Υ t k (N Ĩ )] T
(2.45)

y t k = [y t k (1), • • • , y t k (N Ĩ )] T (2.46) Υ t k (•) contains binary values Υ ∈ {0, 1}
indicating if each pitch is played in the note mixture or not. y t k is the observation vector, where we defined the salience scores s t k (i) as the median of the pitch activity contained in the activation interval U (i) t k , i.e.

s t k (i) = median t∈U (i) t k (P (i, t)) (2.47)
Then, for a given test musical sequence, the set of HMM states will be formed as follows S = {S test , S train }, where

S test = [M t 1 , • • • , M t N S test
] T = , where the suffix = expresses the fact that the elements in this set are all distinct1 , with N S test ≤ K the total number of states. A supplementary set of HMM states S train from the training material is also added to this initial set. This operation can be performed as the observation probability for each note mixture is computed through their individual constitutive pitches. To extract these supplementary HMM states from training data, we adopt the strategy of selecting the mixtures that 1. are the most recurrent in the training material, identified through a simple frequency counting ;

2. share common pitches with the note mixtures from the test sequence. For example, a note mixture {D 3 , F 3 , A 3 , B 3 } in the initial set can bring the supplementary states {D 3 , F 3 , A 3 } and {D 3 , F 3 } from the training data

The numbers N S test and N S train are around 45, depending on the instrument repertoire and test sequences, and 20.

State-conditional observation likelihood

We then need to assign an observation probability likelihood to each M t k HMM state. The first step of this process consists in learning sample histograms of salience values respective to each pitch, and separating values from played notes and not-played notes, as illustrated in figure 2.4. These sample histograms are then smoothed using two analytical PDF models fitting their distributions, H (i) (s(i|i)) and H(i) (s(i| ī)), where s(i|i) and s(i| ī) refer to the salience values of the played and not-played notes of pitch i, respectively. The first one describes the observed salience values given that note of pitch i was played according to the annotation of the training material. Similarly, the second PDF describes the observed salience values when note was not played according to the annotation. Normal and log-normal distributions are respectively used, as follows

H (i) (s(i|i)) = 1 σ s(i) √ 2π e - (s(i)-µ s(i) ) 2 2σ 2 s(i)
(2.48) The state-conditional observation likelihood given a mixture M t k is then defined as

H(i) (s(i| ī)) = 1 s(i)σ s(i) √ 2π e - (ln(s(i))-µ s(i) ) 2 2σ 2 s(i) (2
P (y t k |q t k = M t k ) = N Ĩ i= Ĩ P (s t k (i)|Υ Mt k (i)) (2.50) = N Ĩ1 i= Ĩ1 N Ĩ2 j= Ĩ2 P (s t k (i)|Υ Mt k (i) = 1)
H (i) (st k (i)) • P (s t k (j)|Υ Mt k (j) = 0) H(j) (st k (j)) (2.51)
where Ĩ1 is the set of played pitches in the mixture, Ĩ2 the complementary set of notplayed pitches in the mixture (with N Ĩ1 + N Ĩ2 = N Ĩ). Eventually, at each starting time t k , the different observation probabilities are normalized so as to satisfy condition given by eq. 2.32,

P (y t k |q t k = M t k ) = P (y t k |q t k = M t k ) N S j=1 P (y t k |S j ) (2.52)
with S j ∈ S.

Guiding example

To complete this section, we propose a simple example to help understanding this method, as illustrated in figure 2.5. After learning sample histograms of salience values and their PDF models (step A), the note segmentation stage of the PLCA pitch activity matrix (step B) allows identifying the time frames during which a note is estimated to be played, filled with coloured rectangles, and defining the different time intervals

U (i)
t . The values of P (i, t) are also noted in each pitch-time frame. Then, at step C, we define the different note mixtures M t k , indexed by the starting times t k and the set of vectors Υ,

Υ = [Υ t 1 , • • • , Υ t 4 ] T =    1 0 1 0 0 1 0 0 0 0 0 1    (2.53)
We can then compute the salience values s t k (i) according to eq. 2.47. Eventually, the state-conditional observation likelihood is computed according to eq. 2.50, as detailed at step D.

Conclusion

This chapter has presented the baseline statistical methods we used to perform Automatic Music Transcription (AMT), which are mainly based on Probabilistic Latent Component Analysis (PLCA) and Hidden Markov Models (HMMs). PLCA is a probabilistic subspace analysis method which can be used for decomposing audio spectrograms. HMMs belong to dynamic Bayesian networks, and are especially known for their applications in modeling and recognition of time series. These two statistical methods use Bayes-based probabilistic frameworks, which allow powerful modelings with the incorporation of external musical knowledge. Put in combination, they cover both the time and frequency domains, a necessary condition to model music signals. For example, P (y

t1 |M t1 ) = H (1) (s t1 (i = 1)). H(2) (s t1 (i = 2)). H(3) (s t1 (i = 3)) = H (1) (0.8). H(2) (0.2). H(3) (0.15) = i=1,2,3 P (s t1 (i)|Υ t1 (i)) = 0.01 . 0.04 . 0.05 = 2.e -5
For example, y t1 = s t1 (i| ī)) = median([0.2, 0.1, 0.2, 0.1]) = 0.15, for i = 3

.9

Υ t3 Υ t4 t = 1 t = 5 U (i=1) t=1 U (i=3) t=5 U (i=2) t=5
.8 8 8

. Chapter 3

Incorporating Knowledge Components from Musical Acoustics in AMT systems

Abstract

In this chapter, we develop methods to incorporate our Knowledge Components of Musical Acoustics (KCMA), as identified and extracted in chapter 1, in the baseline statistical methods described in chapter 2. In the PLCA framework, we incorporate frequencydomain knowledge, which acts as priors by constraining parameter estimation, and in the HMM framework, we incorporate time-domain knowledge in a post-processing stage.

Introduction

Background

A musical signal is highly structured, in both time and frequency domains. In time domain, tempo and beat specify the range of likely note transition times. In the frequency domain, as audio signals are both additive and oscillatory (musical objects in polyphonic music superimpose and not conceal each other), several notes played at the same time form chords, or polyphony 1 , merging their respective spectral structures. When designing priors for an AMT system, one basically aims to help the system figuring out "which notes are present at time t" and "by which ones they will be followed". These two types of information belong respectively to frame-wise spectral priors (e.g. sparseness, spectrum modeling including inharmonicity [START_REF] Rigaud | Does inharmonicity improve an nmf-based piano transcription model ?[END_REF]) and to frame-to-frame temporal priors (e.g. harmonic content transitions, smoothing of spectrum envelop).

Many previous works [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF]Grindlay and Ellis, 2011;Benetos and Dixon, 2013) on AMT have used sequential priors to model each pitch activity/inactivity phases, which is done using two-state on/off HMMs for each of them during a postprocessing stage. This operation performs a time filtering of note detection decision, which mainly avoids a lot of single miss errors and smooths note boundaries. But musically, the information is very restricted, as it consists only in knowing how long a given pitch note remains active, which can result from both playing techniques of the musician and vibratory properties of the instrument.

In principle any acoustic or score-related information that can facilitate the transcription process can act as prior information for the system. However, to be of use in a practical application, it is important that it does not require too much time and effort, and that the required information can be reliably extractable by the user, who might not be an expert musician. Depending on the expertise of the targeted users, information that is easy to provide could include key, tempo and time signature of the piece, structural information, information about the instrument types in the recording, or even asking the user to label a number of notes for each instrument. Although many proposed transcription systems -often silently-make assumptions about certain parameters, such as the number or types of instruments in the recording, not many published systems explicitly incorporate prior information from a human user.

In the context of source separation, [START_REF] Ozerov | A general flexible framework for the handling of prior information in audio source separation[END_REF] proposed a framework that enables the incorporation of prior knowledge about the number and types of sources, and the mixing model. The authors showed that by using prior information, a better separation can be achieved than with completely blind systems. A system for user-assisted music transcription was proposed in [START_REF] Kirchhoff | Shift-variant nonnegative matrix deconvolution for music transcription[END_REF], where the user provides information about the instrument identities or labels a number of notes for each instrument. This knowledge enabled the authors to sidestep the error-prone task of source identification or timbre modelling, and to evaluate the proposed non-negative framework in isolation.

Two main families of knowledge incorporation

Knowledge is commonly incorporated in retrieval methods to better fit the decomposition to specific properties of input data. These properties can be either physics-based or signal-based. In the first case, knowledge can be explicitly modeled and incorporated in the statistical framework.

Signal-based

For instance, when isolated note recordings are available, on the same instrument and with the same recording conditions, the spectra of the dictionary can be learned independently [START_REF] Niedermayer | Non-negative matrix division for the automatic transcription of polyphonic music[END_REF][START_REF] Dessein | Real-time polyphonic music transcription with nonnegative matrix factorization and beta-divergence[END_REF]. In that case, since the dictionary is learned on monophonic data and fixed at the learning step, high transcription performances can be obtained.

Physics-based

For example, note templates can be parametrized by some prior information on harmonicity [START_REF] Bertin | Enforcing harmonicity and smoothness in bayesian non-negative matrix factorization applied to polyphonic music transcription[END_REF][START_REF] Hennequin | Time-dependent parametric and harmonic templates in non-negative matrix factorization[END_REF], temporal evolution of spectral envelop [START_REF] Hennequin | Score informed audio source separation using a parametric model of non-negative spectrogram[END_REF], sparsity of simultaneously activated notes [START_REF] Hoyer | Non-negative matrix factorization with sparseness constraints[END_REF], beat structure [START_REF] Ochiai | Explicit beat structure modeling for non-negative matrix factorization-based multipitch analysis[END_REF], etc., which is used to model the dictionary or the time-activation matrices.

Objectives & Plan

In this chapter, we make the link between the music related knowledge detailed in chapter 1, and listed as KCMA in table 1.2, and the algorithms for AMT in which they can be incorporated. The resulting overall transcription system is then of probabilistic nature, with a cascade of note probability weighting, delaying the final decision made on the estimation of active notes. In the first section 3.2, we describe the different theoretical methods used for the incorporation of our KCMA into our baseline methods for AMT. The first two methods incorporate knowledge as priors, which apply a direct constraint on the PLCA parameter estimation through modifications of EM equations. The next three methods incorporate KCMA in a post-processing stage, re-weighting candidate note probabilities in the activation matrix before note segmentation. It is noteworthy that in recent literature [START_REF] Sigtia | An rnn-based music language model for improving automatic music trancription[END_REF][START_REF] Sigtia | A hybrid recurrent neural network for music trancription[END_REF], the expression of "re-transcription" has been used, with the computation of priors from a first transcription step, and using it in a second transcription step by re-weighting model parameter estimation in EM equations. Our processing stage is different from this approach in the sense that we do not go through the EM algorithm again to re-weight pitch probabilities. Eventually, in section 3.4 we list all KCMA implemented in our AMT baseline system. They are categorized accordingly to their respective datasound sources, as already done in Sec. 1.

Mathematical prior incorporation

Sparse entropy-like prior in PLCA

As mentioned in Sec. 2.2, PLCA has the advantage that it enables to use a priori knowledge of domains. From the very first works on PLCA [START_REF] Shashanka | Sparse overcomplete latent variable decomposition of counts data[END_REF]Smaragdis and Mysore, 2009;Smaragdis et al., 2009), sparseness assumptions have been introduced in the PLCA framework as "entropic priors" to PLCA. The most generic mechanic to introduce priors in PLCA models is through the use of a Dirichlet distribution, which is a conjugate prior distribution to the PLCA multinomial distributions P (f |z) and P (t|z) (Smaragdis and Mysore, 2009). It can then be used to constrain the structure of the model distribution. The priors P r for all the frequency distributions Λ f , and temporal distributions Λ t , are

P r(Λ f ) ∼ z f P (f |z) κzα(f |z) (3.1) P r(Λ t ) ∼ z t P (t|z) µzα(t|z) (3.2)
where α(•) are the positive and real hyperparameters defining the Dirichlet distribution. The weight scalars κ z and µ z can be interpreted as the prior strengths. Using the above priors, our EM estimation equations for P (f |z) and P (t|z) (eq. 2.13 and 2.14, respectively) now change to:

P (f |m, i) = δ f ,t P (i, δ f , m|f + δ f , t)X N (f + δ f , t) + κ z α(f |m, i) f,t,δ f P (i, δ f , m|f + δ f , t)X N (f + δ f , t) + κ z α(f |m, i) (3.3) P (i|t) = f,δ f ,m P (i, δ f , m|f, t)X N (f, t) + µ z α(t|m, i) i,f,δ f ,m P (i, δ f , m|f, t)X N (f, t) + µ z α(t|m, i) (3.4)
The first prior type will impose a sparsity on P (f |m, i) so that each spectral basis component consists of only a few bins on the spectrum, while the second prior will impose a sparsity on P (i|t) so that only a few pitches i are active in the same time. A third type of entropy-like prior exists, imposing a sparsity between spectral basis components so that spectral basis components P (f |m, i) for different pitches i are not similar to each other. This prior takes the form of a cross-entropy defined by (Smaragdis et al., 2009)

H({P k } k , {Q k } k ) = - k P k logQ k - k Q k logP k (3.5)
which measures the similarity between the two distributions {P k } k and {Q k } k on their common dimension, and will be used as follows in our context

P r = β i,i |i =i H({P (f |i)} f , {P (f |i )} f ) (3.6)
where we defined the parameter β is the prior strength. These sparseness prior is added to the general likelihood eq. 2.9 as a cost function Pr:

L = f ∈F t∈T n(f, t)log(P (f, t)) -P r (3.7)
which leads, following the classical EM resolving procedure, to this template update equation

P (f |i) = B(0, t P (i|f, t)X N (f, t) - i=i P (f |i )) (3.8)
where

B(β, γ i ) = (i) I-1 i =1, =i (i )
(3.9) and

(i) = βγ i + αG(β, γ i ) α (α -1) I-1 i =1, =i G(β, γ i ) α (3.10)
The estimation of eq. 3.8 is computed iteratively with the two equations 3.9 and 3.10 until convergence. As already mentioned for the concept of conservative transcription, these sparseness assumptions also allow to avoid that one pitch is scattered into P (i|t) and P (f |i) for multiple bases, which consequently degrades the accuracy of P (i|t) and that of P (f |i), leading to an incorrect solution. For our AMT application, in particular this inter-basis sparseness prior is important to impose a discriminability between the different pitches.

Inter-pitch sparse prior in PLCA

The inter-pitch matrix ι (eq. 1.1) is injected in the PLCA framework by constraining the impulse distribution coefficients P (i, t, m) with the term

P (θ) ∝ exp(-θ ι * θ)
where ι * is obtained by, ∀i, j, ι * (i, j) = 0 for i = j Π(ι i,j ) otherwise (3.11) with the operator Π defined as Π(x) = 1 -x max(x) (3.12) which defines an information rejecting the hypothesis of certain pitch combinations, and where ι * is then the matrix M whose diagonal coefficients have been set to 0. It is noteworthy that simpler modeling of prior knowledge, through for example a pitchdependent vector, can also take the form of a diagonal matrix of size I 2 , with the vector values put into this diagonal (the zero-coefficients of ι provide an unitary prior value which does not affect particle weights). When developing eq. 3.2.2, we get the following form

P (θ) ∝ exp(- N I i=1 N I j=1 ι * (i, j)θ i θ j ) (3.13)
The usual maximization step is replaced by a maximum a posteriori (MAP) process, i.e. instead of maximizing the log expected likelihood Q Λ , the log posterior probability D(θ) = Q Λ + ln(P (θ)) is maximized. This maximized function is defined by

D(θ) : Ω =]0, 1[ N I → R (3.14) D(θ) = N I j=1 w j ln(θ j ) - N I i=1 N I j=1 ι * (i, j)θ i θ j (3.15) with w j = f,z V f t P (i, s, z, c = h|f, t).
Working with probability distributions, the following system is obtained :

max D(θ) (3.16) u.c.g(θ) = N I j=1 θ j = 1 (3.17)
A maximum θ exists since D is continuous and upper bounded by 0 on Ω. To maximize that function under the constraint g(θ) = 1, we introduce a Lagrangian system :

L(θ, λ) : Ω × R → R (3.18) (θ, λ) → D(θ) -λ(g(θ) -1) (3.19)
Moreover, the maximum θ verifies the first order necessary conditions proper to local maxima (Lagrange theorem) since D and g are both differentiable According to these two points, there exists a unique λ ∈ R such that :

∇L( θ, λ) = 0 (3.20)
The previous equation (eq. 3.20) leads to, ∀j ∈ {1,

• • • , N I }, θj = w j N I i=1 ι * ij θi + λ (3.21)
There is no closed-form solution for each θ j , but following [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF], a fixed point algorithm can be used to find a solution increasing the posterior probability at each iteration of the EM algorithm. Numerical simulations showed that the fixed point Algorithm 1 always converges to a solution that makes the posterior probability increase from one iteration of the EM algorithm to the next. Algorithm 3 Fixed-point method for the mutual resonance prior

1: ∀j ∈ {1, • • • , N I }, θj ← w j k θ k ; 2: while Convergence not reached do 3: ∀j ∈ {1, • • • , N I }, r j ← k ι * jk θ k 4:
Find the unique λ such that j w j r j +λ = 1 and ∀j,

w j r j +λ ≥ 0 5: ∀j ∈ [1, J], θj ← w j r j +λ
6: end while

A Bayesian estimation of frame-wise note number

At a given time t and playing mode m, conditionally to the data y = P (i, t), let's define the posterior distribution P (θ = k|y) of the impulse distribution vector P (i, t, m). Thanks to Bayes rule, and ∀k ∈ {1, • • • , d max }, we have

P (θ = k|y) ∝ P (y|θ = k)P (θ = k) (3.22)
where d max is the maximum degree of polyphony and P (θ = k) = (α 1 , • • • , α dmax ) identifies to the distribution of the polyphonic degree associated to prior P 1 . In a statistical framework, θ follows a multinomial distribution in {1,

• • • , d max } associated to probabil- ities {α 1 , • • • , α dmax }. Then, as P (y|θ = k) is composed of the k greater components (y * 1 , • • • , y * k )
of y, we simply compute it as the probability P (y|θ = k) = P (y * 1 , • • • , y * k ), i.e. the activity scores (s * 1 , • • • , s * k ) of the corresponding pitches, as y = P (i, t). Considering the independence assumption between pitch activations, the posterior distribution 3.22 can be rewritten in

P (θ|y) =              α 1 s * 1 . . . α k k i=1 s * i . . . α dmax dmax i=1 s * i             
Then, the degree of polyphony at a given time t is the k th element of θ which maximizes P (θ|y).

Modifying activation temporal profiles

We propose here a method to incorporate the KCMA ∼ IsoN o1 and IsoN o2, respectively on note duration and amplitude energy modulation. These descriptors have been computed on the sound template sets of each instrument, to get for each pitch i the prior parameters ∆ i and β i (defined as the median of the descriptor values for the sound samples of pitch i, and normalized by the maximum of all values). The prior parameter ∆ i is used as a pitch-specific minimum duration for pruning, as defined in step 3 of the pseudoalgorithm 4. β i is used in an acoustics-informed smoother function of pitch-wise activation intervals U i,k , indexed by k. Following the pseudo-code 4, this smoother computes within each activation interval, the minimum/maximum filter defined as

P (i, U i,k (n)) = min(P (i, U i,k (Int))) if median(P(i,U i,k (Int))) ≤ Θ max(P (i, U i,k (Int)))
otherwise (3.23) with Int = [n -η/2 : n + η/2] and η defining the temporal window of activity, and Θ the threshold ordering the switch between minimum and maximum filters. These two parameters define the smoothing strength. The higher is η and the lower is Θ, the smoother are the resulting activation intervals. These two parameters are constrained to range from [2 : min(10, length(U i,k ))] frames and [0.1 : 0.9max(P (i, U i,k ))], respectively. Their positions in these ranges are linearly indexed by the value of β i , so that the higher is β i , the smoother are the resulting activation intervals (i.e. the higher is η and the lower is Θ). Also, the value of 250 ms is learnt from the training, defined as the minimal note duration so as a significant AmpM od degree appears. Figure 3.2 illustrates the application of this intermodulation-informed smoother on the activation of an intermodulated note.

Note mixture-based HMM in post-processing

First-order N Mn -state HMM for harmonic transitions

This model follows a classical first-order HMM with N Mn states. Figure 3.3 provides a graphical representation of transition probabilities between three hidden states with this Find pitch-wise intervals U i,k , such that median(P (i, U i,k )) > 0.4 ;

3:

for Each interval U i,k , do % Minimal Duration Pruning P (y t |q t = S 3 ) Figure 3.3 -On the left, graphical representation of transition probabilities between three hidden states. On the right, probability density functions for the observed data given the underlying state of the model y t is the observation at time t, and q t is the underlying state label at time t. model, along with their observation distribution. In this model, transition probabilities are defined as the probability to switch between two successive mixtures of notes in a musical piece, simply computed by counting their occurrence frequency. These probabilities give us a global view of the usual and unusual harmonic transitions of an instrument repertoire, and are trained using MIDI scores. Despite the size of the learning database, novel note transitions may appear in test sequences without being trained, which will then not have estimated likelihoods. The distribution values must be smoothed to address tiny likelihood values for the note transitions that were not found in the learning database. Then, the Witten-Bell discounting algorithm [START_REF] Witten | The zero-frequency problem: estimating the probabilities of novel events in adaptive text compression[END_REF] can be used to perform this smoothing. The algorithm is based on a principle of zero-frequency N-grams, i.e., the Ngrams that have not yet occurred in the database. The zero-frequency N-gram likelihoods, however, may be estimated by the N-grams that have occurred once. Consequently, the zero-frequency likelihoods may be used to smooth the distribution.

4: if length(U i,k ) ≤ ∆ i then 5: Pruning U i,k 6: end if 7: if length(U i,k ) ≥

Second-order N Mn -state HMM for harmonic transitions

We used the extended second-order Viterbi algorithm already published in the literature [START_REF] He | Extended viterbi algorithm for second order hidden markov process[END_REF].

Prior incorporation in the filtering particle framework

Background

In mathematical terms, priors are used to sharpen up estimation of model parameters by emphasizing the most likely values in their distributions. This prior integration is performed during state generation, by re-weighting each particle value with a corresponding prior gain. Adding prior knowledge on parameters p t leads up to sample from the posterior distribution, P (p t |y t ) ∼ P (y t |p t )P (p t ) (3.24) where P (y t |p t ) identifies to the observation density g and P (p t ) the prior knowledge. When the prior and the likelihood are conjugate, sampling from the posterior distribution is rather straightforward. When the posterior does not have a well known form, as it is the case in most real-life applications, computational statistics methods can be introduced to sample from the posterior. The Metropolis-Hasting algorithm [START_REF] Roberts | Weak convergence and optimal scaling of random walk metropolis algorithms[END_REF][START_REF] Newman | Monte Carlo Methods in Statistical Physics[END_REF][START_REF] Robert | Monte Carlo Statistical Methods[END_REF], based on Monte Carlo methods, brings a powerful framework to tackle that issue. This algorithm is a random walk that uses an acceptance/rejection rule to converge to the specified target distribution, and proceeds as described in the algorithm 5.

Algorithm 5 Metropolis-Hasting algorithm for prior integration.

1: Draw a starting point p 0 t , for which P (p 0 t |y t ) > 0, from a starting distribution p 0 (p t ) ; 2: for q = 1, 2, . . . do

3:

Sample a proposal p * t from a jumping distribution at iteration q, J q (p * t |p q-1 t ) ;

4:

Calculate the ratio of densities,

r = P (p * t |y t )/J q (p * t |p q-1 t ) P (p q-1 t |y t )/J q (p q-1 t |p * t ) (3.25) 5:
Set

p q t = p * t
with probability min(r, 1) p q-1 t otherwise (3.26) 6: end for Considering the filtering particle framework defined above, few remarks about the different steps can be highlighted. First, the initial draw is replaced by the PF draw we want to interfere in. Even if the posterior distribution is unknown, the ratio r can be computed as the ratio of the product of the likelihood and the prior since the normalization constant is removed in the ratio.

r = g(y t |p * t )p(p * t )/J q (p * t |p q-1 t ) g(y t |p q-1 t )p(p q-1 t )/J q (p q-1 t |p * t ) (3.27)
The jumping distribution J q is chosen as a normal distribution to simplify the ratio computation. Indeed, the symmetry property of the normal distribution involves that J q can be removed in eq. 3.27, to get

r = g(y t |p * t )p(p * t ) g(y t |p q-1 t )p(p q-1 t )) (3.28)
The PLCA-PF framework allows a general insertion of the matrix ι through the term P (p t ) of eq. 3.24, to which we can give the following form

P (p t ) ∝ exp(-p t ιK pt ) (3.29)
where p t ∈ {A t , B t (s), P r t (s, δ f )}, ∀(s, δ f ) ∈ {1, . . . , S} × {1, . . . , ∆ f }, and K pt is a vector of length N I associated to p t .

Sparse priors

During the multi-pitch estimation step of an AMT process, a too much large number of non-zero activation scores is often observed, making the operation of "finding the right notes" more difficult. In order to overcome this flaw, a sparseness prior can reduce the number of active notes per frame in selecting the most salient ones. Previous works mostly use pitch-independent sparse prior in PLCA-EM algorithms. [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF] compute a sparseness prior P (A t ) to constrain the impulse distribution A t , as follows

P (A t ) ∝ exp -2β √ J||A t )|| 1/2 (3.30) with ||A t || 1/2 = i A t (i)
and β a positive hyperparameter indicating the strength of the prior. With this prior, a numerical fixed point algorithm is required to obtain a solution with the EM algorithm. Other works Grindlay and Ellis (2011); Benetos andDixon (2011, 2013) impose sparsity on the pitch activity matrix and the pitch-wise source contribution matrix by modifying EM equations. The common point to all these EM-based sparse priors is that they are pitch-independent, and rely on hyper-parameters, which are either arbitrary set and/or optimized on a given sound dataset. In this PhD, we define sparse priors informed by explicit musical acoustics related knowledge.

Eventually, for a sparse type prior ι spa , the set of prior parameters p t in eq. 3.29 is equal to A t and K pt is set to A t , which becomes

P spa (A t ) ∝ exp(-A t ι spa A t ) (3.31)

Sequential priors on harmonic transitions

Such a sequential prior P tra is represented through a matrix ι tra . Eventually, for this sequential type prior, p t in eq. 3.29 is equal to A t and K pt is set to A t-1 , which leads to P tra (A t ) ∝ exp(-A t ι tra A t-1 ) (3.32)

Prior combination

The PLCA-PF framework offers an easy-to-implement unifying way of integrating priors from both time and frequency domains. In this framework, priors are injected during the filtering process through eq. 3.24, and modify the parameters without disturbing their generation. In the set of parameters p t , the independence between each parameter p

List of the KCMA implemented in our AMT systems

We complete this chapter by listing the different combinations of KCMA (Chap. 1)baseline AMT methods (Chap. 2) -KCMA incorporation methods (Chap. 3), resuming the different families of KCMA presented in Chapter 1.

From theoretical concepts (KCMA source I)

In statistical modeling methods based on learning data, the problem is that most often typical learning music databases are very small compared to the complexity of the polyphonic music signal. The undesirable effect is that different instances of a given note mixture can be mapped to different available states, usually those that are initialized closely. If we had an infinite amount of data, we could simply represent each note mixture as a distinct observation. To compensate for this, theoretical concepts, or general modelbased knowledge, can be incorporated in the statistical framework for recognition of note mixture progressions, in order to redistribute efficiently a certain amount of probability mass to unseen events during training. The KCMA from theoretical concepts (Source I, from Sec. 1.1.1) we implemented are detailed in the second column from the left of table 3

From isolated note samples (KCMA source II)

The KCMA from isolated note samples (Source II, from Sec. 1.1.1) we implemented are detailed in the second column from the left of table 3.2. There are 7 such KCMA in this PhD project. In the following, we make specific comments for a few. T 1 : Pitch range The pitch range of an instrument, given in a semi-tone spacing, is directly defined by the extrema of an instrument pitch range. For the marovany, taking the range extrema of our different models N 1 to N 4 , we selected the following pitch range : from F 3 to D 6 , with a semi-tone spacing, which gave us a range of 34 notes. By default, an AMT system would consider a much larger pitch range covering the 5 to 6 octaves of instruments such as the piano or the harp. Algorithmically, the knowledge of pitch range is set as the range of spectral basis in the PLCA algorithm. ) extension of the PLCA, exploiting the fact that in a CQT, a change of fundamental frequency is traduced by a simple frequency translation of its partials, resulting in a shift invariance over logfrequency. SI-PLCA then performs a multi-pitch detection with a frequency resolution higher than MIDI scale.

Transcripts from playing (KCMA source III)

The KCMA from transcripts from playing (Source III, from Sec. 1.1.1) we implemented are detailed in the second column from the left of table 3.3. There are 6 such KCMA in this PhD project. In the following, we make specific comments for a few.

M 1 : Polyphonic degree As already mentioned, polyphony is due either to an "acoustic polyphony", for instruments whose repertoires do not have a vertical writing properly speaking (e.g. the marovany repertoire with its fast arpeggios and strong mutual resonances), or a polyphonic writing (e.g. contrapuntal Bach pieces). The former has already been characterized by the prior T 4 , and the prior M 1 deals with the second. M 2 : Knowing the played notes When performing a musical piece, an instrument only plays pitches from a reduced part of its pitch range. This information of knowing which notes are actually played can be used as a simple prior in our AMT system, which is directly used to define the set of PLCA spectral basis which can be activate during parameter estimation, all others receiving a null activity score. M 3 : Probabilistic transition between note mixtures This prior is based on the first-order HMM detailed in Sec. 3.2.5, whose states are defined with the different note mixtures M k identified after note segmentation. The sequential post-processing we propose now consists in correcting the succession of M k based on some knowledge of polyphonic harmonic transitions.

M 6 : Second-order N c -state HMM for harmonic transitions We used the extended second-order Viterbi algorithm already published in the literature [START_REF] He | Extended viterbi algorithm for second order hidden markov process[END_REF]. Only now, training of state transition probabilities implies counting two-state sequences.

Conclusion

In this chapter, we were interested in developing a generic transcription system based on a PLCA post-processed with a HMM block, which makes possible to supply it with all kind of prior knowledge at different levels of abstraction.

Chapter 4

Multichannel capturing sensory systems for transcription ground-truth creation Abstract

In this chapter we propose a method of automating the daunting task of providing the machine-learning models with labeled data for training and evaluation. This method is based on the technology of Multichannel Capturing Sensory Systems (MCSSs), which allow decomposing a multi-source audio signal into simple identifiable components, and simplifying more particularly the complex analysis of a polyphonic sequence by processing individually several monophonic sequences. A large amount of reliable ground truth can thus be provided thanks to this automatic generation process. This is a crucial step in the investigation of automatic transcription of orally transmitted music repertoires, including the marovany zither one.

Introduction

Background on MCSSs

In this chapter, we explore the use of Multichannel Capturing Sensory Systems (MC-SSs) (also named "divided pickup" in specialized literature) for AMT of acoustic plucked string instruments. MCSSs are made of several pickup devices, one for each string. A pickup device is a transducer that captures mechanical vibrations from stringed instruments (e.g. the electric guitar or electric violin), and converts them to an electrical signal that is amplified and recorded. A MCSS is then capable of picking up signals from each individual string and outputting these signals individually. This information allows to detect accurate performance information for each individual string, and supports analysis of an instrument performance that would be challenging to extract and decouple from the audio signal. Efforts made in the development of capturing sensory retrieval systems aim to select the signal type which provides the best support to extract the basic note parameters necessary for music transcription. Additionally, since the output signal is analog -the same as traditional pickups -it faithfully conveys all guitar-playing techniques. For the AMT task, such a system has an obvious advantage in this application, as it allows to break down a polyphonic musical signal into the sum of monophonic signals respective to each string. Then, we come back to a monophonic transcription problem, which can be understood as a special simple case of polyphonic transcription, and is considered as practically solved (Klapuri, 2004b).

Among the pioneer works towards the development of MCSSs, we can mention the Gittler guitar, an experimental guitar with six pickups, one for each string. Some time after, Gibson created the HD-6X Pro guitar with The Hex Pickup that captures a separate signal for each individual string and sends it to the onboard analog/digital converter. The output can be routed as a single summed mono signal to an amplifier or recording console. It can also send the E, A, and D strings to one amp or recording channel and the G, B, and high E to a separate amp or channel. Or it can send the output of all six individual strings to six different amps or channels. These six individualized outputs can used to create various effects.

First projects developing MCSSs dedicated to AMT for acoustic instruments go back some time. In the 1980's, Trimpin designed a system to capture which fingers were pressing which key on a grand piano. Currently one of the most robust systems to capture this information is commercially available. It is the Piano Bar, designed by Don Buchla in 2002 and now sold by Moog Music. It captures the full range of expressive piano performance by using a scanner bar that lies above any 88-key piano, gathering note velocity as well as a pedal sensor which gathers a performer's foot movement. Researchers at Osaka University in Japan designed a system for real-time fingering detection using a camera-based image detection technique, by coloring the finger nails of the performer [START_REF] Takegawa | Design and implementation of a realtime fingering detection system for piano performance[END_REF]. In O' Grady and Rickard (2009), they propose an alternative paradigm to electric guitar transcription, where the signal generated by each string at the guitar pickup is captured separately using a hexaphonic guitar ; thus changing a polyphonic transcription problem into a monophonic one, which ameliorates subsequent analysis and allows finger position identification for tablature generation.

Ground truth for AMT evaluation

First, a caveat: any evaluation system inherently assumes some idea of ground truth against which the candidate is evaluated. Then, when interesting to the evaluation of AMT performances, the question is: what to compare the output of an algorithm with ? It is common practice in MIR fields to collect reference set or ground-truth data by inquiring musicological experts or by crowd-sourcing. These data are considered the intended output of the computational model and the model is evaluated in terms of its ability to reproduce the ground-truth.

In the field of Automatic Music Transcription, annotated sound databases are then needed both to develop and to evaluate algorithms. However, such ground truth databases are rather scarce, mostly due to the fact that manual transcription is a very cumbersome and time-consuming task. Private databases like those used in contests like MIREX exists (MIREX, 2011), but are not made available to the entire community of researchers, hence the development of public databases. For example, the RWC music database [START_REF] Goto | Rwc music database: Popular, classical, and jazz music databases[END_REF] is a widely-used database in this field, which is composed of real-world music excerpts with several levels of polyphony and different instrumental sounds covering different music styles. But actually only a very few exist, as a number of issues are commonly encountered : little amount of sounds, either to copyright and distribution problems, or recording conditions, the ground truth is often generated a posteriori, with some inaccurate or erroneous values of pitch or onset and offset times, and a time-consuming process. Also, [START_REF] Hainsworth | Techniques for the automated analysis of musical audio[END_REF] within MIR literature figured out that manual transcription strategies can be quite different resulting various degrees of divergence from the original performance. Similarly, the study of [START_REF] Cemgil | Bayesian music transcription[END_REF] shows that there is no unique ground truth for manual transcription even among well-trained musicians.

The question of ground truth datasets was then at the core of this PhD thesis, which raises many methodological questions when one is interested to original instrument repertoires.

MCSSs for ground truth generation

One of the main reasons explaining why the investigation of non-eurogenetic music has fallen behind the one dedicated to eurogenetic music is because of the current lack of ground truth sound datasets. This is related to the absence of written supports often associated to orally transmitted repertoires, and so an absence of extensive inventories of these repertoires. Technologically, also, traditional instruments are mostly unplugged, i.e. wholly acoustic, which do not make easier the process of recording and collecting musical pieces of these repertoires, whereas instruments such as piano and guitar have sensory systems to allow for efficient extensive census of their repertoires.

The use of MCSSs has direct application to the automatic generation of ground truth for AMT, by capturing in-situ the musical production of a musician through the resonating parts of its instrument. The classical piano is the most studied music instrument in MIR, which is to be related to the fact that technology similar to MCSSs, such as the Disklavier (used in the MAPS dataset [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF]), is already widely used for this instrument. Research on MCSSs should help transposing such technology to different music instruments.

The main advantage of MCSSs is that it captures original precise musical interpretations of musicians, and is not fixed by scores. Indeed, the problem is whether the original notation or the manual transcription can exactly match with performance due to personal interpretations of both performer and transcriber. However this point especially becomes a problem when automatic transcription is defined as obtaining original notation from performance as a kind of reverse-engineering (Klapuri, 2004a).

Other Applications of MCSSs

Musical analysis

Retrieval systems with multichannel outputs could help in performing string-based musical analysis, such as isolating the specific rhythmical characteristic within groups of strings, and better understanding the playing methods. In our database, low-level scores1 are now available in machine-readable formats, it is possible to perform directly all kinds of musical analysis.

Human-machine interaction

Human-machine interaction is part of research, called Augmented Instruments or Multimodal Environment, aiming to obtain accurate perception about computer system which can analyze, perform and compose human action [START_REF] Rowe | Machine Musicianship[END_REF]. To do so, multimodal sensory retrieval systems are often used. The idea of extending traditional acoustic instruments with sensors to capture performance information has been explored in [START_REF] Machover | Hyperinstruments : musically intelligent and interactive performance and creativity systems[END_REF]. Other systems response which have influenced the community in this domain are Dannenberg's score following system [START_REF] Dannenberg | An online algorithm for real-time accompaniment[END_REF], George Lewis's Voyager [START_REF] Lewis | Too many notes : Computers, complexity and culture in voyager[END_REF], and Pachet's Continuator [START_REF] Pachet | The continuator : musical interaction with style[END_REF]. There are few systems that have closed the loop to create a real live human/robotic performance system: Mari Kimura's recital with the LEMUR GuitarBot [START_REF] Singer | Lemur guitarbot: Midi robotic string instrument[END_REF], Gil Weinberg's robotic drummer Haile [START_REF] Weinberg | Haile -a perceptual robotic percussionist[END_REF](Weinberg et al., , 2006)), [START_REF] Benning | Multimodal sensor analysis of sitar performance: Where is the beat?[END_REF]'s interactive robot for Indian music. The ESitar developed by [START_REF] Kapur | Digitizing north indian music: Preservation and extension using multimodal sensor systems, machine learning and robotics[END_REF] is an Indian sitar retrofitted with a variety of sensors for capturing gestures of the performer while still producing sound acoustically and being playable as a traditional sitar. An exponentially distributed set of resistors is used in order to detect what fret is played by the performer. A sensor is placed under the right hand thumb and is used to deduce the direction and patterns of plucking. A force sensing resistor captures the applied force, which varies based on the stroke direction.

MIDIfication of acoustic instruments

The process of MIDIfication involves to equip an acoustic instrument with an array of electronic features allowing MIDI connectivity that supports communication with computing devices and external MIDI instruments. Famous models of MIDIfication of acoustic instruments already exist. The typical Disklavier is a real acoustic piano outfitted with electronic sensors for recording and electromechanical solenoids for playback. Sensors record the movements of the keys, hammers, and pedals during a performance, and the system saves the performance data as a Standard MIDI File. On playback, the solenoids move the keys and pedals and thus reproduce the original performance.

Sensoring systems: models, set-ups, quality check

This section aims to study comparatively different sensory systems in order to choose the right sensor for the task of AMT. Indeed, as this task calls for very specific signal processing operations, qualifying any of these operations of 'optimal' is strictly tied to the given sensory system, and the performance of an optimal processing with a given sensory system may be less than the performance of a non-optimal processing with another sensory system. We then perform in the following a quantitative study to evaluate the appropriateness of several sensory systems to the task of AMT of plucked-string instruments. We start by listing the three different sensor types studied, and then by characterizing them through a set of specific numerical descriptors corresponding to required criteria on signals for AMT. Specific requirements must be respected when installing MCSSs on acoustic instruments, and exploiting their output signals for AMT. Our choice of sensor types has been made by attempting at best to comply with all the constraints listed above.

Sensor type I: optical

Background

Optical pickup guitars were first shown at the 1969 NAMM in Chicago, by Ron Hoag. In 2000, Christopher Willcox, founder of LightWave Systems2 , unveiled a new beta technology for an optical pickup system using infrared light. In May 2001, LightWave Systems released their second generation pickup, dubbed the "S2". The S2 featured LightWave Systems' monolithic bridge, six-channel motherboard, and a host of other improvements, making the technology more practical for use in both live and recording studio settings. LightWave Systems began producing their own guitars in the late 2000s. Currently the company features the Saber bass and the Atlantis ElectroAcoustic guitar. These models are the only guitars that come with the LightWave Systems optical pickup installed.

Optical-based systems have already found various applications, such as metrological measures of string displacement [START_REF] Seydoux | Mesure de déplacement de cordes avec des fourches optiques[END_REF][START_REF] Chabassier | Modélisation et simulation numérique d'un piano par modéles physiques[END_REF] or a MIDIfication3 of a piano through the Moog piano-bar technology [START_REF] Mowat | Bob moog piano bar : Midi output device for acoustic pianos[END_REF][START_REF] Assayag | Omax. the software improviser[END_REF].

Physical principle

The selected optical sensor are slotted optical switches consisting of an infrared emitting LED and an NPN silicon phototransistor, that work by sensing the interruption of a light beam by a vibrating string. It has a fork design, with the string placed between the two branches as illustrated in the close-up of the figure 4.2. On one side, the lightemitting diode (LED) emits a light beam. On the other side, the phototransistor has a peak of sensitivity at 850 nm. The emitter casts a shadow of the string onto the photodetectors. As the string vibrates, the size and shape of the shadow changes accordingly and modulates a current which passes through the photodetectors. This current is the analog electrical signal which represents an accurate depiction of the vibrating string. In order to maximize the dynamic of the optical signals and obtain sharp transient attacks, the narrowest possible diameter for the laser is used. Such sensor then acts as a digital switch with a robust sensitivity to string displacements.

Sensor models and set-up

We mainly tested the two following models of optical sensors: -Photomicrosensor (Transmissive) EE-SX398/498 at OMRON (technical sheets at ) ; -Slotted Optical Switch OPB610 (technical sheets at http://optekinc.com/datasheets/ opb610.pdf) ;

These two sensors are represented in figure 4.2. The optical sensor OPB610 presents the most advantageous features for our application: reduced size, narrow beam light with a diameter of 0.5mm and a wavelength of 940 nm, reduced sensitivity to external lights. The power pack of the optical sensors needs a continue tension of 5 V. An enhanced low current roll-off is used to improve contrast ratio and immunity to background irradiance. The power pack of the sensors is thermally isolated, which makes it well aligned with field conditions. a) b)

Sensor type II: piezoelectric

Background

Piezoelectric belong to the family of contact pickup. Many semi-acoustic and acoustic guitars, and some electric guitars and basses, as well as other string instruments such as violins, have been fitted with piezoelectric pickups instead of, or in addition to, magnetic pickups. Solid bodied guitars with only a piezo pickup are known as silent guitars, which are usually used for practicing by acoustic guitarists. Piezo pickups can also be built into electric guitar bridges for conversion of existing instruments. Contact microphones are usually used to amplify acoustic instruments for live performance, or to record sounds.

Physical principle

The piezoelectric element is composed of a piezoelectric ceramic and a metal plate held together with adhesive. Both sides of the piezoelectric ceramic plate contain an electrode for electrical conduction. Piezo materials exhibit a specific phenomenon known as the piezoelectric effect and the reverse piezoelectric effect. Exposure to mechanical strain will cause the material to develop an electric field, and vice versa.

Piezo-elements are made from two conductors separated by a layer of piezo crystals. Quartz crystals are used in most piezoelectric sensors to ensure stable, repeatable operation. The quartz crystals are usually preloaded in the housings to ensure good linearity. When the crystal is stressed with an external pressure, bending the metal conductor layers, a voltage is generated across the crystal layer. Piezoelectric pickups have a very high output impedance, appearing as a capacitance in series with a voltage source, and routed through a special low-noise cable to an impedance-converting amplifier.

When an alternating voltage is applied to the piezoceramic element, the element extends and shrinks diametrically. This characteristic of piezoelectric material is utilized to make the ceramic plate vibrate rapidly to generate voltage.

Sensor type III: electromagnetic

Background

Electromagnetic sensors are commonly used as pick-up in electric guitars.

Physical principle

Magnetic pick-ups, as applied in electric guitars, register the vibrations of nickel or steel strings in a magnetic field. A magnetic pickup consists of a permanent magnet with a core of material such as alnico or ceramic, wrapped with a coil of several thousand turns of fine enameled copper wire. The permanent magnet creates a magnetic field; the motion of the nearby soft-magnetic vibrating steel strings modulates the magnetic flux linking the coil, which induces a voltage in the coil. This signal is then carried to amplification or recording equipment via a cable. More generally, the pickup operation can be described using the concept of a magnetic circuit. In this description, the motion of the string varies the magnetic reluctance in the circuit created by the permanent magnet.

Some high-output pickups achieve this by employing very strong magnets, thus creating more flux and thereby more output. This can be detrimental to the final sound because the magnet's pull on the strings can cause problems with intonation as well as damp the strings and reduce sustain. The turns of wire in proximity to each other have an equivalent self-capacitance that, when added to any cable capacitance present, resonates with the inductance of the winding. This resonance can accentuate certain frequencies, giving the pickup a characteristic tonal quality. The more turns of wire in the winding, the higher the output voltage but the lower this resonance frequency. The inductive source impedance inherent in this type of transducer makes it less linear than other forms of pickups, such as piezo-electric or optical.

Sensor models

-The GK-3 by Roland http://roland.com/V-Guitar/about.html, designed a strip of electromagnetic sensors dedicated to acoustic guitars ; -Handcrafted sensor based on magnetic scanning heads of mini-K7 The electromagnetic sensors we used were handcrafted sensors built with magnetic scanning heads of mini-K7. The commercialized GK-3 Roland sensor was used to calibrate our own electromagnetic sensors, illustrated on the right graph of figure 4.6. These two sensors are represented in figure 4.6. The commercialized GK-3 Roland sensor was used to calibrated our own electromagnetic sensors.

b) a)

Set-up installation

The pickup is most often mounted on the body of the instrument, but can be attached to the bridge, neck or pickguard, as on many electro-acoustic archtop jazz guitars and string basses. In our project, we used a portable bridge for our pickup sensors, and attached it to the body of the instrument. This MCSS was especially used for the mvet zither.

Quality check on Sensor Installation

Non-intrusiveness

The system must not be too cumbersome and disturbs the playability of the instrument. Specific playing techniques such as palm muting and excitation point displacements require a playing zone which must not be disturbed by the devices. The ideal sensory system would also not require a specific instrument-making.

Sensor placement in regards to string vibrations

The measuring point of string displacement may create a bias in the amplitude measure. Indeed, as this displacement consists of a superposition of vibratory modes, defined as a succession of nodes and anti-nodes, if a sensor is placed on a modal node the energy contribution of the corresponding mode is null. To answer these two constraints, the bar of sensors is positioned near the easel, in such a way that the playing zone is less disturbed and that the sensors are roughly placed on the ascending slope of the anti-node following directly the easel-related node common to all modes. When installing the sensors on instruments, one has to take care of avoiding placements on "nodal modes".

Quality check on Sensor Signal

We now quantify the quality of each sensor signal in view of AMT on the marovany instrument. Although pickup devices are most often evaluated for sensor transparency in the restitution of sound quality, for the task of AMT we rather focus on the ability of sensors to efficiently retrieve the four basic musical parameters in AMT: onset location, pitch, duration and amplitude. This quality check is performed with classical acoustic descriptors (detailed in Annex A).

A mechanical arm of calibration

A mechanical arm, illustrated in figure 4.7, is used to set precisely a reference strength to the excitation source. Three different heights h of the arm were used to define three different reference excitations (arbitrarily assigned to the musical nuances of pp, mf and ff), while keeping the height hs fixed and equal to the height of the string excited. The distance d is also kept fixed for the different excitations. Also, using this mechanical arm, we were able to calibrate the output level of each sensor, relatively to the reference excitation levels provided by the arm. Electronic gains respective to each sensor can then be deduced and apply in a post-processing stage to the signals.

Evaluation protocol

Eight different signal criteria C c , with c ∈ {1, • • • , 8}, will be evaluated on each sensor signal x s (t), with s ∈ {1, 2, 3}. Each signal criteria C c employs a characteristic function 1. Three different amplitude levels (pp, mf and ff), provided by the mechanical arm ; 2. Six different strings with the following pitches:

D 3 #-A 3 -C 4 -E 4 -B 4 -D 5 ;
The three MCSSs have been installed on the marovany zither N 1 , and their respectively signals are acquired simultaneously. The size of this set of notes is then N Itest = 3 × 6 = 18. Furthermore, microphone signals, on which audio processing is usually applied, are here used as reference to characterize physical properties of our sensor signals. Unless mentioned otherwise, we then define the normalized measure C c as follows

C c = median i∈Itest (χ c (x i s (t)) -χ c (x i 0 (t))) (4.1)
The sign resulting from this operation does not have any physical meaning, but just indicates whether the value is higher or smaller than the respectiveχ c (x 0 (t)) value.

Salience measures

We first compute the onset detection function F odf described in [START_REF] Poliner | A discriminative model for polyphonic piano transcription[END_REF], followed by a peak finding function which identifies the location of each onset. Our first signal quality criteria C 1 is the degree of salience of the detected peaks. We simply compute the signal-to-noise-ratio associated to each test note p, defined as

χ 1 = max(h(t)) h(T n ) (4.2)
where h(t) = F odf (x(t)), and T n is defined as the first 10 windows. The sharpness of the peak (C 2 ), which informs on the accuracy of time location of the onset, is also computed. This measure of sharpness is made through the following dispersion measure

χ 2 = arg t (h(t) = max(h(t)) 4 ) (4.3)
These measures are illustrated in figure 4.8.

Spectral measures

Our third numerical criteria C 3 characterizes the harmonic content of the signal, and is computed using eq. 4.1, with χ 3 defined by the Harmonic Energy descriptor, i.e. HarmErg in [START_REF] Peeters | The timbre toolbox: Extracting audio descriptors from musical signals[END_REF] (detailed in Annex A). This descriptor ranges from 0 to 1. We

Temporal measures

The signal criteria C 5 is on note duration, taking χ 5 ∼ Ef f Dur, with EffDur the descriptor Effective Duration defined in [START_REF] Peeters | The timbre toolbox: Extracting audio descriptors from musical signals[END_REF]. The Modulation Ratio of the temporal envelop (MOD) is also computed for criteria C 6 , taking χ 6 ∼ M OD. This second descriptor ranges from 0 to 1.

Inter-note temporal interval measure

We must have a good separability of successive notes of a same string, with ideally inter-notes blanks resulting corresponding to the finger-string contacts. The descriptor C 7 measures the time interval between two successive notes. As illustrated in figure 4.9, we first computed the energy envelop in the location of the inter-note space, and then simply measured the time interval T s between these notes at a same energy level, set at three times the noise level of the sensor signal.

Multichannel separability

This signal criteria C 8 is a measure of the degree of "repisse" in our multichannel output, using all pitches of our instrument pitch range. Each of these pitches is successively played, and we measured the residual RMS-energy level occurring at the same time in all other sensor signals. A single value is obtained by summing all elements of this matrix, providing a general estimation of sensor energy residual due to inter-sensor repisse.

Results and Discussion

Table 4.1 provides the numerical results obtained for each signal criteria with our different sensor types. We also propose in figure 4.10 a more concise representation of our results, allowing a direct quantitative comparison between our different MCSSs. We normalized the values of table 4.1 using simple mathematical transformations following the rules of quality for the task of AMT:

-C 1 The higher the onset salience, the easier the identification of onset locations ; -C 2 The higher the onset sharpness, the easier the identification of onset locations ; -C 3 The richer the harmonic content of a sensor signal, the harder the pitch estimation is. In other words, sensors filtering out higher frequency components while emphasizing the string fundamental vibratory mode tends to greatly make the pitch estimation easier ; -C 4 The higher the accuracy of F 0 measure, the better the transcription ; -C 5 The higher the accuracy of note duration, the better the transcription ; -C 6 The smaller the temporal envelop modulation, the easier it is to segment note boundaries ; -C 7 The larger the inter-note space, the easier it is to segment successive notes ; -C 8 To perform monophonic transcriptions, the independence of each sensor signal is of first importance, each sensor having to detect solely the vibration associated to its string. Then, the higher the independence of each sensor signal, the easier the monophonic transcription is. Furthermore, although the multichannel "repisse" can result from sympathetic resonances between strings, it is still seen as a parasite signal from the point of view of AMT, and sensor signals which flatten out residual signals are consequently more advantageous. Based on table 4.1 and figure 4.10, we can first observe that globally all sensor signals have sharper onsets than those in the microphone signal, which can be partly explained by the fact that a sharp signal pattern is obtained at the beginning of plucking during the period of finger-string contact. However salience values are slightly below microphone ones. Piezoelectric and optical sensors offer the highest salience and sharpness of onsets. The magnetic pickups appear to extraneous noise to the sound, while optical has virtually no inherent noise. Even when putting an amp at high volume, piezoelectric and optical sensors experience an increased dynamic range and sensitivity, i.e. loud notes are loud and clear, while soft notes and subtle nuances are not masked by background noise.

Second, our sensor signals tend to have a poorer harmonic energy than microphone signals, but a stronger harmonicity, which can be explained by the fact that a direct measure of string displacement privileges its own vibratory behavior, emphasizing the string fundamental frequency and minimizing the effects of coupling with the more complex modes of the soundboard. Furthermore, piezoelectric and electromagnetic sensors tend to exert more their own influence or color on the strings' vibratory properties.

Third, all sensors have a detrimental impact on temporal profiles of marovany sounds, and none are able to read string vibration for its full duration. Piezoelectric and electromagnetic sensors interfere with string vibration. Piezoelectrics as they are in direct contact with strings, and electromagnetic sensors with the magnetic influence which "pulls" on the metal strings. As a non-contact sensor, optics do not interact with the movement of the string, and even if they restore the most faithfully the string vibration temporal waveform, they also distort it due to their intrinsic response.

Fourth, one of the most conspicuous advantages of sensors over the microphone is the formation of a space between successive notes, both in time and amplitude, resulting from the mechanical contact between string and finger. Piezoelectric sensors appear to provide the largest inter-note space. Fifth, we observed that globally most leakage signal are negligible in our sensor signals, expect for certain couples of pitches resulting from the sympathetic resonances of the instrument. Electromagnetic sensors offer the best channel separability, while optical and piezoelectric sensors appear to be a little more affected by the repisse between each string.

Methods for Automatic transcription of Monophonic sensor signals

We will restrict ourselves to retrieve the musical information necessary for low-level transcription, i.e. measuring the activity and basic parameters (onset location, amplitude, pitch, duration) of played notes. Figure 4.11 shows the block diagram of successive processing steps in the MCSS-based AMT. We tested two basic transcription algorithms: a feature based algorithm, which can be implemented in real-time applications, and a spectrogram factorization based algorithm, based on a PLCA-learning of a template dictionary for both harmonic and noise signals.

Feature-based

Left bottom graph in figure 4.11 represents the evolution of a waveform signal processed through this algorithm. From left top to right bottom: original optical signal, denoised signal, residual r with location of onsets, and segmented signals.

Blind adaptive denoiser

A blind denoising method [START_REF] Ephraim | Speech enhancement using a minmum-mean square error log-spectral amplitude estimator[END_REF] is applied to optimize the signal to noise ratio, mainly deteriorated by parasite noise coming indiscriminately from electronics and mutual resonances of strings. This algorithm of denoising takes as inputs segments of noises (defined as the first second of acquisition signals), and allows their subtraction to the signal by minimizing a prediction error with a least-mean square optimization.

Onset detector

Then, a 0.049-s hamming window with a 0.005-s overlapping (that is 11.6 ms, providing a temporal resolution whose order of magnitude is similar to the time attack) scans the entire sequence. Each onset of notes is detected using a spectral difference which takes into account the phase increment, as introduced by [START_REF] Bello | On the use of phase and energy for musical onset detection in the complex domain[END_REF]:

Xk,n = |X k,n-1 |e j(2φ k,n-1 -φ k,n-2 ) (4.4)
with n the index of each window. As marovany sounds consist roughly of a superposition of short stationary sinusoids, the occurrence of an onset generates a peak in the prediction error defined by

r(n) = N n=1 | Xk,n -X k,n | (4.5)
Windows for which this residual exceeds a fixed threshold are validated as onsets.

Note segmentation

From this detected onset, the descriptor E (eq. A.7) is computed for the neighbouring windows to search the local maximum E max (i) associated to pitch i, assuming this maximum is located near the onset, as expected for notes played by plucked string instruments. Then, E is computed on all the windows following the onset until the energetic value decreases below 5 % of E max (i), which may then be read as an adaptive note-specific energy threshold, or until another peak in the residual r is found. This estimation allows us to deduce note duration (eq. A.2), and its amplitude by averaging the energy over all windows within the note. We are not interested in the absolute amplitude of the notes, but only in their relative values within an air, in reference to a value determined by the MIDI gain.

Matrix factorization based method

The frequency marginals P (f |z) of a PLCA model can be used as a model for certain kinds of sounds, which will now allow us to tackle the tasks of detection using separate class modeling. In a given MCSS sequence, two sound classes are assumed to be present, namely the background noise from the electronic device, and the instrumental notes. In the following, we then define two different template dictionaries, whose elements are defined by the frequency marginals distribution P i (f |z) and P n (f |z), with z ∈ Z i and Z n , respectively the sets of latent variables for the note and noise classes. The following two-class PLCA for sound/noise detection will be used,

P (f, t) = P (t)( z∈Zn P (f |z)P (z|t) + z∈Z i P (f |z)P (z|t)) (4.6)
To impose a better discriminability between the spectral basis of our two latent families, i.e. enforcing a certain dissimilarity between them, we use the sparseness prior eq. 3.8 (Sec. 3.2.1). In particular, as we do not know exactly the optimal number of latent variables to best explain vocal sounds, this prior then allows to adapt our basis decomposition to the specific data multi-dimensionality.

A crucial step before performing a PLCA-based analysis is to properly initialize template dictionaries, which allows EM-based parameter estimation to be much more precise. We perform a first initialization of these templates using generic sounds from each class. Figure 4.12 provides some examples of these template dictionaries. The frequency marginals extracted from the instrument notes have a clear and spaced harmonics structure with a distinct pitch, while the marginals of noise display a more broadband and inharmonic spectrum. Once the frequency marginals are known for a certain sound in a mixture, they can be used to extract this kind of sound from the mixture in a supervised way [START_REF] Smaragdis | Supervised and semi-supervised separation of sounds from single-channel mixtures[END_REF]. We then perform a data-based learning of template dictionaries using the conservative transcription method (see Sec. 2.2.8). After adapting our template dictionaries to input data, we extract the instrumental ntotes from the mixture by setting noise activations P (z) for z ∈ Z n to zero, and therefore, a denoised reconstructed PLCA model of sound spectra P i (f, t) can be obtained by:

P i (f, t) ≈ P (t) z∈Z i P (f |z)P (z|t) (4.7)
To make this PLCA-based denoising process even more efficient, we also implemented in the preceding system the spectral re-weighting operation developed by [START_REF] Ye | Robust acoustic feature extraction for sound classification based on noise reduction[END_REF], based on a frequency band-wise reconstruction error. It is grounded on the fact in human auditory that noise corrupted sound is recognized via processing the audio in local high SNR frequency bands [START_REF] Cooke | A glimpsing model of speech recognition in noise[END_REF]. It computes the following frequency band-wise reconstruction error possibility distribution

P (f |e) = t (X(f, t) -P i (f, t)) 2 t f (X(f, t) -P i (f, t)) 2 (4.8)
where X is input noise spectrogram and X is the denoised reconstructed spectrogram by PLCA model. P (f |e) presents spectral error possibility distribution.

ω f = 1 - P (f |e) max(P (f |e)) (4.9)
From this model error distribution ω f , we can know which frequency bands of the the denoised reconstructed spectrogram by PLCA model present the strongest similarity with the input spectrogram. After this denoising process, we use a simple threshold-based detection of the note activations from the activity matrix P (z, t), followed by a minimum duration pruning. The threshold for minimum duration for pruning was set to 200 ms.

Results and discussion

We created ground truth by semi-automatically hand-labeling several musical sequences captured with different MCSSs, extracting precise information on note onset location, duration, amplitude and pitch. To do so, we use a custom user-friendly interface, developed in Matlab and illustrated in figure 4.13, which implements an energy-based semi-automatic system optimized for the transcription of our MCSS signals. [START_REF] Kay | Fundamentals of Statistical Signal Processing[END_REF] for details) curves with the error metrics FPR and TPR on our monophonic transcriptions of sensor signals, for our two feature-based and PLCA-based methods. Solid lines show transcription performance with an evaluation based only on pitch and onset location, and dashed lines show performance with a full evaluation based on the four parameters of a note, i.e. adding duration and amplitude. Our PLCA-based method provides the best performance for this transcription task, with accuracy (average F-measure) higher than 96 % over a large interval of thresholds (i.e. 0.11 -0.87). When considering all note parameters, transcription performance decreases. Large and very diverse solo database are needed for training and evaluation of the AMT process. In the international MIR community, euro-genetic music, especially through the classical piano, has received the most attention from researchers, while non-eurogentic music has been rather left outside. Also, sound database for AMT may include a Musical Instrument Sample Database of Isolated Notes (MISDIN), which is a collection of sound samples of one or more musical instruments where each sample contains a recording of a single note played by one instrument. MISDINs are commonly used by electronic musical instruments, such as synthesizers and samplers, to reproduce sounds of other instruments. MISDINs are also utilized by the majority of music information retrieval (MIR) algorithms, including pitch estimation [START_REF] Li | Pitch detection in polyphonic music using instrument tone models[END_REF], music representation [START_REF] Leveau | Instrument-specific harmonic atoms for mid-level music representation[END_REF] and others, as evaluation data for experiments and for modeling sounds of different musical instruments. One major issue with creating a spectrogram factorization based transcription system for non-eurogenetic music is the lack of isolated note recordings for creating a training set. An isolated sounds database tends to considerably improve the performance [START_REF] Benetos | Automatic transcription of turkish makam music[END_REF].

Motivations

In this context, we have launched this research project in June 2013 of developing a Big Music Database of non-commercial recordings dedicated to plucked-string instruments. Our main long-term motivation is to provide original sound material for AMT evaluation, so as to bring the development of AMT systems closer to the actual world-wide music diversity and complexity. In response to this concern, our sound database first consists exclusively of the family of plucked-string instruments. These instruments are particularly interesting as they present a great diversity in terms of timbre (e.g. inharmonicity, envelop spectrum variability, temporal profile modulation), playing styles and effects (e.g. different string excitation modes, palm muting, glissandi, vibrato, tremolo, harmonic notes). Second, our sound database wishes to include non-eurogenetic instrument repertoires, emphasizing even more the diversity related to timbre and playing techniques mentioned above. Indeed, acoustic measurements on isolated tone datasets of the marovany and the mvet reveal a more complex timbre (noticeably in terms of spectral envelop variability) than a standard instrument like the classical piano. Also, non-eurogentic musical repertoires encompass a great diversity in musicological codes (e.g. micro-rhythm, tuning based on non-equal temperament, modal scale), especially for orally transmitted musical repertoires, which also raise complex questions on transcription representation.

Technical specifications for recording

Place

Recordings mainly took place in a semi-anechoic chamber of the LAM, designed to perform recordings with absorbent panels on the walls (SNR ≈ 50 dBA). Recordings in Madagascar with insular musicians have also been performed, where we took care of finding rooms as quiet as possible. Furthermore, close proximity of the microphone to the instrument (around 40 cm) also favours to remove surrounding sound ambient. This homogenization of recording conditions allows for inter-recording comparisons which process only features related to the instrument repertoire, such as timbre, musicology and playing technique. It is noteworthy that nowadays, diversity in recording conditions of real-life audio recordings can be simulated subsequently, with computational tools such as the Degradation Toolbox [START_REF] Mauch | The audio degradation toolbox and its application to robustness evaluation[END_REF].

Equipment

Figure 4.15 details the different elements of our recording equipment. It is composed of a sound card RME Fireface UFX, connected to RME pre-amps. Sensor signals are eventually stored synchronously onto the hard drive of a PC laptop with the music production/recording software Cubase LE. Each string's signal then starts out analog, and stays that way throughout up to the digital sound card.

All our recordings respect the following technical details: -Schoeps stereo microphone, CMC 6 (capsule MK4), arranged following the ORTF setup (18 cm d'Ãľcart inter-microphones) ; -Microphone is held at the instrument height, in front of it, with an average distance microphone/instrument soundboard of 50 cm ; -The recordings are sampled at 44.1 kHz and stored as 24 bit WAV files ; -To remove infrasonic disturbances the signals were highpass filtered with a fourthorder Butterworth filter with a cutoff frequency of 52 Hz ;

Recording types

Musical pieces

All the musical pieces recorded in this database are fully human-played. All the musicians are recognized professional, specialist of their instrument, and are asked to play representative pieces of their repertoire. Each piece can last from 2 to 6 minutes. The musician is also asked to define a pulsation track on which he will play all his musical Instrument with MCSS RME sound cards and pre-amps sequences. Certain musicians preferred to play their instruments without predefined pulsation, and reproduced it afterwards. We did not consider it as a bias in our experiment, as such musicians feel an interior pulsation. The tuning of each musical piece is identified trough precise F 0 measures on isolated note samples for each pitch.

Ground truth

Our ground truth has been built from transcriptions not based on examinations of musicological experts but rather on signal features, making it more objective. Our ground truth for each musical piece results from the use of Multichannel Capturing Sensory Systems, able to capture independent string-specific signals. When fulfilling the specific signal criteria previously exposed in Sec. 4.2.5, their transcription does not pose great difficulties, in comparison to the processing of the audio microphone signal. Although quite invasive as needing a complex experimental set-up during recording sessions, such a system allows for fast and very reliable polyphonic transcriptions.

Isolated tones

Each single instrument model recorded in our database has its own set of isolated notes, recorded in a same period (never exceeding 1 week). Pitch tones are tuned in the MIDI scale, so as to normalize our different datasets on the same pitch scale. 10 different occurrences of notes are recorded for each pitch, varying playing dynamic over three levels (pp, mf and ff), as well as more variable factors due to playing techniques such as distance to the easel, string pluck intensity, attack mode (nail or finger pulp) and finger inclination. The complete pitch range of the instrument is recorded. Also, we minded all spurious noises (e.g. finger scraping on the string) during recording. Also, each musician was asked to play all notes of the instrument once, from which we extracted the precise tuning (± 20 cents) respective to each musical piece.

For each pitch of the instrument pitch range, we have 12 different tones per instrument

Conclusion

In this chapter we propose to use the technology of Multichannel Capturing Sensory Systems (MCSSs) to automatically generate a large amount of reliable ground truth, in particular for the repertoires of the marovany zither. Among the most conspicuous technological advantages of these systems, we can mention the high signal to noise ratio, the multichannel output with independent signals corresponding to the played strings and the automatic demarcation between successive notes of a same string.

Chapter 5

Results and Discussions Abstract

In this chapter, we present our main results obtained in this PhD project, and propose several discussions around the use of knowledge from musical acoustics in automatic transcription systems of music. Our different components of musical knowledge have been successively injected in our baseline transcription systems, consisting mainly of a Probabilistic Latent Component Analysis (PLCA) algorithm post-processed with a Hidden Markov Model (HMM), and their impacts on transcription results have been comparatively evaluated.

Evaluation Methods

Evaluation procedure

Background

There are various evaluation metrics for AMT based on comparison of transcription and reference notation. [START_REF] Fonseca | Measuring music transcription results based on a hybrid decay/sustain evaluation[END_REF] classify evaluation metrics as framebased and note-based approaches. Frame-based approach is based on the comparison of two notations for every analysis temporal windows. The note-oriented approach is based on numerical rules identifying transcribed notes as corrects, depending on whether their onsets, pitches and/or durations are within a certain neighbourhood of the respective reference data. Another note-oriented evaluation metric is edit distance (ED), where the transcription is compared with reference notation on the basis of number of correct, inserted and deleted notes [START_REF] Unal | Challenging uncertainty in query by humming systems: A fingerprinting approach[END_REF].

Note-based evaluation

For assessing the performance of our proposed transcription system, we perform a notelevel evaluation stating that a note event is assumed to be correct if it fills four conditions on:

1. Onset location. Its onset must be within a ∆ o (in ms) range of a ground-truth onset. A value of 50 ms is commonly used for ∆ o (Bello et al., 2005;[START_REF] Mirex | Music information retrieval evaluation exchange (mirex)[END_REF]. Such tolerance level is considered to be "a fair margin for an accurate transcription" according to [START_REF] Seeger | Prescriptive and descriptive music writing[END_REF], although it is far more tolerant than human ears would, as we remind that those are able to distinguish between two onsets as close as 10 ms apart (sounds arriving to the ear with lower time intervals are perceptually merged) [START_REF] Moore | An Introduction to the Psychology of Hearing[END_REF]. A stricter tolerance error of 40 ms has been used by [START_REF] Collins | Using a pitch detector for onset detection[END_REF] for percussive sounds. Our error tolerance was set based on figure 5.2, which provides the distribution of inter-onset intervals in the studied corpus. We put our tolerance error far below the smallest value of these intervals in order to be sure that our transcription systems are able to distinguish between all successive notes of our corpus. A value of 40 ms was retained for ∆ o ;

2. Pitch. Its pitch needs to be within a ± ∆ i (in cents) tolerance around the groundtruth pitch. A value of 20 cents is commonly used for ∆ i in eurogenetic music [START_REF] Fonseca | Measuring music transcription results based on a hybrid decay/sustain evaluation[END_REF]. This error resolution is equal to one fifth of a semitone ;

3. Duration. Its duration needs to be within a ± ∆ d (in s) tolerance around the ground-truth duration. In most AMT studies, duration is not evaluated, stating that especially for "decay instruments", the offset time is only important in a limited period of time, because after some point the sound energy will be below the threshold of hearing [START_REF] Zwicker | Psychoacoustics: facts and models[END_REF], even if from the musician point of view the note is still playing. But ∆ d can be set relatively to 20% of the ground truth note duration [START_REF] Mirex | Music information retrieval evaluation exchange (mirex)[END_REF]. Actually, the definition of a note offset has been a long-time ill-posed problem, and its effect on the success rate has actually never been clear in past studies. Much more than any other musical parameters, the offset suffers from the arbitrary of the ground truth, that is why we set a relatively higher error tolerance on this parameter ; to be evaluated are first normalized by the maximum value, and only relative amplitude rapports to this value are then evaluated. Only a few AMT studies have tackled the problem of estimating note amplitudes, among which we can mention [START_REF] Marolt | A connectionist approach to automatic transcription of polyphonic piano music[END_REF], who computed this note amplitude based on the energy of its first harmonic, and [START_REF] Ewert | Estimating note intensities in music recordings[END_REF], who used a parametric model of the spectrogram to estimate this amplitude. Table 5.1 details our ∆ values for our different note parameters. As mentioned above, our thresholds of error tolerances on note duration and amplitude are pretty high, both because these parameters have never been properly defined in literature, and also because we know from section 4.3.3, that our ground truth transcriptions do not have a sufficient precision on their estimation (i.e. T P R and F P R ≤ 80 % from the ROC curves).

Note parameters ∆ values Onset location

∆ o = 25 ms Pitch ∆ i = 20 cents Duration ∆ d = 40% . R GroundT ruth Velocity ∆ a = 40% . A GroundT ruth
Table 5.1 -Tolerance thresholds for the evaluation of each note parameter. We defined R GroundT ruth and A GroundT ruth the note duration and amplitude from the ground truth.

Furthermore, we used a binary approach to transcription (i.e. the original note is perfectly transcribed or it is not transcribed at all) with a one-to-one mapping when comparing transcription results, i.e., if a match is detected between an original note and a transcribed one, none of those can be used again on other note matches [START_REF] Ryynanen | Polyphonic music trancription usng note event modeling[END_REF][START_REF] Mirex | Music information retrieval evaluation exchange (mirex)[END_REF]. The main idea is to avoid situations on which 2 different original notes could be matched to a single transcribed one (for instance, two quick notes been transcribed as one), or vice-versa, allowing a perfect score even in situations that have different number of notes.

Evaluation metrics

Evaluation metrics are defined by equations 5.1-5.4 MIREX (2011), resulting in the note-based recall (TPR), precision (PPV), fall-out (FPR) and the harmonic mean of precision and recall (F-measure):

T P R = T P T P + F N (5.1) P P V = T P T P + F P (5.2)

F P R = F P F P + T N (5.3)
F -measure = 2.P P V.T P R P P V + T P R (5.4) where TP, FP and FN scores stand for the well-known True Positive, False Positive and False Negative detections. The recall is the ratio between the number of relevant and original items; the precision is the ratio between the number of relevant and detected items; and the F-measure is the harmonic mean between precision and recall. For all these evaluation metrics, a value of 1 represents a perfect match between the estimated transcription and the reference one.

Evaluation technique

From our different sound databases, we extracted different sets of training and test data. Within each dataset, the musical sequences were randomly split into training and testing sequences, using by default 30 % of sequences for testing, and the 70% remaining ones for training. Sequences from a same musical piece were constrained to appear either in training or in test data. Also, the learning of note templates (see T 3 , T 5 and T 6 ) was made on instrument models different from the one used in test sequences. This procedure allows preventing any overfitting of our data in our simulation experiments, and is repeated five times, with an average computed on the resulting transcription scores. It is noteworthy that usually, results on AMT evaluation [START_REF] Dessein | Real-time polyphonic music transcription with nonnegative matrix factorization and beta-divergence[END_REF][START_REF] Benetos | Multiple-instrument polyphonic music trancription using a convolutive probabilistic model[END_REF]Grindlay and Ellis, 2011) are presented by selecting the parameter values (e.g. the sparsity coefficient in Benetos and Dixon (2013)) that maximizes the average accuracy in a dataset.

Evaluation AMT algorithms

Tolonen2000's algorithm

This algorithm [START_REF] Tolonen | A computationally efficient multipitch analysis model[END_REF] is an efficient model for multipitch and periodicity analysis of complex audio signals. The model essentially divides the signal into two channels, below and above 1000 Hz, computes a "generalized" autocorrelation of the low-channel signal and of the envelope of the high-channel signal, and sums the autocorrelation functions.

Emiya2010's algorithm

This algorithm [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF] models the spectral envelope of the overtones of each note with a smooth autoregressive model. For the background noise, a movingaverage model is used and the combination of both tends to eliminate harmonic and subharmonic erroneous pitch estimations. This leads to a complete generative spectral model for simultaneous piano notes, which also explicitly includes the typical deviation from exact harmonicity in a piano overtone series. The pitch set which maximizes an approximate likelihood is selected from among a restricted number of possible pitch combinations as the one.

Fuentes2013's HALCA algorithm

The Harmonic Adaptive Latent Component Analysis (HALCA) algorithm 1 [START_REF] Fuentes | Harmonic adaptive latent component analysis of audio and application to music transcription[END_REF] is used as a state-of-the-art reference. This algorithm was recently evaluated by MIREX and obtained the 2 nd best score in the MPE task, 2009-2012[START_REF] Mirex | Music information retrieval evaluation exchange (mirex)[END_REF]. It mainly differentiates from our baseline algorithm B 0 by the EM estimation of kernel weights, which are not kept fixed but update with data. This model also includes a noise model and different priors on sparsity, monomodality and temporal continuity of spectral envelops.

Benetos2013's algorithm

This PLCA-based AMT system 2 (Benetos et al., 2013a) uses pre-fixed templates, and has been ranked first in the MIREX transcription tasks MIREX (2011).

Proposed system 1 (BaseCaz1(C i ))

This first system corresponds to our baseline PLCA plus HMM system, with the incorporation of a combination of KCMA given by C i . When C i is empty, this system then uses the simple PLCA system presented in 2.2.8.

Proposed system 2 (BaseCaz2(C i ))

This first system corresponds to our baseline PLCA plus PF system, with the incorporation of a combination of KCMA given by C i .

Numerical parameters & Default configuration

For all methods, as a time-frequency representation, the constant-Q transform (CQT) with 60 bins/octave was used, with a half-overlapped Hamming window of 23 ms (i.e. 1024 coefficients at 44.1-kHz sampling rate). The number of EM iterations is experimentally set to 25. By default, and unless said otherwise, we use the simple thresholding method (see section 2.3.3), which should allow one to better highlight the differences brought by musical knowledge. Also, unless said otherwise, we will only evaluate Onset location and Pitch, as judging that we did not have sufficiently reliable ground truth data for this evaluation, and also because those are not evaluated in most AMT studies. Also, after numerical experimentations, an optimal value of ε = 0.08 in eq. 2.28 has been set.

1. Codes are available at http://www.benoit-fuentes.fr/. 2. Codes are available at https://code.soundsoftware.ac.uk/projects/amt_mssiplca_fast.

Results of AMT performance I : on the marovany repertoires

In this section, we wish to investigate the impact of KCMA incorporation in AMT systems on our different marovany repertoires. Our research publication Article 2 resumes a large proportion of the content of this chapter, although with a smaller sound dataset.

Evaluation sound dataset

For this first set of numerical experimentations, three different repertoires R i of the marovany zither were composed from our PSIFAMT database, corresponding respectively to R 1 = {D 1 , D 5 } (Velonjoro's repertoire), R 2 = {D 4 , D 6 } (Charles Kely's repertoire) and R 3 = {D 3 , D 7 } (Kilema's repertoire). 12 different musical pieces were taken from each musical repertoire. We then selected 8 non-overlapping sequences of 30 seconds in each piece, for a total of 144 sequences (i.e. 48 minutes) per repertoire. All musicians played the major musical pieces of their repertoire on the same marovany model (model N 1 ), equipped with an original multi-sensor retrieval system Conference 2. Datasets R 1 to R 3 in combination with the note sample dataset N 1 compose our oracle systems3 . A fourth "universal" sound database R 4 was also constituted by mixing all MIDI repertoires and template sets.

Performance of Baseline systems

In this first section, we test the performance of the different algorithms on the test bench, which allows placing our baseline transcription algorithm performance in the context of state-of-the-art algorithms.

Evaluation on Onset location and Pitch

As displayed in table 5.2, our baseline system algorithm BaseCaz1 provided transcription results of 57.5 %, 60.3 % and 62.5 % for the different R 1 to R 3 , respectively, with slightly worst results for our algorithm BaseCaz2. Globally, the state-of-the-art AMT system of Benetos2013 provides the best transcription results. However, it must be reminded that, on the contrary to other the other systems, Benetos2013 uses pre-fixed instrument templates per pitch, i.e. musical knowledge specific to the instrument repertoire. Whereas the worst performing system is Tolonen2000, which is a feature-based algorithm without any prior information on the signals to be retrieved. This last approach appears to suffer the most from ambiguities in multi-pitch estimation.

Evaluation on all note parameters

As displayed in table 5.3, our baseline system algorithm BaseCaz1 provided transcription results of 52.7 %, 48.5 % and 50.2 % for the different R 1 to R 3 , respectively. As expected, by taking into account all note parameters in the AMT evaluation (with the error tolerance thresholds detailed in table 5.1), our transcription performance is drastically deteriorated.

Baseline systems

R 1 R 2 R 3 R tot Tolonen2000

Evaluation of a monophonic mode transcription

This lower level AMT task of monophonic transcriptions may be useful in many different application such as melody extraction, and identification of motive structures on larger time-scales. As displayed in table 5.4, our baseline system algorithm BaseCaz1 (using the note segmentation mode described in section 2.3.2) provided transcription results of 67.1 %, 67.9 % and 65.1 % for the different R 1 to R 3 , respectively. In this evaluation set-up, our different AMT systems are rebalanced, while still outperforming more feature-based algorithms such as Tolonen2000.

Sensitivity to threshold-based note segmentation methods

We now study dependency of our results on our threshold-based note segmentation methods, being either the simple fixed threshold (see section 2.3.3) or the adaptive one (see section 2.3.4). Figure 5.4 displays the evolution of the F -measure (in %) against the value of the fixed threshold T hres f ix (in dB) for three different baseline methods. A same tendency can be observed for the different curves, with a clearly identified optimal threshold for transcription, i.e. best compromise between true transcribed notes and false alarms. The adaptive-threshold segmentation method brings more enhancements for the AMT systems HALCA and BaseCaz1, supposedly because more variations occur in their activity matrices, in comparison to methods based on pre-fixed templates such as Benetos2013.

Sensitivity to template learning methods

Figure 5.5 proposes a comparison of transcription performance based on ROC curves using our different template learning methods, presented in section 2.2.8, in our baseline PLCA model BaseCaz1. The method of pre-fixing the templates before model parameter estimation proves to provide the highest performance in our simulations, while the method of data-based template adaptation using a conservative transcription presents similar performance to the HALCA template learning process.

Testing each KCMA individually

Each single KCMA incorporated in our AMT system BaseCaz1, described in tables 3.1-3.3, was successively incorporated in our transcription framework, composed of the three blocks (PLCA-Postprocessing-HMM). To quantify the impact of KCMA on transcription performance, we computed the measure G p , called Gain prior, and defined as F P rior k -F B 0 , where F B 0 is our reference F-measure obtained with our baseline algorithm BaseCaz1, and F P rior k the F-measure obtained by integrating the prior k to the transcription system. This gain can be either positive or negative, depending on whether it enhances or degrades transcription results of the PLCA baseline system algorithm. Figure 5.6 shows their impacts on transcription performance in terms of the measure G p . From this figure, we see that the G p values respective to each individual prior ranged from + 4.2 to -1.3. Also, the dataset R 1 is the only one to benefit from a positive G p for all priors, whereas the dataset R 4 is the one on which priors have the strongest negative impact, with only 4 positive contributions.

Testing combinations of KCMA

Best transcription gains

Table 5.5 shows the best transcription gains obtained from our baseline algorithms BaseCaz1 and BaseCaz2. By incorporating KCMA in AMT systems, we see that the prior gains brought in our system BaseCaz1(P 1 c ), which is as high as +6.6% for Velonjoro's repertoire, outperforms the state-of-the-art AMT algorithms reported in table 5.2. This tendency is confirmed through our different marovany repertoires, although the prior gains significantly differ from one repertoire to another.

Methods R 1 R 2 R 3 R tot BaseCaz1(P 1 c ) 64.1 % 66 % 67.3 % 60.7 % G p (P 1 c ) + 6.6 % + 5.7 % + 4.8 % + 2.1 % P 1 c {T 1, T 3, T 4, T 7, M 5} {T 1, T 5, M 2, M 5} {H1, T 1, T 5, M 2, M 4} {H2,

In details

We will now evaluate the impact of prior knowledge on AMT performance by considering different combinations of individual priors. To do so, we first formed all possible combinations of priors, and successively evaluated their impact on transcription results using each time the same test and learning sequences. In the same way that for figure 5.6, figure 5.7 plots the prior gain G p for each prior combinations and each dataset. It can be noted that the dataset R 1 has a positive prior gain G p over the 40 first best performing prior combinations, while the dataset R 2 , R 3 and R 4 cross the zero gain score at the ranks 36, 37 and 24, respectively. The higher this rank, and the higher the usefulness of knowledge prior in enhancing the transcription accuracy of a repertoire. It can then be mentioned that the repertoire R 1 benefits the most from prior knowledge. On the contrary, dataset R 4 has a negative prior gain G p from the 24 th prior combination.

To provide a more detailed insight in these prior combinations, figure 5.8 represents the relative proportions according to which each individual prior is present over the first 40 prior combinations presenting the highest G p values, for the four repertoires. For example, we can see that in most repertoires, high values are obtained for priors T 5 and T 6 , which mean that these priors are present in many of the best performing prior combinations.

Discussion

Our baseline performance remains not satisfactory enough to provide robust musical analysis supports, but they remain close to the average performance level of current AMT systems on polyphonic solo-instruments, which is around 60 % in the average F -measure with note-based evaluation metrics [START_REF] Benetos | Automatic music transcription: Challenges and future directions[END_REF]. By incorporating KCMA in our system BaseCaz1 (see table 5.5), we achieve transcription enhancements as high as +6.6% in the average F -measure for Velonjoro's repertoire, and outperforms significantly the state-of-the-art AMT algorithms reported in table 5.2 for all repertoires. This confirms well that model-driven transcription systems [START_REF] Ellis | Prediction-driven computational auditory scene analysis[END_REF], which make use of prior knowledge specific to their data under analysis, outperform globally data-based systems, whose more general application often costs a decrease in transcription performance.

We highlight the major contribution of pre-recorded templates per pitch in transcription tasks, which was particularly noticeable in contrast to the individual parametric spectral models, as in the HALCA algorithm. Templates have been used in patternmatching operations and are very efficient in multi-pitch tracking tasks [START_REF] Mysore | A non-negative framework for joint modeling of spectral structure and temporal dynamics in sound mixtures[END_REF]Benetos and Dixon, 2013). As far as the choice of templates is concerned, [START_REF] Kirchhoff | Shift-variant nonnegative matrix deconvolution for music transcription[END_REF] stated that the highest transcription accuracies are obtained when spectral templates are learned directly from the recording under analysis or from isolated notes of the same instrument model, whereas Benetos and Dixon (2013) seemed to suggest that having a large set of templates that might include instruments not present in the recording does in fact improve transcription accuracy. Also, Benetos and Dixon (2013) suggested the use of isolated sounds database to build spectral templates, instead of extracting them from solo performances. Our simulation experiments seem to confirm these last two statements, especially when adding a variety of template models able to encompass both the intrinsic timbre properties of an instrument and the acoustic modifications which can be induced to different playing style, as proposed by the RWC database [START_REF] Goto | Rwc music database: Popular, classical, and jazz music databases[END_REF]. Such a combination of different instrument templates might better approximate the spectra of the produced notes, as observed by Benetos and Dixon (2013). Prior gains associated to the use of templates show great promises for all marovany repertoires, and even higher transcription performance can be expected with a more advanced characterization of their instrument acoustics. One can speculate on the fact that more complex and variable acoustic signatures exist for these instruments due to their non-standardization and "precarity" of their making, in contrast to commercialized eurogenetic instruments. A higher acoustic complexity which has been indeed observed in Conference 4, particularly in regards to inharmonicity and spectral envelope variability. Further studies will be needed to validate a correlation between acoustic timbre complexity and the usefulness of timbre-related prior knowledge for non-eurogenetic repertoires. However, the current rarity of datasounds formatted for AMT evaluation of more traditional instrument repertoires is the first problem to be solved. In that direction, the authors have undertaken the development of a sound database dedicated to plucked string instruments of traditional African repertoires. Also, our results provide interesting basis to discuss inter-repertoire differences, which will be further explored in the following by studying how prior knowledge impact each repertoire transcription. It is the musical repertoire R 1 of Velonjoro which gets both the lowest baseline and HALCA transcription results and the highest positive gain from priors. In other words, this repertoire leaves much room for prior-based transcription improvements, and benefits the most from specific prior knowledge. The repertoire of Velonjoro consists of a succession of melodic phrases most often played in arpeggios. There is no vertical writing properly speaking in this music, excepting a few punctual chords. However, the high tempo at which notes are played, combined with the facts that strings are barely muted within a musical phrase and that they often resonate with each other, confers to this music a complexity of analysis comparable to that provided by polyphonic music. The high-level of polyphony and virtuosity style of this musical corpus constitutes classical difficulties for automatic transcription. In addition to that, Velonjoro tuning is quite deviating from the typical well-tempered scale, as well as "non-octaving", i.e. scale is not the same depending on the octave (e.g., in 1 st variation of Sojerina, the flat C in octave 3 is a natural C in octave 4 and 5). All these musical features contribute in decreasing transcription performance obtained with the HALCA algorithm, while their proper identification and characterization allowed the development of specific priors quite efficient on this repertoire. Mostly, this repertoire benefits the most from the KCMA category of timbre (see relations between KCMA and their categories in table 1.1), which fits well the fact that in the traditional musical playing of the marovany, players like Velonjoro interfere at the minimum on its intrinsic timbre, and even excite it as louder as possible (an effect called mafo be). In particular, prior T 3 (related to timbre) is at the same level as the other template-based priors T 5 and T 6 (related to playing style), which are much more predominant in the other marovany repertoires.

Concerning sequential prior knowledge, injected in the HMM framework, we can mention that the knowledge-based priors H globally outperforms the data-based prior M 3 for the modeling of harmonic transitions, resulting simply from the fact that taking into account musical knowledge such as tonality provides more accurate transition probabilities, which are robust to the different datasets, whereas the probabilistic note transition contains a superposition of the different tonalities, with a too local information to be representative of different musical repertoires. In repertoires with simpler harmonic structures, as in Velonjoro, the prior gains of these two types of priors (knowledge-based vs data-based) are rebalanced.

Priors integrating information on note duration and temporal envelop appears to be beneficial, especially for Velonjoro's repertoire. In particular, a previous in-depth analysis of model errors on an annotated marovany dataset showed that most of the false negatives are actually produced by a strong intermodulation on certain notes. The valleys appearing in temporal envelops of intermodulated notes are a strong cause of false alarm detection, as they can be seen as quick re-occurrences of a same note, which should actually be caused by particular musical figures (e.g. fast arpeggio). These two quite simple priors work surprisingly well against these errors, especially in Velonjoro's repertoire where notes are most often let in free-resonance.

Eventually, when considering the dataset R 4 , in comparison to the oracle datasets R 1 to R 3 , we observe that globally all prior contributions decrease, with more noticeable negative impacts on priors T 5 and T 6 , the two multi-template based priors. Then, we can state that even within a same instrument repertoire, the integration of prior knowledge should also take into account specificities from different musicians and instrument models, as is the case here for the marovany.

Results of AMT performance II : on inter-instrument repertoires

In this section, we wish to investigate the impact of KCMA incorporation in AMT systems on different instrument repertoires, especially between classical piano, folk guitar and the marovany. To do so, we will proceed using the same steps as in section 5.2. Our research publication Article 1 resumes a large proportion of the content of this chapter.

Evaluation sound dataset

Three different decay instruments were selected, namely classical piano, steel-string acoustic guitar and the marovany zither from Madagascar, labelled R 1 to R 3 . For the first two instruments, isolated note samples were extracted both from the RWC database [START_REF] Goto | Rwc music database: Popular, classical, and jazz music databases[END_REF]. For the marovany, all note samples were recorded by the authors in a semi-anechoic chamber of our laboratory, as no publically available sound databases exist for this instrument. For each pitch of an instrument pitch range, we have three different note samples from three different instrument models, for a total of 9 note samples per pitch per instrument type. These note samples are obtained by varying playing dynamic only (covering approximately the nuances of piano, mezzo forte, and forte), and with a fixed position of the excitation point, centred in the playing area, for the two plucked-string instruments.

For what concerns musical pieces, as in section 5.2, 12 different musical pieces were taken from each musical repertoire. We then selected 8 non-overlapping sequences of 30 seconds in each piece, for a total of 144 sequences (i.e. 48 minutes) per repertoire. For classical piano, musical pieces were taken randomly from the MAPS database [START_REF] Emiya | Multipitch estimation of piano sounds using a new probabilistic spectral smoothness principle[END_REF]. These pieces belong to the euro-genetic classical music from the 18 th and 19 th centuries. For our two plucked-string instruments, our musical pieces were randomly taken from our PSIFAMT database, corresponding respectively to R 2 = {D 10 : D 12 } and R 3 = {D 1 , D 4 , D 5 }. Figure 5.3 provides an illustrative example of a test sequence (cut to 15 seconds) from a classical piano piece (MAPS database), with different transcription outputs.

Performance of Baseline systems

In this first section, we test the performance of the different algorithms on the test bench, which allows placing our baseline transcription algorithm performance in the context of state-of-the-art algorithms. As displayed in table 5.6, our baseline system algorithm BaseCaz1 provided transcription results of 62.2 %, 59.7 % and 59.2 % for the different repertoires R 1 to R 3 , respectively. The best performing system is the one of Benetos2013, with a score of 64.8 % for the classical piano repertoire, which is of the order of magnitude found in the literature (Benetos et al., 2013a), and 61.5 % for the marovany repertoire.

Testing combinations of KCMA

Best transcription gains

Table 5.7 shows the best transcription gains obtained from our baseline algorithms BaseCaz1 and BaseCaz2. With the prior gain, we see that our AMT system BaseCaz1(P 1 c ) Figure 5.9 -Illustration of different stages of our BaseCaz2 system on a test musical sequence, with from top to bottom: ground truth, pitch activity matrix P (i, t) and pianoroll transcription output. outperforms the state-of-the-art AMT algorithms reported in table 5.6. Once again, the incorporation of optimal KCMA in our systems BaseCaz1 and BaseCaz2 enhances significantly our transcription performance, which brings it to the same order of magnitude as Benetos2013 for the classical piano, and outperforms this one for the marovany repertoires. Also, we can observe strong inter-repertoire differences in the impact of KCMA on the AMT system BaseCaz1(P 1 c ) , with a difference in G p (P 1 c ) as high as 3.1 % between the repertoires of classical piano R 1 and marovany R 3 .

In details

We will now evaluate the impact of prior knowledge on AMT performance by considering different combinations of individual priors. To do so, we first formed all possible combinations of priors, and successively evaluated their impact on transcription results using each time the same test and learning sequences. Figure 5.10 plots the prior gain G p for each prior combinations and each dataset. It can be noted that the dataset R 1 has a positive prior gain G p over the 40 first best performing prior combinations, while the dataset R 1 , R 2 and R 4 cross the zero gain score at the ranks 31, 33 and 15, respectively. The higher this rank, and the higher the usefulness of knowledge prior in enhancing the transcription accuracy of a repertoire. It can then be mentioned that the repertoire R 3 benefits the most from prior knowledge. On the contrary, dataset R 4 has a negative prior gain G p from the 15 th prior combination.

To provide a more detailed insight in these prior combinations, figure 5.11 represents the relative proportions according to which each individual prior is present over the first 40 prior combinations presenting the highest G p values, for the four repertoires.

Discussion

To begin with, it can be first noticed that all priors considered in our analysis induce enhancements in the transcription results of state-of-the-art algorithms, on at least one of the instrument repertoires. With the results from this section, some inter-repertoire differences related to KCMA incorporation are clearly highlighted. Numerically, prior gains range are of 3 %, 4.4 % to 6.1 %, respectively for the classical piano, folk guitar and marovany repertoires. This tendency can be first explained by the fact that transcription performance of baseline systems are higher for classical piano (see table 5.6), and so there is less room for improvement with the incorporation of musical knowledge. Also, transcription enhancements are more important for our two plucked-string instruments, in comparison to classical piano, but are more similar for knowledge-based KCMA (source I). This supports the idea that musical features related to both timbre and playing style may be more complicated in plucked-string instrument repertoires. This makes sense indeed when comparing the different playing techniques feasible on plucked-string instruments, such as plucking position and excitation modes.

Then, the results of this section 5.3 learns us that the marovany repertoire benefits the most from musical knowledge, whatever musical knowledge is considered, whereas our "universal" dataset R 4 calls for more carefully selected and well-dosed external information (in our simulation experiments, most knowledge components result unnecessary and often detrimental), risking to decrease drastically transcription results. A tendency supported by the important predominance of very specific musical information which do not depend on data, such as the known played notes and the tonality-based probabilistic transitions between note mixtures. Such a result suggests the non-universality of prior knowledge, and the consequent need to adapt information depending on repertoires. Also, when comparing this universal dataset from section 5.3 to the one from section 5.2, we see that the transcription performance gains from KCMA are less degraded within the marovany repertoires.

Like for results from section 5.2, the most stable KCMAs are, as expected, the ones from the Source I, whose contributions are not really affected by the mixing of the different repertoires in the dataset R 4 . Such KCMAs are naturally in generic AMT systems (see e.g. [START_REF] Benetos | Improving automatic music transcription through key detection[END_REF]), although our results reveal that their performance gains are quite limited in comparison to KCMA specific to an instrument repertoire.

Conclusion

In this chapter, we were interested in performing an in-depth investigation of their impact on music transcription. In order to identify prior contributions in transcription results, we used only knowledge components built on explicit information from various music-related domains of knowledge, namely the timbre, musicological and playing style classes. Our numerical experiments lead us to several interesting observations on the use of knowledge in a state-of-the art AMT system, such as the complementarity of the different knowledge classes and the non-universality of prior knowledge in regards to the diversity of instrument repertoires across the world.

Conclusion Summary content

This PhD thesis deals with the development of an automatic transcription system dedicated to the repertoires of the traditional Marovany zither from Madagascar. The common denominator of the different approaches developed to the task of AMT lays in the use of explicit music-related prior knowledge in computational systems. The AMT framework was build upon two main methods, namely Probabilistic Latent Component Analysis (PLCA) for multi-pitch estimation and Hidden Markov Models (HMMs) for note segmentation and sequential post-processing, and different configurations of these methods were developed to provide a powerful probabilistic framework, covering the time-frequency domain on different time-scales, in which original integration methods of prior knowledge were developed. We were interested in developing a generic transcription system based on a PLCA post-processed with a HMM block, which makes possible to supply it with all kind of prior knowledge at different levels of abstraction, and performing an in-depth investigation of their impact on music transcription. In order to identify prior contributions in transcription results, we used only knowledge components built on explicit information from various music-related domains of knowledge, namely the timbre, musicological and playing style classes. Our numerical experiments lead us to several interesting observations on the use of knowledge in a state-of-the art AMT system, such as the complementarity of the different knowledge classes and the non-universality of prior knowledge in regards to the diversity of instrument repertoires across the world.

Also, an important part of this PhD project dealt with the constitution of a database of ground-truth transcriptions of a not yet studied instrument repertoires. This involved the development of specific acquisition and processing tools to automatize this process, and takes part to a larger ambition of making available to the MIR community a sound database dedicated to various plucked-string instrument repertoires, especially including traditional ones.

Review of Objectives

In terms of our initial objectives, this PhD has achieved the three main following points -First, the development of Multichannel Capturing Sensory Systems dedicated to plucked-string instruments, and more specifically to the marovany, has achieved to provide numerous reliable transcriptions of instrument repertoires, in an automatic and time-saving way. Such transcriptions have been exploited as ground truth transcriptions in the task of Automatic Music Transcription (see below), and also in computational musicological studies ; -Second, the development of an automatic music system with original knowledge components from musical acoustics incorporated in it, specially conceived and op-timized for the marovany instrument repertoire. A first major result was the greater transcription performance of templates learned from pre-recorded samples, including "interpreted" note samples which result from specific playing techniques, in comparison to more generic parametric note models. The timbre of marovany, which can vary a lot in playing condition, on the contrary to notes of classical piano for example, then really benefit from a more instrument-specific modeling of note spectral envelops. Other knowledge components specific to this instrument repertoire include sympathetic resonances, intermodulated notes, shift-invariant templates, pitch-wise note duration. Optimal incorporations of this knowledge in music transcription systems have allowed significant enhancements in transcription performance, outperforming state-of-the-art systems ; -Third, the development of a vast set of knowledge components from musical acoustics, covering both timbre and music language features of an instrument repertoire, have allowed investigating the impact of knowledge components on different instrument repertoires. We then highlighted significant inter-repertoire differences through the incorporation of these components in our baseline music transcription system. We really wish this work to help researchers in the community of computational ethnomusicology, who would like to apply tools of automatic music transcription to their repertoires. The analytical framework developed in this PhD should help such researchers in choosing the appropriate knowledge to incorporate in computational systems, as we showed that it is inappropriate to use all music knowledge, in terms of both the quantity and nature, regardless the instrument repertoire under study. The methodology and results of this PhD should also help in better understanding how current music transcription algorithms could be adapted to new instrument repertoires. Through the explicit links we made between musical acoustics knowledge and automatic transcription accuracy, one could directly predict the best knowledge components to use for transcription after having identified and characterized the most characteristic musical features in its repertoires (e.g. highly variable spectral envelops of played notes → learning of pre-recorded templates; non-tempered tuning → shifting templates in frequency)

Prospects

Dealing with musicological issues, and ethnomusicological issues in particular, is not a simple subject for computer scientists, as explained by [START_REF] Lartillot | Commentary on "comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction" by emilia gomez, and perfecto herrera[END_REF]: "We hope expert ethnomusicologists will understand the experimental aspect of such a crossdisciplinary undertaking, and will pardon the potential imperfection in this computational attempt toward cross-cultural understanding. The argumentation of the paper might betray for some readers some residual of Eurocentrism. We would like to emphasize that the authors of this work are aware of the limitation of this opposition between Western and non-Western music, which is proposed as an experimental state before the development of further research aimed at the modeling of "stylistic features proper to different stylistic areas"."This PhD thesis tried at best to follow these recommendations, while bringing clear evidences that the study of a non-eurogenetic repertoire such as the marovany repertoire raises specific issues, and a consequent need of adapting and developing specific tools to answer them. Also, the authors are aware of focusing on a very specific music culture, which make the selected priors and results not easily generalizable. However, there are clear analogies and similar phenomenon occurring in many music cultures. Therefore we believe this document can be a good reference to many researchers working in the AMT tasks, and also a useful starting point in tailoring AMT systems as an aid to musicological studies involving melody.

Our research interests for future studies will be to test the validity of our results on sound databases with real recordings, to quantify the precise representativeness and amount of musical pieces in regards to a complete instrument repertoire, and to undertake the development of a semi-automatic transcription tool, integrating a large and flexible number of prior knowledge which can all be directly understood and modified by an external user.

For what our PSIFAMT database, the motivations of developing this new sound database are multifold:

-Making available a new sound database for AMT evaluation in the MIR community, which is now indispensable to test the robustness of current algorithms to music diversity, and face them to more complex musical features ; -Plucked-string instruments present a great diversity in terms of timbre (e.g. inharmonicity, envelop spectrum variability, temporal profile modulation), playing styles and effects (e.g. different string excitation modes, palm muting, glissandi, vibrato, tremolo, harmonic notes) ; -Put the attention on non-eurogentic traditional repertoires, encompassing the greatest diversity of musicological codes (e.g. micro-rhythm, tuning based on non-equal temperament, modal scale) and of musical practice, and complexity in music encoding ; We wish to complete the formatting of this sound database so it can be permanently hosted on the Web, and keep on expanding it with other recordings from new instruments.

Complementary research studies

The marovany in ImproteK Some research (e.g. [START_REF] Dubnov | Using machine-learning methods for musical style modeling[END_REF]) seek to capture some of the regularity apparent in the composition process by using statistical and information-theoretic tools to analyze musical pieces. The resulting models can be used for inference and prediction and, to a certain extent, to generate new works that imitate the style of the great masters. A generative theory of music can be constructed by explicitly coding music rules in some logic or formal grammar. HMMs have been used in this context of style modeling, allowing to build a computational representation of the musical surface that captures important stylistic features. Statistical analysis of a corpus reveals some of the possible recombinations that comply with the constraints or redundancies typically found in a particular style. Interesting applications include style characterization tools for the musicologist, generation of stylistic metadata for intelligent retrieval in musical databases, music generation for Web and game applications, machine improvisation with or without interaction with human performers, and computer-assisted composition. One of machine learning's main purposes is to create the capability to sensibly generalize.

While the real-time conversion of an acoustic instrument into a multichannel numerical signal provides a robust optimal method for audio acquisition and post-processing, it also opens original artistic prospects. The capacity of integrating real-time audio transformations (filtering, dynamic and timbre changing âĂę) in live performance has already found some interests in renowned cithara players like Rajery. Also, a MIDI conversion method of the cithara audio signal could also bring new applications in the field of musician-machine interaction systems. Actually this study is connected to the IMPROTECH project entitled "Technologies and Musical Improvisation" (grant ANR-09-SSOC-068 by the French National Research Agency) that includes a partnership with IRCAM. A computer environment called OMax, which learns in real time from human performers, has been developed in this context. The improvisation kernel is based on sequence modeling and statistical learning. It allows capturing stylistic musical surface rules in a manner that allows musically meaningful interaction between humans and computers. Recently an international workshop was held in New York ? dedicated to the exploration of the links between musical improvisation and digital technologies that gathered researchers and artists from both research & creation scenes. Through this current study we aim to connect our optical-based MIDI conversion system developed for the marovany cithara to improvisation environments such as OMax, in order to explore the musical interest of such an interaction and help folk music to be represented in such events.

Study of the marovany repertoire in trance tromba, founded on musical criteria, and complementing other behavioral indices observed with an audio-visual device and audio data, should bring original elements of investigation to the fascinating relationships between music and trance. Another application theme of such a system would be the Human-Machine musical interaction, through the OMAX improvisation IT environment [START_REF] Nika | Improtek, integrating harmonic controls into improvisation in the filiation of omax[END_REF] (developed from the OMax environment Assayag et al. (2010Assayag et al. ( -2014) ) in collaboration with IRCAM). Future musical projects could involve malagasy musicians in this environment, using MIDI data from our retrieval system. Questions of a more aesthetic character (acceptability of musical formulas derived from a known repertoire, oral transmission of this skill, musical interest in the amplification, virtual re-orchestration of a musical environment and real-time modifications of musical parameters) will be considered in future investigations following this direction.

Research publications This thematic has been the subject of the following research publications:

Annex

Appendix A

Acoustic & Musical Descriptors

The features are calculated using a short time analysis window with duration 10-40 milliseconds. In addition, the means and variances of the features over a larger texture window (0.2-1.0 seconds) are computed resulting in a feature set with eight dimensions.

A system for the extraction of audio descriptors is usually organized according to the properties of the descriptors. We can distinguish three main properties of an audio descriptor: (1) the temporal extent over which the descriptor is computed (a specific region in time, such as the sustain, or the whole duration of a sound file), ( 2) the signal representation used to compute it (e.g., the waveform, the energy envelope or the short-term Fourier transform), and (3) the descriptor concept described by it (e.g., the description of the spectral envelope or the energy envelope over time) [START_REF] Peeters | The timbre toolbox: Extracting audio descriptors from musical signals[END_REF]. We first used classical descriptors covering both physical domains, temporal and spectral, allowing to quantify temporal variations of acoustic properties within a sound unit. Most of them have already been used in the acoustic characterization of mysticetes sound units [START_REF] Fristrup | Characterizing acoustic features of marine animal sounds[END_REF][START_REF] Au | Acoustic properties of humpback whale songs[END_REF][START_REF] Mercado | Sound production by singing humpback whales[END_REF]. They were selected for their physical meaning easily interpreted.

The temporal extent denotes the segment duration over which the descriptor is derived. A descriptor can either directly represent the whole sound event (e.g., the Attack Time descriptor, because there is only one attack in a sound sample) or represent a shortduration segment inside the event (e.g., the time-varying spectral centroid, which is derived from a spectral analysis of consecutive short-duration segments of a sound, usually of 60 ms duration).

Descriptors of the first group are called "global descriptors," and those of the second group are called "time-varying descriptors." Time-varying descriptors are extracted within each time frame of the sound and therefore form a sequence of values. In order to summarize the sequence in terms of a single value, we use descriptive statistics, such as minimum or maximum values, the mean or median, and the standard deviation or interquartile range (i.e., the difference between the 75th and 25th percentiles of the sequence of values). As such, the structure of an audio descriptor system usually separates the extraction of global descriptors (which are directly considered as the final results) from the extraction of time-varying descriptors (which are subsequently processed to derive the descriptive statistics).

A.1 Global descriptors

AT , the attack time (in s) is defined by the necessary time for the signal to reach 95 % of its maximal energy E max All descriptors will be computed computed in a 0.049-s (1024 samples) hamming window at 0.01-s intervals with a 0.005-s overlapping. Classical speech analysis is usually conducted over such a duration.

Energy , the energetic level rms (in Pa) of a signal is defined by

E(k) = 1 K K k=1 |(x + kN )| 2 (A.7)
computed for K successive frames of N samples. Given the difficulty of characterizing amplitude values of sounds recorded from free-ranging animals in an open acoustic environment [START_REF] Au | Acoustic properties of humpback whale songs[END_REF], as the amplitude of any field-recorded sound is necessarily affected by a number of sources of measurement error, including variations in the distance between source and microphone, in the relative orientations of source and microphone, in the effects of intervening sound barriers, and in ambient background noise levels, these figures on the energy must be only seen as relative indicators allowing to compare very roughly the intensity levels respective to each type. Also, since microphone sensitivity (or clipping level) is often unavailable, sound levels could only be found as values relative to some arbitrary level, chosen such that the weakest click level corresponded to 0 dB. Hence, we report only "relative vocalization levels", by which we mean the difference between the current click level and the minimum click level (over all clicks) ; HD, the Harmonicity Detector (adimensioned) is an indicator of harmonicity. The principle [START_REF] Youngmoo | Singer identification in popular music recordings using voice coding features[END_REF] is to automatically scan the spectral density of a signal with a comb filter whose fundamental frequency F 0 and varies within a given range of interest. When the valleys of this filter coincides with the peaks of an harmonic sequence for a particular F 0 , their product will result in a very weak value which traduces the presence of an important harmonicity.

Mathematically, we define it as .8) with E init = |Y (k)| 2 and E pond = F ilt(k, k o )E init , where Filt is a comb filter defined as Filt = 2(1 -|cos( πF F 0 )|). SGC, the Spectral Gravity Center is the average value of the spectral energy distribution, which divides a spectral profile in two parts of equal energy :

HD = min( E pond E init ) (A
SGC = K k=0 f k a k K k=0 a k (A.9)
where a k represents the spectral amplitude of the sample k, f k its bin and K the total number of bins ; ROF, the roll-off point estimates the amount of high frequency in the signal consists in finding the frequency such that a certain fraction (α ROF ) of the total energy is contained below that frequency. This ratio is fixed by default to .85 [START_REF] Tzanetakis | Musical genre classification of audio signals[END_REF]. We can define as The natural logarithm, the logarithm base e with e being Euler's number, is noted as ln. To convert a value in cent c to a ratio t: t = exp cln( 2) 1200 (B.9)

Discussion of the different representations Further discussion on cents as pitch ratios can be be found in appendix B of [START_REF] Sethares | Tuning timbre spectrum scale[END_REF]. There it is noted that: There are two reasons to prefer cents to ratios: Where cents are added, ratios are multiplied; and it is always obvious which of two intervals is larger when both are expressed in cents. For instance, an interval of a just fifth, followed by a just third is (3/2)(5/4) = 15/8, a just seventh. In cents, this is 702 + 386 = 1088. Is this larger or smaller than the Pythagorean seventh 243/128? Knowing that the latter is 1110 cents makes the comparison obvious. The cent unit is mostly used for pitch interval representation while the midi key and Hz units are used mainly to represent absolute pitch. The main difference between cent and fractional midi note numbers is the standardized reference frequency. In our software platform Tarsos we use the exact same standardized reference frequency of 8.176 Hz which enables us to use cents to represent absolute pitch and it makes conversion to midi note numbers trivial. Tarsos also uses cents to represent pitch intervals and ratios. t is a weight attached to particle x (i) t . Given this particle approximation to the posterior distribution, we can estimate the expected value of any function f w.r.t the distribution I(f, t), defined as I(f t ) = f (x t )p(x t |y 1:t )dx t , using the following Monte Carlo approximation

B.1 Table of main harmonic relations

I(f t ) ≈ N i=1 f (x (i) t )w t (i) (C.8)
Particle smoothers generate batched realisations of p(x 1:T |y 1:T ) based on the forward PF results. In other words, the particle smoothers are an efficient method for generating realisations from the entire smoothing density p(x 1:T |y 1:T ) using filtering approximation.

C.1.1 Filtering

We consider the filtering distribution p(x t |y 1:t ). Using the Bayes' rule, this distribution can be rewritten as follows, Then, assuming that f (x t |x t-1 ) and g(y t |x t ) can be evaluated pointwise, we generate, for each state trajectory x (i) 1:t-1 , a random sample from a proposal distribution q(x t |x (i) 1:t-1 , y 1:t ). Then, the weights w t of the filtering distribution (eq. C.6) can be approximated by

w (i) t ≈ g(y t |x (i) t )f (x (i) t |x (i) t-1 ) q(x (i) t |x (i) 1:t-1 , y 1:t ) (C.14)
Finally, we perform a multinomial resampling step, such that the probability that x (i) t is selected is proportional to w (i) t , to obtain an unweighted approximate random draw from the filtering distribution p(x t |y 1:t ). It is noteworthy that if the resampling step is forgotten, a degeneracy phenomenon can occur. Indeed, after a few iterations, all but one particle will have negligible weight. [START_REF] Doucet | On sequential monte carlo sampling methods for bayesian filtering[END_REF] has shown that the variance of the importance weights can only increase over time, and thus, it is impossible to avoid the degeneracy phenomenon. This degeneracy implies that a large computational effort is devoted to updating particles whose contribution is almost zero. As a result, a resampling step is needed to eliminate particles with small weights and generate a new set {x (i) t } i , which is an i.i.d. (independent and identically distributed) sample from the approximate density p(x t |y 1:t ), with a resetting of the weights {w

C.1.2 Smoothing

The entire smoothing density p(x 1:T |y 1:T ) can be factorized as :

C.2 Our contributing work

The main objective of this paper is to propose an alternative formulation of current PLCA models applied to audio signals, replacing the EM algorithm by a more generic parameter estimation algorithm based on a PF method. We call this new algorithm PLCA-PF in the following. The main advantage expected from this new algorithm is to be able to scan the whole parameter space so as to take into account any features of the parameters, and thus overcoming the limitations underlined in our introduction specific to current PLCA models. In regards to prior integration particularly, this new framework allows releasing the constraints on prior mathematical forms and number. This paves the way towards more complete modelings of the multi-faceted information carried by musical signals, covering both time (e.g. tempo and rhythm) and frequency (e.g. note spectra and chords) domains, and the different prior knowledge classes related to musicology, timbre and playing style. 2 On the left, graphical representation of transition probabilities between three hidden states. On the right, probability density functions for the observed data given the underlying state of the model y t is the observation at time t, and q t is the underlying state label at time t. . . . . . . . . . . .

List of Figures

NF 0

 0 boldA Number of elements in the set A () ι Inter-Pitch Matrix of mutual influences between the I × I couples of different pitches ι ij Likelihood scores between pitches i and j i Pitch index, i ∈ {1, • • • , I} I Number of pitches M Set of note mixtures M t k Note mixture indexed by its starting time t k PDF Probability Density Functions t ∈ [1,T ] time-frame index, with T the number of frames f ∈ [1,F ] frequency-bin index f t frequency of bin with time-frame index t Fundamental frequency, or pitch F s Sample frequency Sound Database MAPS MIDI Aligned Piano Sounds

  x n (t) noise sensor signalMPE parametersU i,k pitch-wise activation intervals for pitch i, indexed by k in timeL N (i,k) list ofbinary note candidates for pitch i, indexed by k in time LN (i,k) list of probabilistic note candidates for pitch i, indexed by k in time PLCA parameters m Playing mode index M n Binary mixtures of notes N Mn Number of estimated mixtures of notes Mn Probabilistic mixtures of notes

Figure 1 -

 1 Figure 1 -A general musical chain production with the successive steps of (1) Musical Knowledge, (2) Musical Production, (3) Signal Acquisition and (4) Signal Analysis.

FrequencyFigure 2 -

 2 Figure 2 -Spectrogram illustrating the different hierarchical structures present in musical signal. This 10-s musical excerpt comes from the piano piece Hungarian Rhapsody N o 2 by Liszt. Spectrogram parameters: sampling rate F s : 44.1 kHz, frame size: 22 ms (1024 samples), 50% overlap (temporal resolution: 11 ms), FFT size: 1024 samples (spectral resolution: 11 Hz), Hamming window.

Figure 3 -

 3 Figure 3 -Spectrogram of a 15-s musical signal, with its piano-roll plotted below.

  Photo-Credits:-http://www.museedelhistoire.ca/ cmc/exhibitions/arts/resonance/res1_14f.shtml

Figure 5 -

 5 Figure 5 -Photos of the four marovany models used in this PhD, labelled from N 1 to N 4 .

  Figure 1.2 shows an example of a GMM-based parametric model (top graph) for the note mixture C 4 , F 4 , G 4 , A 4 #, with its corresponding perceptual binwise loudness l k (middle graph) and perceptual loudness wrapped within an octave (bottom graph).

Figure 1 . 2 -

 12 Figure 1.2 -Example of a GMM-based parametric note mixture model (top graph, each color representing a note model) for the note mixture C 4 , F 4 , G 4 , A 4 #, with its corresponding perceptual bin-wise loudness l k (middle graph) and perceptual loudness within an octave v i (bottom graph).

Figure 1 . 3 -

 13 Figure 1.3 -Major and minor chords (from http://hymns.reactor-core.org/ keysignatures.html).

Figure 1 . 5 -

 15 Figure 1.5 -State transition matrices of musicological modeling 1. It gets its knowledge from the theoretical keys described in Sec. 1.2.2.

  Figure 1.7 -Krumhansl tone profiles for major and minor keys.

  Figure 1.8 -Illustration in the temporal waveforms and spectra of two different notes, with pitches C 4 (on the top) and D 2 (on the bottom).

Figure 1 . 9 -

 19 Figure 1.9 -Illustration of the phenomena of inharmonicity in the note D 2 of marovany.

F3Figure 1

 1 Figure1.10 -Box plots of different timbre descriptors, computed on an isolated note dataset of a single marovany model (on the left), and on four different models of marovany (on the right). From top to bottom, these descriptors are the duration, the intermodulation, the spectrum inharmonicity and the spectrum variability.

  Figure 1.11 -Illustration of sympathetic resonances through an inter-pitch influence matrix ι, computed on a marovany model.

Figure 1 .

 1 Figure 1.12 -Theoretical tuning for the marovany model N 3 based on equal temperament.

Figure 1 .

 1 Figure1.13 -Tuning deviation (in cents) from the equal temperament against MIDI pitches, computed for a marovany musical repertoire.

  Figure 1.15 provides an example of such a matrix.

Figure 1 .

 1 Figure 1.14 -Pitch co-occurrence probability matrix corresponding to a data-based modeling with a simple frequency counting of note mixture observations.

Figure 1 .

 1 Figure 1.15 -Representation of the different note mixtures (on the left), and log-transition probability matrix between these mixtures obtained with a simple frequency counting (on the right).

Figure 1

 1 Figure 1.16 -Ground-truth example of the motive structure in the marovany musical piece called Folera by Kilema.

Figure 1

 1 Figure1.17 -Example of a motif dictionary computed from the instrument repertoire of classical piano.

Figure 2 .Figure 2 . 1 -

 221 Figure 2.1 -Schematic diagram of the PhD organization for chapter 2.

Figure 2 . 4 -

 24 Figure 2.4 -On left graphs, sample histograms of the salience values s(i|i) in the training data when the notes with pitch i were played according to the annotation (bars), and the corresponding model PDFs H (i) (s(i|i)) (red curves). On right graphs, sample histograms of the salience values s(i| ī) in the training data when the notes with pitch i were not played according to the annotation (bars), and the corresponding model PDFs H(i) (s(i| ī)) (red curves).

s

  t1 (i|i) = median([0.9, 0.8, 0.8, 0.8]) = 0.8, for i = 1 s t1 (i| ī) = median([0.2, 0.1, 0.3, 0.2]) = 0.2, for i =

Figure 2 . 5 -

 25 Figure 2.5 -Illustrative example explaining the computation of the state-conditional likelihood probability.

Figure 3 .Figure 3 . 1 -

 331 Figure 3.1 -Schematic diagram of the PhD organization for chapter 3.

Figure 3 . 2 -Algorithm 4

 324 Figure 3.2 -Activation curve obtained with three different AMT systems, on an intermodulation note. Our intermodulation-informed smoother is also plotted, which results in flatting the activation dip.

Figure 4 .

 4 1 illustrates the relation of this chapter with the others.

Figure 4 . 1 -

 41 Figure 4.1 -Schematic diagram of the PhD organization for chapter 4.

Figure 4 . 2 -

 42 Figure 4.2 -Photos of the two models of optical sensors tested, with on the left the EE-SX398/498 model, and on the right the OPB610 model.

Figure 4 .

 4 Figure 4.3 shows photos of the set-up installation, where all optical sensors are fixed on a vertical bar along the strings, and also this MCSS in condition of playing.

Figure 4 . 3 -

 43 Figure 4.3 -Photos of the set-up installation, where all optical sensors are fixed on a vertical bar along the strings, and also this MCSS in condition of playing

Figure 4 . 4 -

 44 Figure 4.4 -Scheme of the functioning principle of piezolectric sensors (from http://www. cui.com/product-spotlight/piezo-and-magnetic-buzzers/).

Figure 4 . 5 -

 45 Figure 4.5 -Photos of the two models of piezolectric sensors tested, with on the left the Gold RMC pick-up, and on the right the Variax piezoelement.

Figure 4 .

 4 Figure 4.6 -Photos of the two models of electromagnetic sensors tested, with on the left the GK3 model, and on the right our handcrafted sensors.

Figure 4 . 7 -

 47 Figure 4.7 -Mechanical device used to provide reference excitation amplitudes.

Figure 4 . 8 -

 48 Figure 4.8 -Illustration of the salience measures S1 and S2, respectively the salience and the sharpness of the onset, through the onset detection function F odf .

Figure 4

 4 Figure 4.9 -Illustration of the inter-note temporal interval T s measure.

Figure 4 .

 4 Figure 4.10 -Numerical scores on our signal criteria for the different sensor types.

Figure 4 .

 4 Figure 4.11 -Block diagram.

Frequencyh

Figure 4 .

 4 Figure 4.12 -Examples of spectral templates for the two PLCA latent classes of instrument notes and noise.

Figure 4 .

 4 Figure 4.13 -Screen shot of our semi-automatic transcription system to extracted instrumental notes from our MCSS signals.

Figure 4 .

 4 Figure 4.14 provides the ROC (Receiver Operating Characteristic, see[START_REF] Kay | Fundamentals of Statistical Signal Processing[END_REF] for details) curves with the error metrics FPR and TPR on our monophonic transcriptions of sensor signals, for our two feature-based and PLCA-based methods. Solid lines show transcription performance with an evaluation based only on pitch and onset location, and dashed lines show performance with a full evaluation based on the four parameters of a note, i.e. adding duration and amplitude. Our PLCA-based method provides the best performance for this transcription task, with accuracy (average F-measure) higher than 96 % over a large interval of thresholds (i.e. 0.11 -0.87). When considering all note parameters, transcription performance decreases.

Figure 4 .

 4 Figure 4.14 -ROC curves with the error metrics FPR and TPR on our monophonic transcriptions of sensor signals, for our two feature-based and PLCA-based methods. Solid lines show transcription performance with an evaluation based only on pitch and onset location, and dashed lines show performance with a full evaluation based on the four parameters of a note, i.e. adding duration and amplitude.

Figure 4 .

 4 Figure 4.15 -Photos of the different elements of our recording set-up.

  Website: http://www.aido.fr/kilema.html Charles Kely Born in Tananarive, the capital of Madagascar, Charles Kely is a professional player of guitar, marovany, valiha. He begun his career with his brother by covering traditional malagasy songs that they adapt to folk music. In 1997, Charles Kely is chosen by Rajery, the great valiha player, to tour around the world with his band. Together, they play in Paris, Chicago, Seattle, New Orleans and at the International festival of Louisiane. Since 2008, as a guitarist, Charles has also toured with prestigious world music artists as Tony Rabeson, Mounira Mitchala, Razia Said... He is now focusing on his solo career with the release of the CD Zoma Zoma in his own and unique manner : the open gasy style, an acoustic music from Madagascar, with a touch of bossa, jazz, blues, funk and subtle African influences.Website: http://www.charleskely.com/VelonjoroKilema Charles Kely

Figure 4 .

 4 Figure 4.16 -Photos of the marovany musicians recorded in the PSIFAMT database.

Figure 5 .

 5 1 illustrates the relation of this chapter with the others.

Figure 5 .

 5 Figure 5.1 -Schematic diagram of the PhD organization for chapter 5.

4 .Figure 5 . 2 -

 452 Figure 5.2 -Distribution of Inter-Onset intervals on the complete repertoire of the marovany., including in red the tolerance threshold ∆ Onset for onset evaluation. This distribution present two broad beaks around 120 ms and 250 ms.

  Figure 5.3 provides an illustrative example of a test sequence (cut to 15 seconds) from the marovany piece Folera played by the musician Kilema, with different transcription outputs.

Figure 5 . 3 -

 53 Figure 5.3 -Illustration of different stages of our AMT system on a test musical sequence, with from top to bottom: ground truth, pitch activity matrix P (i, t), piano-roll transcription output using a simple thresholding and piano-roll transcription output using the combination of priors {T 2, T 4, T 5, M 4} (taken from our Article 2).

Figure 5 . 4 -

 54 Figure 5.4 -Evolution of the F -measure (in %) against the value of the fixed threshold T hres f ix (in dB) for three different baseline methods. ST is short for the Simple Threshold method from section 2.3.3 and AT for the Adaptive Threshold method from section 2.3.4.

Figure 5 . 5 -

 55 Figure 5.5 -Comparison of transcription performance based on ROC curves using our different template learning methods, presented in section 2.2.8, in our baseline PLCA model BaseCaz1.

Figure 5

 5 Figure 5.6 -Prior gains G p for each individual prior, detailed for the different datasets.

Figure 5

 5 Figure 5.7 -Prior gains G p of the first 40 best ranking prior combinations, detailed for the different datasets.

Figure 5

 5 Figure 5.8 -G p values cumulated over the first 40 best ranking prior combinations, respective to each prior, and for the four different datasets R 1 to R 4 .

Figure 5 .

 5 Figure 5.10 -Prior gains G p of the first 40 best ranking prior combinations, detailed for the different datasets.

Figure 5 .

 5 Figure 5.11 -G p values cumulated over the first 40 best ranking prior combinations, respective to each prior, and for the four different datasets R 1 to R 4 .

FMA(

  Folk Music Analysis workshop), Paris, France, June 10th-12th, 2015 Seminar Seminar 1 Cazau, D. "Multichannel capturing sensory system for the marovany zither of Madagascar", Seminar at EHESS, Paris, 2014 May 14 th , video available on the web 4 . Seminar 2 Cazau, D. "Vers une automatisation de l'analyse ethnomusicologique ? Un bilan", Journées Doctorales d'Ethnomusicologie at Sorbonne, Paris, France, 2014 October 18 th . Book chapter Book 1 Cazau, D.; Chemillier, M. and Adam, O. (Accepted) "Computational Music Analysis approaches for automatic transcription of orally transmitted repertoires, with application to human-machine interaction environment", in book : Trends in Music Information Seeking, Behavior, and Retrieval for Creativity, Editors: Kostagiolas, P., Martzoukou, K. and Lavranos, C. Technical note Note 1 Cazau, D. and Nuel, G. "Understanding the Probabilistic Latent Component Analysis Framework" Note 2 Cazau, D. and Nuel, G. "Relating sigmoid parameters to explicit musical features in view of HMM-based note segmentation" Other research works -Adam, O.; Cazau, D.; Gandilhon, N.; Fabre, B.; Laitman, J. T. and Reidenberg, J. S. (2013). New acoustic model for Humpback whale sound production, Applied Acoustics, 74, 1182-1190 -Cazau, D.; Adam, O.; Laitman, J. T. and Reidenberg, J. S. (2013). Understanding the intentional acoustic behavior of humpback whales: a production-based approach, J. Acoust. Soc. Am., 134, 2268-2273 -Gandilhon, N.; Adam, O.; Cazau, D.; Laitman, J. T. and Reidenberg, J. S. (2014). Two new theoretical roles of the laryngeal sac of humpback whales, Marine Mammal Science, DOI: 10.1111/mms.12187 4. http://www.dailymotion.com/video/x1zkcla_kilema-ehess-5-7_music

  reference frequency r Equations (A5) and (A2) differ only by a constant factor of exactly 100. In an environment with pitch representations in midi note numbers and cent values it is practical to use the standardized reference frequency of 8.176 Hz.r = r.2 c/2000 (B.7)Savart & Millioctaves Divide the octave in 301.5 and 1000 parts respectively, which is the only difference with cents.Pitch ratio representation Pitch ratios are essentially pitch intervals, an interval of one octave, 1200 cents equal to a frequency ratio of 2/1. To convert a ratio t to a value in cent c:

  p(x 1:T |y 1:T ) = p(x T |y 1:T ) T -1 t=1 p(x t |x t+1:T , y 1:T ) (C.15) Using the filter approximation (eq. C.6) to p(x t |y 1:t ) and the Markovian assumptions of the model, we can write, p(x t |x t+1:T , y 1:T ) ∝ p(x t |y 1:t )f (x t+1 |x t ) distribution can be used to generate states successively in the reverse-time direction, conditioning upon future states.

  chain production. . . . . . . . . . . . . . . . . . . . . . . 2 Spectrogram of a musical sequence showing hierarchical structures. . . . . . 3 Spectrogram and piano-roll of a musical sequence. . . . . . . . . . . . . . . 4 Photo a valiha and a map of Madagascar. . . . . . . . . . . . . . . . . . . . 5 Photos of the four marovany models used in this PhD. . . . . . . . . . . . . 1.1 Schematic diagram of the PhD organization for chapter 1. . . . . . . . . . . 1.2 Example of a GMM-based parametric note mixture model (top graph, each color representing a note model) for the note mixture C 4 , F 4 , G 4 , A 4 #, with its corresponding perceptual bin-wise loudness l k (middle graph) and perceptual loudness within an octave v i (bottom graph). . . . . . . . . . . . . 1.3 Major and minor chords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Transition probability matrices of tonality . . . . . . . . . . . . . . . . . . . 1.5 State transition matrices of musicological modeling 1. It gets its knowledge from the theoretical keys described in Sec. 1.2.2. . . . . . . . . . . . . . . . 1.6 Doubly-nested circle of fifths . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 Krumhansl tone profiles for major and minor keys. . . . . . . . . . . . . . . 1.8 Illustration in the temporal waveforms and spectra of two different notes . . 1.9 Illustration of the phenomena of inharmonicity . . . . . . . . . . . . . . . . 1.10 Box plots of different timbre descriptors . . . . . . . . . . . . . . . . . . . . 1.11 Illustration of sympathetic resonances through its inter-pitch influence matrix ι ij . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.12 Theoretical tuning for the marovany model N 3 based on equal temperament. 1.13 Tuning deviation (in cents) from the equal temperament against MIDI pitches 1.14 Pitch co-occurrence probability matrix corresponding to a data-based modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.15 Representation of the different note mixtures (on the left), and log-transition probability matrix between these mixtures obtained with a simple frequency counting (on the right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.16 Ground-truth example of the motive structure . . . . . . . . . . . . . . . . . 1.17 Example of a motif dictionary computed from the instrument repertoire of classical piano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Schematic diagram of the PhD organization for chapter 2. . . . . . . . . . . 2.

Table 2 -

 2 Physical characteristics (L=Length, H= Height, W=Width) of the four marovany models used in this PhD, labelled from N 1 to N 4 .

	ent plucked-string instruments, including non-eurogenetic ones (Conference 1,
	Conference 2, Conference 3) ;
	2. with the development of an automatic music system with original priors spe-
	cially conceived and optimized for the marovany instrument (Article 1) ;
	3. by investigating the impact of priors on different instrument repertoires, and
	addressing the broader question of the appropriateness of current state-of-the-
	art AMT algorithms for traditional instruments ;
	Computational General Musicology,
	1. with an extensive analytical work dedicated to the marovany zither, includ-
	ing the multifacets of its repertoires and instrument makings (Conference 4,
	Conference 5);
	2. by making the marovany a benchmark instrument on test/validation experi-
	ments of MIR tools (Article 4) ;
	Statistical modeling & Audio Signal Processing,
	1. with the theoretical development of a particular filter based system for PLCA
	parameter estimation, aiming to resolve current limits of the EM algorithm
	(Article 3) ;

Table 1 .

 1 Chapter 1. Generating knowledge from Musical Acoustics 2 -Table detailing the characteristics of the different KCMA.

	Knowledge sources	KCMA name	Likelihood princple	Polyphony dimension	KCMA category
		Theo1	Psychoacoustic emphasis of chord acoustic properties	Transition	Musical codes
		Theo2 / Theo3	Minor and Major keys given key	Combination / Transition	Musical codes
	Theoretical concepts	Theo4 / Theo5	Minor and Major keys	Combination / Transition	Musical codes
		Theo6	Circle of fifths	Transition	Musical codes
		Theo7	Krumhansl tone profiles	Combination	Musical codes
		IsoNo0	Signal samples	Note object	Timbre / Playing techniques
		IsoNo1-IsoNo4	Acoustic descriptors	Note object	Timbre
	Isolated note samples	IsoNo5	Acoustic phenomena of sympathetic resonances between strings	Combination	Timbre
		IsoNo6	Non-tempered tuning, i.e. deviation in cents from equal temperament	Combination	Musical codes
		TraPla1 / TraPla2	Frequency counting	Combination / Transition	Musical codes
	Transcripts from playing	TraPla3	Pitch-wise frame-to-frame transition	Transition	Playing style
		TraPla4	Pitch-wise note duration	Transition	Playing style
		TraPla5	Melodic motive dictionnary	Transition	Musical codes

P

  250 ms then % AmpMod-informed smoother (y t |q t = S 1 ) P (y t |q t = S 2 )

			a 11
			S 1
		a 21	a 31
		a 12	a 13
		S 2	a 23	S 3
		a 32
		a 22		a 33
	8:	Apply Minimum/Maximum filtering (eq. 3.23) to U i,k
	9:	end if
	10:	end for
	11: end for

  .1. There are 7 such KCMA.

	Prior index KCMA Names Baseline methods KCMA incorporation methods
	H 1:3	T heo2, 4, 7	PLCA	Sparse entropy prior (eq. 3.3, Sec. 3.2.1)
	H 4:7	T heo1, 3, 5, 6	HMM	First-order N

Mn -state HMM for harmonic transitions (Sec.

3.2.5)

Table 3

 3

.1 -Table of the different KCMA from theoretical concepts (source I) implemented in our AMT system.

Table 3 .

 3 2 -Table of the different KCMA from isolated note samples (source II) implemented in our AMT system.

Table 4 .

 4 1 -Numerical values of the different signal criteria evaluated, for our three different MCSS, being respectively optical, piezoelectric and electromagnetic. Values of C c allows comparison between the different sensor signals, as well as with the microphone signal. Values of C 8 allow only relative comparisons between sensor signals. The sign X means dimensionless criteria.

	Signal criteria Measures Unity M CSS 1	M CSS 2	M CSS 3
	Salience	C 1 C 2	dB ms	-5.1 ± 2.2 -23 ± 23	-2.3 ± 2.9 -11.5 ± 11.5 -11.5 ± 23 -8.7 ± 1.7
	Spectral	C 3 C 4	X Hz	-0.23 ± 0.09 -0.11 ± 0.03 0.19 ± 0.05 23.4 ± 8.2 10.1 ± 4.5 55.9 ± 5.6
	Temporal	C 5 C 6	ms X	-320 ± 32 -0.1 ± 0.03	-168 ± 21 -0.36 ± 0.07 -0.26 ± 0.11 -563 ± 69
	Segmentation C 7	ms	19	16	6
	Separability	C 8	X	2.7	3.4	1.1

Table 5 .

 5 2 -Average F-measures obtained with our baseline systems using the two note parameters of Onset location and Pitch for evaluation, for our different evaluation datasets.

	Baseline systems	R 1	R 2	R 3	R tot
	Tolonen2000	52.2 % 57.4 % 57.1 % 55.5 %
	Emiya2010	55.9 % 56.1 % 55.7 % 56.8 %
	Fuentes2013	58.3 % 61.6 % 61.2 % 59.1 %
	Benetos2013	60.5 % 64.1 % 60.1 % 61.3 %
	BaseCaz1	57.5 % 60.3 % 62.5 % 58.6 %
	BaseCaz2	56.8 % 60.6 % 61.4 % 58.2 %

Table 5 .

 5 3 -Average F-measures obtained with our baseline systems using all note parameters for evaluation, for our different evaluation datasets.

		49.1 % 51.9 % 45.8 % 48.7 %
	Emiya2010	49.7 % 53.1 % 52.3 % 51.2 %
	Fuentes2013	49.8 % 51.2 % 54.6 % 52.9 %
	Benetos2013	51 %	54.2 % 49.5 % 50.6 %
	BaseCaz1	52.7 % 48.5 % 50.2 % 51.4 %
	BaseCaz2	48.7 % 49.5 % 50.1 % 49.9 %

Table 5 .

 5 4 -Average F-measures obtained with our baseline systems in the monophonic mode transcription, for our different evaluation datasets.

	Baseline systems	R 1	R 2	R 3	R tot
	Tolonen2000	59.4 % 62.3 % 59.1 % 59.6 %
	Emiya2010	59.9 % 62.5 % 64.2 % 61.3 %
	Fuentes2013	68.4 % 69.5 % 66.7 % 68.3 %
	Benetos2013	67.5 % 68.9 % 66.2 % 68.7 %
	BaseCaz1	67.1 % 67.9 % 65.1 % 66.4 %
	BaseCaz2	62.1 % 63.4% 64.8 % 61.2 %

Table 5 .

 5 5 -Average F-measures obtained with the systems BaseCaz1(P 1 c ) and BaseCaz2(P 2 c ) (with P 1 c and P 2 c the combination of priors with the highest G p value, for each dataset, respectively for BaseCaz1 and BaseCaz2), and the G

p (P 1 c ) and the G p (P 2 c ) values, for our different evaluation datasets.

Table 5 .

 5 7 -Average F-measures obtained with the systems BaseCaz1 + P 1 c and BaseCaz2 + P 2 c (with P 1 c and P 2 c the combination of priors with the highest G p value, for each dataset, respectively for BaseCaz1 and BaseCaz2), and the G p (P 1 c ) and the G p (P 2 c ) values, for our different evaluation datasets.

  models, there is no general analytic expression for the computations of these density functions. As a consequence, an approximation strategy is required to estimate the filtering and smoothing densities, which is commonly performed with the PF method, also known as sequential Monte Carlo methods. Within the PF framework, the filtering distribution is approximated with an empirical distribution formed from point masses also called particles, p(x t |y 1:t ) ≈

					N	w	(i) t δ(x t -x	(i) t )	(C.6)
					i=1
			N	w	(i) t = 1, w t ≥ 0 (i)	(C.7)
			i=1
	where δ(.) is the Dirac delta function and w	(i)
	Interval name	Size (semitones)	F 0 relation
	Octave	12	2:1
	Perfect fifth	7	3:2
	Perfect fourth	5	4:3
	Major third	4	5:4
	Minor third	3	6:5
	Major second	2	9:8

  p(x t |y 1:t ) = p(x t |y t , y 1:t-1 ) (C.9)

	N	δ(x 1:t-1 -x	(i) 1:t-1 )	(C.13)
	i=1

∝ p(y t |x t , y 1:t-1 )p(x t |y 1:t-1 ) (C.10)

∝ g(y t |x t )p(x t |y 1:t-1 ) (C.11) ∝ g(y t |x t )f (x t |x t-1 )p(x 1:t-1 |y 1:t-1 )dx 1:t-1 (C.12)

Assuming that a particle approximation to p(x 1:t-1 |y 1:t-1 ) has already been generated, p(x 1:t-1 |y 1:t-1 ) ≈

The term euro-genetic is used in this paper to avoid the misleading dichotomy of western and nonwestern music[START_REF] Lartillot | Commentary on "comparative analysis of music recordings from western and non-western traditions by automatic tonal feature extraction" by emilia gomez, and perfecto herrera[END_REF]. It was proposed by Prof. Robert Reigle (MIAM, Istanbul) in personal communication, and was first used in specialized AMT literature by[START_REF] Benetos | Automatic transcription of turkish makam music[END_REF]. Eurogenetic music refers mostly to "savant" classical music developed in Europe between the 17 th and 19 th centuries.

[START_REF] Blacking | The Performing Arts: Music and Dance[END_REF] proposed the following definition of ethnomusicology : "The main task of ethnomusicology is to explain music and music making with reference to the social, but in terms of the musical factors involved in performance and appreciation".

This spectral resolution higher than the twelve-tone equal temperament commonly used in chord modeling allows us taking into account tuning deviations and inharmonicity.

This distinctiveness between elements is done w.r.t the Υ t k vectors.

Similar to physical pianoroll, where notes are represented in terms of their physical duration and onset location, but using discrete MIDI-scaled pitches and amplitudes.

See http://lightwave-systems.com/

Acronym meaning the in-situ conversion of an acoustic instrument into its homologous MIDI.

An oracle system is defined as a transcription system using note samples from the same instrument source, which is meant to demonstrate the upper performance limit of the transcription system.

Remerciements

model, which are equally divided into two categories. A first category in which note samples are obtained by varying playing dynamic only, with a fixed position of the excitation point, centred in the playing area. A second category in which only excitation modes are varied, i.e. distance from the easel and plucking mode (e.g. fingertip or nail), with an almost-constant playing dynamic. All the isolated note samples have been recorded using a semi-tone scale, so as to guarantee generality regardless specific repertoires presenting micro-deviations in tunings.

Current state of database (ongoing database)

Details of the databases are given in table 4.2. All in all, this database currently holds X solo performances, by X different musicians. The overall size of the database is about X GB, i.e. about X hours of audio recordings. This sound database contains mainly recordings of the maraovany, as it was the research instrument of this PhD. For this instrument, it supplies a good generalization of the different sound possibilities of each instrument in various playing techniques and musical repertoires. It thus provides a good generalization of the sounds each instrument is producing in different recordings. We wish to make this sound database available under a Creative Commons license in a near feature.

Biographies of the marovany players

In the following, we provide short biographies of the three marovany players who participated to this PhD project.

Velonjoro Born in Ambovombe, Velonjoro has never left Madagascar, and has lived in Tuléar in isolation from western culture. One of his occupations is marovany player in trance rituals of tromba, making him an emblematic player of a vernacular and traditional repertoire of the marovany. Kilema Born in Tuléar, Cément Randrianantoandro KILEMA is a malagasy profesionnal musician of marovany, but also of Kabosy and Katsa (native instruments), as well as a lead vocal singer. With the "Justin Vali Trio", a malagasy band, he performed in Japan, Australia, New Zealand, and Woodstock 1994 among others. In 1999, Kilema presented his first solo album Ka Malisa, an introspective into the music of southern Madagascar, his own birthplace. During the past three years, Kilema has been touring the main Ethnic and world Music Festivals in Spain. Through the recordings of four CDs, Kilema's continued investigation and diffusion of traditionnal Malagasy music.

Publication list

This thesis consists of several publications, listed in the following, along with their section locations of the thesis in which they appear, or partially appear.

Peer-reviewed journal article

EffDur , the physical duration of the signal (in s) will be defined as the time during which the signal energy remains between 5 % and 95 % of its maximal energy

DT , the decreasing time (in s) is defined by the necessary time for the signal to reach 5 % of its maximal energy E max from this value

TGC , the Temporal Gravity Center (in s) is the average value of the time energy distribution, which divides a temporal profile in two parts of equal energy :

with Env(k) the temporal envelop of the signal ;

AmpMod, the Amplitude Modulation quantifies energetic variations within a temporal profile :

A value close to one traduces a stationary behavior of the signal, whereas strong values traduce transitory variations ;

FM, the Frequency Modulation quantifies energetic variations within a temporal profile :

with F 0 the temporal frequency curve. A value close to one traduces a stationary behavior of the signal, whereas strong values traduce transitory variations ;

Enveloppe , the temporal envelope e(tn) of the audio signal s(tn) is derived from the amplitude of the analytic signal sa(tn) given by the Hilbert transform of s(tn). This amplitude signal is then low-pass filtered using a third-order Butterworth filter with a cutoff frequency of 5 Hz. e(tn) has the same sampling rate and duration as that of s(tn).

A.2 Time-varying descriptors

It should be noted that, in our system, all window durations and hop sizes are defined in seconds and then converted to samples according to the sampling rate of the input audio signal. This guarantees that the same spectral resolution will be obtained whatever the sampling rate of the signal. However, the content of the representation itself will differ according to the sampling rate. This is because the upper frequency of the STFT depends on the sampling rate (it is equal to f max .s r /2).

THD, the Total Harmonic Distorsion measures the energetic weigth of harmonics

relatively to the fundamental one, defined by :

with N the number of harmonics with non-negligible amplitudes ;

Df, the formant dispersion (in Hz), which is the average distance between each adjacent pair of formants, was calculated using the following formula

where N is the total number of formants measured, and F i is the frequency (in Hz) of the formant i ;

ZCR, the Zero Crossing Rate is a simple indicator of noisiness consisting in counting the number of times the signal crosses the X-axis (or, in other words, changes sign).

Delta F 0, this descriptor computes the difference between successive frames of the F 0 curve, i.e.

The sign of this descriptor gives the sense (upwards / backwards) of the sound modulation.

A.3 Musical descriptors and transforms

A.3.1 Key detection

Prior knowledge of the key results from an automatic detection based on a chromabased frequency analysis [START_REF] Shenoy | Key determination of acoustic musical signals[END_REF], performed on each training and test sequence.

A.3.2 Constant-Q transform

Appendix B

Musical sound representation

For humans the perceptual distance between 220 and 440 Hz is the same as between 440 and 880 Hz. A pitch representation that takes this logarithmic relation into account is more practical for some purposes. Luckily there are a few:

MIDI note number

The midi standard defines note numbers from 0 to 127, inclusive. Normally only integers are used but any frequency f in Hz can be represented with a fractional note number n using this equation Rewriting Equation (A1) to (A2) shows that midi note number 0 corresponds with a reference frequency of 8.176 Hz which is C -1 on a keyboard with A4 tuned to 440 Hz. It also shows that the midi standard divides the octave into 12 equal parts. To convert a midi note number n to a frequency f in Hz one of the following equations can be used.

Using pitch represented as fractional midi note numbers makes sense when working with midi instruments and midi data. Although the midi note numbering scheme seems oriented towards Western pitch organization (12 semitones) it is conceptually equal to the cent unit which is more widely used in ethnomusicology.

Cent Von Helmholtz and [START_REF] Von Helmholtz | On the sensations of tone as a physiological basis for the theory of music[END_REF] introduced the nowadays widely accepted cent unit. To convert a frequency f in Hz to a cent value c relative to a reference frequency r also in Hz:

Filtering Particle

In the framework of Bayesian variable selection, Markov Chain Monte Carlo, or Particle filtering (PF), type approaches have been proposed Févotte and Godsill (2006a); [START_REF] Févotte | Sparse linear regression with structured priors and application to denoising of musical audio[END_REF]. These methods consist in scanning the whole posterior distribution, making them more demanding than their EM-like counterparts, but which also, in return, offer increased robustness in convergence (i.e. reduced problems of convergence to local minima) and a complete Monte Carlo description of this parameter posterior density Févotte and Godsill (2006b,a); [START_REF] Févotte | Sparse linear regression with structured priors and application to denoising of musical audio[END_REF].

C.1 General Overview

Many problems in statistical signal processing [START_REF] Fong | Monte carlo smoothing with application to audio signal enhancement[END_REF]; [START_REF] Andrieu | Efficient particle filtering for jump markov systems. application to time-varying autoregressions[END_REF]; [START_REF] Vermaak | Particle filtering for non-stationary speech modelling and enhancement[END_REF] can be stated in a state space form as follows,

where {x t } are unobserved states of the system and {y t } are observations made over some time, t. f (.|.) and g(.|.) are pre-specified state evolution and observation densities. A primary concern in many state-space inference problems is the sequential estimation of the filtering distribution p(x t |y 1:t ), and the simulation of the entire smoothing distribution p(x 1:t |y 1:t ), where y 1:t = (y 1 , y 2 , • • • , y t ) and x 1:t = (x 1 , x 2 , • • • , x t ). Updating of the filtering distribution can be achieved, in principle, using the standard filtering recursions [START_REF] Robert | Monte Carlo Statistical Methods[END_REF] In practice, these filtering (eq. C.3) and smoothing (eq. C.5) computations can only be performed in closed form for linear Gaussian models using the Kalman filter / smoother, and for finite state-space hidden Markov models. In the case of non-linear non-Gaussian

List of Tables