Fahima Djelil
email: fahima.djelil@udamail.fr

Benjamin Albouy-Kissi
email: benjamin.albouy@udamail.fr

Adélaïde Albouy-Kissi

Eric Sanchez
email: eric.sanchez@ens-lyon.fr

Jean-Marc Lavest
email: j-marc.lavest@udamail.fr

Towards a 3D Virtual Game for Learning Object-Oriented Programming Fundamentals and C++ Language Theoretical Considerations and Empirical Results

Keywords:

Object-Oriented Programming (OOP) paradigm is one of the most common paradigm in introductory programming courses. However, novices often have difficulties to understand the basic concepts which are of a high level of abstraction. Either tangible and virtual constructive games provide the students with a more familiar way for learning programming. This paper applies a construction game metaphor approach for learning OOP concepts and C++ syntax. After introducing some tangible and virtual constructive games for learning programming, we present an experimental prototype of a new 3D virtual game for learning OOP called PrOgO as well as the results of an experiment conducted with beginner student using PrOgO.

INTRODUCTION

Learning programming can be perceived difficult from novices [START_REF] Yan | Teaching object-oriented programming with games[END_REF][START_REF] Overmars | Learning object-oriented design by creating games[END_REF]. A novice student has to face a primary challenge which is problem solving, he has to understand how a computer program runs and has to learn a specific syntax in which his program will be transcribed. Abstract concepts understanding is another challenge. The basic Object-Oriented Programming (OOP) concepts which are on a higher level of abstraction have become the most common programming concepts for introductory programming courses, and the transition to the OOP paradigm has proven to be more difficult than expected [START_REF] Börstler | Transitioning to OOP/java -a never ending story[END_REF]. Therefore, a lot of systems have been designed to support the acquisition and development of programming concepts. They aim to take abstract programming concepts to the real world for learners. They are based either on tangible interfaces or on virtual simulations and games that connect with the learners interest. This has proven to be very engaging and empowering for novices [START_REF] Cooper | Alice: a 3D tool for introductory programming concepts[END_REF][START_REF] Kölling | The greenfoot programming environment[END_REF].

In this work, we apply a construction game metaphor approach for learning OOP. Our ultimate goal is to provide novice students with realistic and relevant representations that will help them in learn-ing OOP concepts and C++ programming syntax in an entertaining way. For this purpose, we present an experimental prototype of a new three-dimensional (3D) virtual game for basic OOP concepts such as class, object, object creation, method invocation, parameter passing and object destruction. First we present a literature review on tangible interfaces and virtual environments for learning programming (section 2), then we describe our first prototype of the game we called PrOgO (section 3) and finally, we describe an experiment we conducted with beginner students using this prototype, we will proceed to interpret the results (section 4) and to extract the corresponding conclusions (section 5).

THEORETICAL BACKGROUND

There have been several tangible interfaces for learning programming which aim to bring the abstract concepts of programming closer to the real world, to be more easily graspable [START_REF] Jetsu | Tangible user interfaces and programming[END_REF]. These interfaces consist of tangible components i.e. physical and mechanical that the user can assemble to construct program's sequences. One of the first tangible programming interfaces was Radia Perlman's Slot Machine [START_REF] Perlman | Using computer technology to provide a creative learning environment for preschool children[END_REF] which was based on the LOGO pro-gramming language. LOGO was one of the most widely used programming languages for beginners [START_REF] Jetsu | Tangible user interfaces and programming[END_REF], allowing students to use the computer to control a robotic turtle to follow their instructions [START_REF] Papert | Mindstorms: Children, computers, and powerful ideas[END_REF]. Slot Machine allowed students to control either a real mechanical robot or a screen based simulation of such a robot [START_REF] Perlman | Using computer technology to provide a creative learning environment for preschool children[END_REF]. Algo-Block is another interface that is somewhat similar to Slot Machine. It consists of a set of physical blocks that connect to one another to form a program [START_REF] Suzuki | Algoblock: a tangible programming language, a tool for collaborative learning[END_REF]. Each block corresponds to a single command in the LOGO programming language. The term tangible programming language was invented by Suziki and Kato in order to describe the AlgoBlock collaborative programming environment [START_REF] Suzuki | Algoblock: a tangible programming language, a tool for collaborative learning[END_REF]). Topobo is another example of this kind of interfaces that was designed to learn programming thinking [START_REF] Raffle | Topobo: a 3-D constructive assembly system with kinetic memory[END_REF], it consists of a set of active and passive blocks which can be combined together to assemble and animate robotic structures. The active blocks are embedded with a kinetic memory which allows recording and playback of physical motion of the constructed assembly. This constructed assembly is animated by moving passive blocks that are connected to the active ones. After the animation has been created, it can be played back by pressing a button [START_REF] Raffle | Topobo: a 3-D constructive assembly system with kinetic memory[END_REF].

Several environments have been designed with a view to reduce the complexity of programming learning for novice students and increase their motivation. These environments use graphic representations to introduce abstract programming concepts and allow students to create games and animations by manipulating programs. Alice [START_REF] Conway | Alice: lessons learned from building a 3D system for novices[END_REF] and Greenfoot [START_REF] Kölling | The greenfoot programming environment[END_REF] are examples of such environments, which strive to engage students by allowing them to write programs about games, stories and simulations. One of the important characteristics of Alice and Greenfoot is a design that explicitly visualizes fundamental concepts of OOP in a realistic and a meaningful context. Students do not typically start by manipulating source code, they rather start by creating objects by selecting graphics which represent classes. Once an object has been created, it can be placed into a 2D (Greenfoot) or a 3D (Alice) scene, when the object is selected the user can see all its methods which constitute all its available actions it can perform. Once a method has been selected, the user can see immediately the effect of its execution onscreen. After the students are introduced to the fundamentals of OOP, scenarios that have been already implemented are presented to them enabling easy manipulation of programs in their syntax. This characteristic is very interesting for novice students, since it has been proved [START_REF] Cooper | Teaching objects-first in introductory computer science[END_REF][START_REF] Kölling | The greenfoot programming environment[END_REF] that this helps novices to avoid syntax errors while focusing on understanding abstract concepts before manipulating source code. According to [START_REF] Utting | Alice, greenfoot, and scratcha discussion[END_REF], these environments' principal harks back to Logo's turtle. Since they reify objects so that the result of command execution is visible as the position, size, rotation, and other visible state and behavior of the object changes. These environments provide highlevel commands like move and hide low-level details such as graphics primitives.

By analogy with any construction game such as Topobo, a screen based simulation of such a game that allows students to manipulate 3D graphics to create and animate robotic structures, whereas each graphical elementary block represents an object, would be relevant to introduce OOP concepts. Indeed, an object is an active entity within a program, including a set of attributes (knowledge) and methods (skills) that act on its attributes to achieve some functionalities which are the objectives of the program.

A 3D VIRTUAL GAME PROTOTYPE FOR LEARNING OOP FUNDAMENTALS AND C++ PROGRAMMING LANGUAGE

A constructive game dedicated to learning OOP fundamentals, should enable students creating objects, visualizing their state and behavior, interacting with them by executing their methods and make them communicate with one another. Direct manipulation of graphics representing object-oriented concepts can help students, as in real life, to think in terms of objects. The student should be able to create as many class instances as he likes. The concept of inheritance can be materialized by creating new objects based on other objects having the same behavior with new attributes or the same attributes with new behaviors. The concept of composition can be represented by objects that contain other objects. In order to enable the student to understand more easily the implementation of the abstract concepts in a specific syntax such as C++, all his manipulations executed directly on the graphics should be interpreted into source code. Since the representations are very precise, each student's action can be interpreted into a C++ instruction. This will constitute a complete program, that the student can visualize and modify in order to practice programming, after the abstract concepts are well understood. This design principle provides students with a primary introductory step to OOP concepts, through fun and realistic 3D graphics, before manipulating source code. This also allows students to express their own creativity, and provides them with an engaging learning tool.

PrOgO (Object-Oriented PrOgramming) is an experimental prototype which has been developed as a proof of concept of such a game. This prototype generates C++ source code corresponding to direct manipulations of 3D graphics. Figure 1 shows the interface of PrOgO with an example of a student's realization. The interface comprises 3D elementary blocks representing classes that the student can instantiate to obtain objects, a 3D scene to place in the objects that have been created, and a separate tab anchored on the right side of the 3D scene to display the C++ code that is being generated during the direct manipulations of the 3D graphics. Once an object is created and placed into the 3D scene, the student can view all its methods, and invoke them. Once a method is executed, its result on the state of the object is immediately visualized, and the corresponding C++ code is automatically generated. Each object has three elements of state that are automatically visualized on screen: a position which is specified in x, y, z coordinates, a rotation and a color. Separate objects can connect to one another and assembled objects can be disassembled. Each of these can be manipulated trough method calls to create a robotic or a mechanic structure. Note that the code generated is not yet editable.

EXPERIMENTAL RESULTS

In the Institute of Technology of le Puy-en-Velay (University of Auvergne, France), the course Fundamentals of Computer Science is imparted in the first year of the Degree in Digital Imaging. This course consists of two main topics: Introduction to algorithms and programming, and OOP fundamentals. The first topic covers the basics about programming on C++: data types, variables and operators, control flow statements, arrays and pointers, procedures and functions, dynamic allocation. The students, at this stage, are initiated to C++ procedural programming and don't have knowledge about OOP. The second topic is an introduction to OOP in C++: classes and objects, encapsulation, composition, inheritance, polymorphism, etc.

We have experimented PrOgO with the first year students in Digital Imaging Degree. This experimentation aims to identify potential design, ergonomic and pedagogical issues of the game, and intends to provide a conceptual basis for a future evolutive prototype on the basis of the students' expectations in terms of learning and entertaining aspects. This was conducted after the students have completed the first topic of the course, and once they started the second one. They have been introduced for the first time to some elementary concepts such as class, object, encapsulation and method invocation. The participants were a total of 48. Their average age was 18.7 years and all except one reported to play to video games very frequently (31 participants play every day and 16 of them play several times a week). They also mostly reported not to play to serious games (37 participants don't play against 11 who play regularly). We collected pictures of the graphical realizations (Figure 2) as well as the corresponding generated source code files, to determine whether the students were getting involved into the game. We also recorded all the students' actions within logs including objects creation and destruction and method calls within the 3D scene, as well as code selection within the tabs containing the code which is automatically generated and the code which is already implemented that includes the declaration of the available classes. This have been made in order to know whether the students make the effort to engage in understanding the relationship between the 3D graphics and the generated code, rather than only manipulating the graphics in the 3D scene. We also used an attitude survey which is focused on participant's experience and opinions about using PrOgO in the future for learning other OOP concepts. They were asked to rate the pedagogical and entertaining aspects of the game, to indicate whether they have improved their understanding of the concepts of class, object, method invocation and C++ syntax related to these concepts, and what aspects of the game they would like to be improved.

From the collected graphical realizations (Figure 2), we can observe that some students clearly succeeded to construct complex and meaningful structures. This indicates that the students were quiet engaged in the game and motivated to accomplish the game's objective. From the generated interaction logs, we observed that the students were mostly focused on manipulating graphics in the 3D scene. The logs have shown that the actions that have been mostly performed are objects instantiating and objects methods calls such as assemble(), translate() and rotate(). Some students were also interested to analyse the generated code, since the logs included actions such as "select code" and "select tab".

From the attitude survey, we collected the students' opinions expressed upon four levels (not at all, sparsely, quite, a lot) about the amusing and pedagogical aspects of PrOgO, as well as improvement of their understanding of the presented concepts. Table 1 summarizes the obtained results. We have linked the amusing judgement to the student's frequency of play to video games and the pedagogical judgement to the student's frequency of play to serious games and we compared the obtained results. We concluded that the students have mostly found PrOgO sparsely or quite amusing and quite pedagogical, regardless of the fact that they play every day or several times a week to video games, and the fact that they play or not to serious games, since the obtained results are not very different from the separate students categories, as it is reported in table 1: 51% of the students who play to video games everyday and 39% of those who play several times a week stated that they have found PrOgO "sparsely" amusing. 39% of the students who play everyday and 44% of those who play several times a week found it "quite" amusing. They are totally 47% to find it "sparsely" amusing and 41% to find it "quite" amusing. Moreover, no one of the students who play to serious games has found it "not at all" pedagogical, 58% rather found it "quite" pedagogical and 34% found it "sparsely" pedagogical, those who don't play to serious games have mostly found it "quite" pedagogical (67% against 22% who found it "sparsely" pedagogical).

We have also concluded that the students have mostly sparsely or quite improved their understanding of the presented concepts (class, object and related C++ syntax), regardless of the fact that they previously understood these concepts in classroom or not. As it is shown in table 1, the students who said that they have already understood these concepts in classroom have though improved their understanding ( 44% stated that they have "sparsely" improved their understanding of the concept of class, 37% have stated the same regarding to the concept of object, and 33% regarding to the syntax of these concepts). There are also a number of them who stated "quite" improving their understanding ( 36% regarding to the concept of class, 26% regarding to the concept of object, and 28% regarding to the syntax of these concepts).

Most of the students who thought not to understand before in classroom have also presented quite similar percentages (59% stated improving "sparsely" their understanding of the concept of class, the percentages for the concepts of object and syntax are respectively 54% and 62%). A lot of them have reported having "quite" improved their understanding of these concepts, the percentages are respectively (33% , 43% and 32%).

We gathered some students' statements regarding to what they learned, during the experiment such as: "I have learned how to declare an object and how to make method calls", "I understand more about the C++ syntax, and this tool can help me to review some concepts at home for example", "I understand better the syntax related to object instantiation and method calls, especially about pointers", "I understand better the relationship between the class and the object".

We have also collected the students opinions about using PrOgO in the future for learning additional concepts, and linked the results to their judgements regarding to weather they found it amusing, pedagogical and attractive. We concluded that the students who want to use PrOgO again are mostly those who found it quite amusing, quite pedagogical and quite attractive, and those who don't want to use it again are those who found it sparsely amusing, sparsely pedagogical and sparsely attractive (Figure 3). Furthermore, we asked the students what aspects of the game they would like to be improved. The answers were mostly about ergonomic and learning aspects. Most of the students indicated that they would like more graphical elements, and more options such as having the possibility to construct other objects from the available ones, in order to be able to manipulate groups of objects. Most of the students expressed an interest in modifying the generated code, to interact more easily with the virtual environment and to have more freedom in their actions.

CONCLUSIONS

In this work, we have applied a construction game metaphor for learning OOP and C++ programming From the analysis of results presented in the previous section, we concluded that although PrOgO is at the prototype stage, the students have mostly found it quite pedagogical and amusing. They also mostly sparsely improved their understanding of the concepts of class, object, object instantiation, method calls and C++ syntax related to these concepts. Moreover, more than half of them stated that they want to use PrOgO in the future for learning additional concepts, and most of them have expressed the wish to have more options regarding to the interaction interface and the content of the game, such as having the possibility to modify the generated code, which is something we have already planned to do before. Although this work is a preliminary study, such results are significant. In this sense, a stable release of the game could have a positive influence on the learning of OOP basics as well as C++ programming language.

Actually, we work on the design and the development of an evolutive prototype dedicated to tactile interfaces such as tangible tabletops and tablets. This work allowed us to validate our design principal, especially the construction metaphor on which the game is based on, and the students feedback allowed us to confirm the extensions we have already planned for a future evolutive prototype. A future work will also include the study of the impact of PrOgO on the students motivation and learning.

Figure 1 :

 1 Figure 1: PrOgO main window showing a realization example.

Figure 2 :

 2 Figure 2: Some students' realizations.

  Using PrOgO in the future according to weather the students found it amusing, pedagogical and attractive: a) students who want to use PrOgO again. b) students who don't want to use PrOgO any more language. We have presented a first experimental prototype of a 3D virtual game called PrOgO by which abstract OOP concepts are represented in a meaningful and a visible way, and we experimented it with the first year students in Digital Imaging Degree.

Table 1 :

 1 Participants' opinions about PrOgO collected in the attitude survey.

	comparison item	participants	not at all sparsely quite a lot
		students who play to video games	10%	51%	39%	0%
	how is PrOgO amusing?	everyday
		students who play to video games	17%	39%	44%	0%
		several times a week
		all	12%	47%	41%	0%
		students who play to serious games	0%	34%	58%	8%
	how is PrOgO pedagogical?	students who don't play to serious	3%	22%	67%	8%
		games
		all	2%	25%	65%	8%
	how did you improve your	students who think they have al-	16%	44%	36%	4%
	understanding of the	ready understood in classroom
	concept of class?	students who think they have not	8%	59%	33%	0%
		understood in classroom
		all	12 %	51%	35%	2%
	how did you improve your	students who think they have al-	26%	37%	26% 11%
	understanding of the	ready understood in classroom
	concept of object?	students who think they have not	3%	54%	43%	0%
		understood in classroom
		all	12 %	47%	37%	4%
	how did you improve your	students who think they have al-	39%	33%	28%	0%
	understanding of the syntax	ready understood in classroom
	related to the concepts of	students who think they have not	3%	62%	32%	3%
	class and object?	understood in classroom
		all	16 %	51%	31%	2%


Semantic bleaching of 'nu' in Old Saxon

Elise Louviot

To cite this version:

Elise Louviot. Semantic bleaching of 'nu' in Old Saxon. Folia Linguistica, 2018, 39 (2), pp.441-473. 10.1515/flih-2018-0016. hal-02452574

1 1


Adrien Meynard

Spectral Estimation for Multivariate Locally Time-Warped Signals

Keywords: Nonstationarity, spectral analysis, blind source separation, time warping, wavelet transform

Spectral estimation generally aims at determining from a single realization of a given signal, the distribution of its power as a function of frequency. In this paper, we focus on multivariate, locally time-warped signals. We show that the spectral estimation problem can also be interpreted as a doubly nonstationary blind source separation (BSS) problem, where both the mixing matrix and the original sources contribute to nonstationarity. We then introduce a BSS algorithm for joint spectral estimation of nonstationary sources. The performance of the proposed approach is evaluated on numerical simulations, and compared with other nonstationary BSS algorithms.

I. INTRODUCTION

I N time series analysis or statistical signal processing, spectral estimation is generally understood as the problem of estimating the frequency content of a signal. Many spectral estimation techniques have been proposed in this context, that can handle estimation from a single signal realization [START_REF] Stoica | Spectral analysis of signals[END_REF]. A familiar example is the so-called Welch estimator. In a more general nonstationary context, spectral estimation can also be interpreted as the problem of estimating and diagonalizing the covariance operator of the signal. Both notions coincide in the stationary case, where the covariance operator is a convolution operator, which is diagonalized by a suitable version of the Fourier transform. To depart from the stationary situation, several classes of locally stationary random processes have been proposed and studied in the literature [START_REF] Silverman | Locally stationary random processes[END_REF], [START_REF] Priestley | Spectral analysis and time series, ser. Probability and mathematical statistics[END_REF], [START_REF] Mallat | Adaptive covariance estimation of locally stationary processes[END_REF], [START_REF] Dahlhaus | Locally stationary processes[END_REF], each of them being characterized by a specific form of covariance operator. In this paper, we focus on a specific, physically relevant model of nonstationarity: the locally deformed signals. In a few words, those signals are stationary signals transformed by some stationarity-breaking operator. In particular, we focus on time warping, which is a specific type of deformation introduced in [START_REF] Clerc | Estimating deformations of stationary processes[END_REF]. Such a model, combined with amplitude modulation, can account for physical phenomena as diverse as Doppler effect, speed variations of an engine, animal vocalization [START_REF] Stowell | Computational Analysis of Sound Scenes and Events[END_REF], or speech. The covariance operator of locally deformed signals only depends on the spectrum of the underlying stationary signal and the deformation operator. In this case, spectral analysis can be regarded as the joint estimation of these two quantities. We introduced in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] the JEFAS algorithm (Joint Estimation of Frequency Amplitude and Spectrum), which performs this estimation.

In this paper, we extend the results of [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF] to the multivariate case where the amplitude modulation is matrix-valued and Adrien Meynard is with Aix Marseille Univ, CNRS, Centrale Marseille, I2M, Marseille, France mixes decorrelated locally time-warped signals. We formulate this problem as a joint blind source separation (BSS) and nonstationary spectral analysis problem, which we call doubly nonstationary BSS: sources and mixing matrix are nonstationary.

Most BSS algorithms assume stationarity of the sources, and often also address instantaneous linear mixtures, where the mixing matrix is constant over time (see [START_REF] Comon | Handbook of Blind Source Separation, Independent Component Analysis and Applications[END_REF] for a thorough review of the field). For such problems, SOBI [START_REF] Belouchrani | A blind source separation technique using second-order statistics[END_REF] is a reference approach that exploits second-order statistics and performs approximate joint diagonalization of covariance matrices for the estimation. Several adaptations to the BSS of nonstationary signals mixed by a constant matrix have been proposed. Many of them are based on quadratic timefrequency analysis [START_REF] Belouchrani | Source separation and localization using time-frequency distributions: An overview[END_REF], [START_REF] Thirion-Moreau | Chapter 11 -quadratic timefrequency domain methods[END_REF] and rely on a selection of specific points of the time-frequency domain where a single source is active. In this paper we will use a specific one of these termed QTF-BSS, for comparison. BSS algorithms relying on local approximations of the observations by piecewise stationary signals have also been studied [START_REF] Pham | Blind separation of instantaneous mixtures of nonstationary sources[END_REF]. These methods include the piecewise SOBI estimation on non-overlapping subintervals, called p-SOBI in the article. Models of non-instantaneous mixtures have also been considered. In [START_REF] Kaftory | Blind separation of time/position varying mixtures[END_REF], the authors review the general case of time-varying convolutive mixing (including instantaneous time-varying mixing). The proposed method assumes the existence of a sparse representation of the sources, which is not relevant for the situation considered here.

We tackle here the doubly nonstationary BSS problem, in the framework of Gaussian models. Based on prior works on locally time-warped signals [START_REF] Omer | Time-frequency and time-scale analysis of deformed stationary processes, with application to non-stationary sound modeling[END_REF], [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF], we use an approximation of the wavelet transforms of the observations, and formulate an approximate maximum likelihood approach for the joint estimation of the mixing matrix and the time warping functions. The corresponding BSS algorithm consists in the alternate estimation of these two quantities, the latter being estimated via the above mentioned JEFAS method. The resulting algorithm is termed JEFAS-BSS. Its performance is evaluated on a synthetic example.

II. MODEL OF NONSTATIONARITY

A. A class of nonstationary signals

In the following, we consider nonstationary signals generated by deformation of stationary signals. Various classes of deformation operators have been studied before: time warping, frequency modulation [START_REF] Omer | Time-frequency and time-scale analysis of deformed stationary processes, with application to non-stationary sound modeling[END_REF], or amplitude modulation [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]. As mentioned above, we focus here on time warping combined with amplitude modulation, which allows addressing physically relevant situations. More precisely, let x denote a stationary signal, modeled as a realization of a stationary random process X with power spectrum S X . Acting on x with a time warping operator D γ yields a signal y given by

y(t) = D γ x(t) ∆ = γ (t) x(γ(t)) , (1)
where γ ∈ C 2 is a strictly increasing smooth function. If γ is not an affine function, y is nonstationary. Time warping is combined with an amplitude modulation operator A α , the observed signal is then of the form

z(t) = A α y(t) ∆ = α(t)y(t) where α ∈ C 1 . (2)
In [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF], we showed that the wavelet transform is a natural tool to analyze such signals. Given a wavelet function ψ, let W x denote the wavelet transform of a signal x, defined as

W x (s, τ ) = R x(t)q -s 2 ψ t -τ q s dt with q > 1 . ( 3
)
When the deformation functions α and γ are slowly varying compared to variations of x, the respective wavelet transforms W z and W x of z and x are approximately related by:

W z (s, τ ) ≈ α(τ )W x (s + log q (γ (τ )), γ(τ )) . (4)
This mapping W x -→ W z can be interpreted in terms of local time-scale shifts of the wavelet coefficients: the time warping function γ yields a translation along the scale axis, while the amplitude modulation function α modifies the magnitude. In the setting of random processes, we showed (Theorem 1 in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]) that the approximation error in (4) is a zero-mean complex random field, whose variance is controlled by the decay properties of the wavelet ψ, and the slowness of variations of γ and α.

B. Nonstationary instantaneous mixture

From now on, we assume that N z observed signals z i (t) are available, each of which is an instantaneous linear mixture of N y nonstationary signals, called sources, modeled as in equation [START_REF] Stoica | Spectral analysis of signals[END_REF]. The number of mixtures is assumed to be at least equal to the number of sources (i.e. N z ≥ N y ). The under-determined case (i.e. N z < N y ) is out of the scope of the present work. In addition, these sources are assumed to be uncorrelated.

More formally, let y(t) ∈ R Ny , z(t) ∈ R Nz denote the column vectors containing respectively all the nonstationary sources and observations at time t. Then, the linear instantaneous mixture is written as follows:

z(t) = A(t)y(t) , (5)
where A(t) ∈ R Nz×Ny denotes the mixing matrix at time t. This model extends the amplitude modulation model [START_REF] Silverman | Locally stationary random processes[END_REF], which corresponds to the special case where all matrices A(t) are diagonal. The aim of the spectral analysis of the multivariate locally deformed signal z is to determine jointly: -the mixing matrices A(t); -the time warping functions γ i ; -the spectra of the stationary sources S Xi .

In other words, we aim at performing a doubly nonstationary blind source separation. Indeed, this problem consists of the estimation of the nonstationary mixing matrix A(t) and the spectral analysis of the nonstationary sources y. Let us consider a fixed time τ . For each observation z i , we denote by w zi,τ = W zi (s, τ ) the row vector containing the values of the wavelet transform for a vector of scales s (of size denoted by M s ). These vectors are gathered into the

N z × M s matrix w z,τ = w T z1,τ • • • w T z N ,τ
T . We use the same notation for the wavelet transform of the sources w y,τ . The matrix A(t) is assumed to vary slowly with respect to the oscillations of the signals. Then the wavelet transforms of y and z approximately satisfy a linear relationship of the type (5), i.e.

w z,τ ≈ A(τ )w y,τ . (6)
The following theorem provides a quantitative bound for the approximation error in equation ( 6).

Theorem 1. With the above notations, let τ ∈ C Nz×Ms be the approximation error in the wavelet domain, defined as:

τ = w z,τ -A(τ )w y,τ . (7)
Let the underlying stationary sources X i (i = 1, . . . , N y ) be second-order zero-mean stationary random processes of power σ 2 X . Then the approximation error τ is a second-order zeromean, complex circular random matrix. Besides, the variance of the error matrix coefficients is bounded as follows:

E | τ | 2 ≤ σ 2 X k 2 ψ A •2 ∞ γ ∞ (q 3s ) T , (8)
where A •2 is the entrywise square of the matrix A, and

k ψ = R |tψ(t)|dt, A ∞ ∈ R Nz×Ny +
, and γ ∞ ∈ R Ny + are such that:

(A ∞ ) ij = sup t |A ij (t)| , (γ ∞ ) i = sup t |γ i (t)| .
A proof of Theorem 1 is provided in the Supplementary material. A Taylor expansion of A centered on τ enables the construction of the bound [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]. Aside from the terms controlling the error bound for (4), the error bound for ( 6) is also controlled by the variations of the mixing matrix coefficients.

III. ESTIMATION PROCEDURE

The estimation procedure relies on the approximation equations ( 4) and ( 6), which we assume to be valid, i.e. A and γ are supposed to be slowly varying enough. Assuming a Gaussian signal, the wavelet coefficients follow a complex Gaussian law, from which a likelihood can be evaluated. Time-varying parameters are therefore estimated on a discrete-time grid D by maximizing the likelihood. In this section, we describe the estimation procedure for a given τ ∈ D. For simplicity, we set θ i,τ = log q (γ i (τ )) and

θ τ = (θ 1,τ • • • θ Ny,τ ) T .

A. Dimension reduction

Our approach requires the mixing matrix to be invertible (see section III-B). The first step is then a dimension reduction step so that A is a square matrix. We first perform the singular value decomposition of consecutive segments of z and set N y to the number of significant singular values. We then select N y observations out of the N z available (this selection is further discussed in section III-C). Let z l denote the retained observations, A l,τ the restriction of A(τ ) to the corresponding rows, and B l,τ = A -1 l,τ the corresponding separation matrix.

B. Probabilistic setting

The underlying stationary signals X i are henceforth assumed to be Gaussian. It follows that the wavelet transform of the i-th source Y i is a complex circular Gaussian random matrix (see Proposition 3 in [START_REF] Omer | Time-frequency and time-scale analysis of deformed stationary processes, with application to non-stationary sound modeling[END_REF]). Then, w yi,τ ∼ CN (0, Σ i (θ i,τ )), where

[Σ i (θ i,τ )] kk = q s k +s k 2 R S Xi (q -θi,τ ξ) ψ (q s k ξ) ψ (q s k ξ) dξ.
Therefore, the sources independence hypothesis yields the following negative log-likelihood of w z,τ :

τ (B l,τ , θ τ ) = -M s log |det(B τ )|+ 1 2 N i=1 log |det Σ i (θ i,τ ))| + 1 2 N i=1 [B l,τ w z,τ ] i• Σ i (θ i,τ ) -1 [B l,τ w z,τ ] H i• ,
where [M] i• denotes the i-th row of the matrix M, and M H is its conjugate transpose. Maximum likelihood (ML) estimates, i.e. minimizers of τ (B l,τ , θ τ ), can be evaluated numerically. These estimates are respectively denoted by Bl,τ and θτ .

C. Estimation algorithm

The algorithm performs the same estimation strategy on all possible combinations of N y observations. It consists of an alternating estimation of B l,τ , θ τ , and the underlying spectra. Algorithm 1 (named JEFAS-BSS) synthesizes all the estimation steps, which are detailed below.

a) Initialization: Piecewise SOBI estimation (named p-SOBI) provides a baseline method, which is also used to determine the initial time-varying mixing matrix.

b) Mixing matrix estimation: We numerically solve the ML problem using Newton's method. Besides, because of the assumption of slow variations of the matrix coefficients, we estimate B l,τ with a time step ∆ τ greater than the sampling period of the observations. Then, the unmixing is performed approximating B l,τ by a constant matrix on the interval

I τ = τ -∆τ 2 , τ + ∆τ 2 .
The estimated sources ỹ are obtained via ỹ(t) = Bl,τ z l (t) when t ∈ I τ . Besides, continuity of Bl,τ coefficients is ensured by initializing the Newton's method with Bl,τ-∆τ . This prevents the descent from converging to another minimum of τ caused by the BSS indeterminacies.

c) Deformations and spectra estimations: For each source, we obtain the joint estimation of {θ i,τ } τ ∈D and S Xi via the JEFAS algorithm (which we detail in [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF]) where the input wavelet transform w yi of the source y i is replaced with its estimate Bl,τ w z l ,τ i• τ ∈D . d) Stopping criterion: The Source to Interference Ratio (SIR), introduced in [START_REF] Vincent | Performance measurement in blind audio source separation[END_REF], quantifies the interference in an estimated source originating from the other sources. The weaker the interference, the larger the SIR. As we do not have access to the ground truth sources, we measure the SIR between ỹ(k-1) and ỹ(k) (instead of y), which gives an evaluation of the BSS update and is, therefore, a relevant convergence assessment. Practically, the convergence criterion consists in the comparison of the SIR averaged over sources with a given threshold, which is chosen sufficiently large. e) Algorithm estimations: Finally, each row of the global separation matrix B τ is estimated Nz-1

Ny-1 times. The final estimation Bτ is naturally obtained by averaging those different estimations. This enables us to get the final estimation ỹ of the sources. Finally, JEFAS applied to ỹ gives the final estimations of the time warping functions and spectra.

IV. RESULTS

We evaluate the performance of JEFAS-BSS on a synthetic example with N y = 3 and N z = 4. Observations are 1 second long, and sampled at F s = 44.1 kHz. The power spectra of the three underlying Gaussian sources consist of non-overlapping Hann windows. The time-varying mixing matrix coefficients are sinusoidally varying over time (with different frequencies). The wavelet transforms of the locally time-warped sources, as well as those of the observations, are displayed in Fig. 1.

JEFAS-BSS is applied to the observations. It leads to 4 different estimates based on the 4 available combinations of 3 observations. Each of these estimations converges in 5 to 10 iterations. Each iteration takes about 60 seconds of CPU time on a computer running an Intel Xeon E5-2680 v4 processor. Notice that the computing time increases linearly with the observations length and dramatically increases with N y , which limits the number of sources that we can handle. The scalograms of the estimated sources and the corresponding estimated time warping functions and spectra are displayed in Fig. 2. The estimated spectra are of good quality, even though one can notice a slight overestimation of the bandwidth and residual interference. For a complete study on the performance of JEFAS (i.e. spectra and time warping functions), we refer to [START_REF] Meynard | Spectral Analysis for Nonstationary Audio[END_REF], which in particular gives bounds on the variance of the time warping estimator and the bias of the spectrum estimator.

In the following, we focus on the BSS performance.

To evaluate the quality of the BSS obtained via JEFAS-BSS, we compare it with other BSS algorithms (SOBI, p-SOBI, QTF-BSS). It is also compared with the strategy consisting in applying JEFAS-BSS on a single combination of N y observations, called Single JEFAS-BSS. We apply those algorithms to 20 independent realizations of the above synthetic example. We evaluate the quality of the BSS algorithms thanks to the mean over 20 simulations of the SIR between the ground truth sources and their estimations. We give the results in Table I, together with the corresponding standard deviations (SD). Nonetheless, this quantity is a global indicator of BSS quality. To follow the temporal evolution of the BSS quality, we introduce the so-called normalized Amari index, initially constructed in [START_REF] Moreau | A one stage self-adaptive algorithm for source separation[END_REF] and called here ρ(t). The Amari index takes values in [0, 1] and actually measures the departure from Bl,τ A l,τ to the identity matrix. The smaller the Amari index, the better the BSS. The mean over 20 simulations of ρ(t) is displayed in Fig. 3 for each of the algorithms. Amari indices averaged with respect to time and simulations are also given in Table I, with the corresponding SD with respect to simulations.

As expected, the Amari indices and SIR clearly show that JEFAS-BSS outperforms all other algorithms. JEFAS-BSS takes advantage of the a priori knowledge of the nonstationarity class involved in the observations. p-SOBI allows improving the Amari index compared to SOBI, but the SIR is impaired because of the imposed trade-off on the subintervals lengths: they must be long enough to ensure the relevance of SOBI, but must not be too long so that the mixing matrix can be approximately considered as piecewise constant. More surprisingly, QTF-BSS performance is of the same order as that of SOBI. This may be due to the fact that QTF-BSS and SOBI estimations rely on a constant mixing matrix model, then the time-varying mixing matrix deteriorates in the same way for both algorithms even though QTF-BSS is adapted to BSS of nonstationary sources while SOBI is not. Lastly, the results show that the strategy consisting in taking into account all the available combinations of N y does not really improve the BSS quality in comparison with Single JEFAS-BSS, where a single combination of sources is considered. Nevertheless, JEFAS-BSS remains valuable thanks to its robustness in situations where one of the submatrices A l,τ is ill-conditioned. In contrast, Single JEFAS-BSS could diverge in such a case, as illustrated on an extra example available online1 .

V. CONCLUSION

We have introduced JEFAS-BSS, a spectral estimation algorithm for a class of multivariate nonstationary signals. We have shown that the problem can also be interpreted as a doubly nonstationary BSS problem. We evaluated JEFAS-BSS on a synthetic example on which it outperforms concurrent BSS methods, which were not designed to handle double nonstationarity. We believe the model is general enough to be applied to real situations, including audio and EEG signals. Such applications are ongoing works. Extension to other types of stationarity-breaking deformations [START_REF] Meynard | Spectral estimation for non-stationary signal classes[END_REF] is also of interest.

The code and datasets used to produce the numerical results of this paper are available online 1 . Another illustration on a synthetic mixture of real audio signals, where we evaluate the influence of the speed variation of the mixing matrix on JEFAS-BSS results, is given in Supplementary material.

Spectral Estimation for Multivariate Locally Time

Warped Signals: Supplementary Material Adrien Meynard I. PROOF OF THEOREM 1

The notations are the same as in the main body of the article.

Proof. The approximation error is given by: τ = w z,τ -A(τ)w y,τ = Ay, ψ sτ -A(τ)y, ψ sτ = (A -A(τ))y, ψ sτ .

The Taylor-Lagrange formula states that for all the values of t, there exists a value u τ (t) between t and τ such that:

A(t) -A(τ) = A (u τ (t))(t -τ) .
Let M τ (t) denote the real matrix given by: M τ (t) = A (u τ (t))(tτ) ∈ R N z ×N y . Then:

[ τ ] im = N y ∑ j=1 [M τ ] ij D γ j X j , ψ s m τ = N y ∑ j=1 X j , D γ -1 j [M τ ] ij ψ s m τ . Let us denote g ij,sτ = D γ -1 j [M τ ] ij ψ sτ , then E |[ τ ] im | 2 = N y ∑ j=1 N y ∑ k=1 E X j , g ik,s m τ X k , g ij,s m τ .
Since the stationary sources X j are two-by-two independent, only the terms where k = j are non zero. Besides, by definition of the power spectrum, we have the following relation:

N y ∑ j=1 E X j , g ij,s m τ X j , g ij,s m τ = N y ∑ j=1 S X j , ĝij,s m τ 2 .
Let us find an upper bound for ĝij,sτ :

ĝij,s m τ (ξ) = R [M τ ] ij (t)ψ s m τ (t) γ j (t)e -2iπγ j (t)ξ dt ≤ R [A ∞ ] ij |t -τ| |ψ s m τ (t)| γ j 1 2 ∞ dt = [A ∞ ] ij [γ ∞ ] 1 2 j q 3sm 2 k ψ .
Then, we obtain:

E |[ τ ] im | 2 ≤ N y ∑ j=1 q 3s m σ 2 X [A •2 ∞ ] ij [γ ∞ ] j k 2 ψ = q 3s m σ 2 X [A •2 ∞ ] i• γ ∞ k 2 ψ ,
and finally, we get the main result of the theorem:

E | τ | •2 ≤ σ 2 X k 2 ψ A •2 ∞ γ ∞ (q 3s ) T .

II. ILLUSTRATION ON A SYNTHETIC MIXING OF AUDIO

SIGNALS

We complement the results of the paper with a performance evaluation on a synthetic mixture of two audio signals in the determined case i.e. N z = N y = 2. We stress that a full quantitative assessment is not possible here, as the underlying spectra are not available. The code and datasets used to produce the numerical results of this example are available online 1 .

The two nonstationary sources we have mixed are the sound produced by the wind blowing through a door and the sound of a singing woman. The mixing matrix coefficients are sinusoidally time-varying. The wavelet transforms of the observations are displayed on the top of Fig. 4. The interest of this example comes from the fact that each observation is composed of two source signals whose wavelet transforms are superimposed on large time-scale zones. Therefore, it highlights JEFAS-BSS ability to separate broadband sources. The scalograms of both estimated sources are displayed on the middle of Fig. 4, the one on the left corresponds to the estimated wind sound, while the one on the right corresponds to the voice sound. JEFAS-BSS also estimates the associated time warping functions and power spectra. Both estimated power spectra are superimposed on the bottom of Fig. 4. This figure shows the superposition of the spectral contents of the two sources, notably between 0 and 2 kHz (beyond 2 kHz, the values of the singing spectrum remain significant, and strongly dominate the values of the wind spectrum). The harmonic structure of the voice spectrum appears clearly.

Besides, we apply the same BSS algorithms on mixtures obtained from the same sources where the oscillation frequencies of the coefficients of the mixing matrix are evenly increased. We display on Fig. 5 the evolution of the performance indices as functions of the speed variation of the mixing matrix (quantified by A ∞ ). This highlights the ability of JEFAS-BSS to outperform the other BSS algorithms when the variations of the mixing matrix are controlled, whereas JEFAS-BSS is not able to capture the variability of A(t) when the speed variation increases. Therefore, its SIR and Amari index converge to those of a stationary-based BSS algorithm, namely SOBI.

Algorithm 1 Fig. 1 .

 11 Fig. 1. Top: scalograms of the nonstationary sources. Bottom: scalograms of the observations.

Fig. 2 .

 2 Fig. 2. Top: scalograms of the estimated sources. Middle: estimated time warping functions. Bottom: estimated spectra of the stationary sources.

Fig. 3 .

 3 Fig.3. Mean over 20 realizations of the temporal evolution of the Amari index (in dB) for various BSS algorithms.

Fig. 4 .

 4 Fig. 4. Synthetic mixing of audio signals. Top: Scalograms of both observations. Middle: Scalograms of both estimated sources. Bottom: Estimated power spectra of both underlying stationary sources.

5 Fig. 5 .

 55 Fig. 5. Evolution of the SIR (left) and the Amari index (right) in function of the speed variation of the mixing matrix.

TABLE I MEAN

 I OVER 20 REALIZATIONS OF THE SIR AND THE AVERAGED AMARI INDEX FOR DIFFERENT BSS STRATEGIES.

			Algorithm		SIR (dB) Mean SD		Averaged ρ (dB) Mean SD
			SOBI			5.93	0.60	-6.04	1.56
			p-SOBI			4.54	1.62	-7.18	2.37
			QTF-BSS		-0.13 3.05	-3.84	0.95
			Single JEFAS-BSS	29.40	2.16	-15.43	3.08
			JEFAS-BSS		29.96	2.61	-15.33	3.90
										SOBI
	Amari index (dB)	-15 -10 -5								p-SOBI QTF-BSS Single JEFAS-BSS JEFAS-BSS
		0	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
						Time (s)

MATLAB code at https://github.com/AdMeynard/JEFAS