Yannick Deville
email: ydeville@ast.obs-mip.fr

Shahram Hosseini
email: shahram.hosseini@ast.obs-mip.fr

Recurrent networks for separating extractable-target nonlinear mixtures. Part I: non-blind configurations

Keywords: source separation, inverse problem, nonlinear mixture, nonlinear recurrent neural network, stability

While most reported source separation methods concern linear mixtures, we here address the nonlinear case. Even for a known nonlinear mixing model, creating a system which implements the exact inverse of this model is not straightforward for most nonlinear models. We first define a large class of possibly nonlinear models, i.e. "Additive-Target Mixtures" (ATM), for which this inversion may be achieved thanks to the nonlinear recurrent networks that we propose to this end. We then further extend this approach to the "extractable-target mixtures" (ETM) that we also introduce in this paper. We illustrate these general approaches for two specific classes of mixtures, i.e. linear-quadratic mixtures, and quadratic ones. We then focus on our networks suited to linear-quadratic mixtures and we provide a detailed analysis of their equilibrium points and their stability. This allows us to introduce an automated procedure for selecting their free weights so as to guarantee the stability of a separating point for any source signals. Test results show the effectiveness of this approach for various types of source signals.

Problem statement

Source separation is a generic signal, image, and data processing problem [START_REF] Hyvarinen | Independent Component Analysis[END_REF]. The situation that it addresses may be defined as follows, considering one-dimensional signals for the sake of simplicity. A set of P observed signals x i (n) are available, e.g. from sensors. These observations are "mixtures" of a set of N unknown source signals s j (n), i.e. the observation vector x(n) = [x 1 (n), . . . , x P (n)] T may be expressed with respect to the source vector s(n) = [s 1 (n), . . . , s N (n)] T (where T denotes transposition), according to a mixing operator M :

x(n) = M [s(n)], (1)
where the class of functions corresponding to this model M is usually fixed, and its parameter values may be known or not, as detailed below. Source separation methods then aim at retrieving the unknown source signals s j (n) from their observed mixtures x i (n). The most natural methods therefore consist in applying an unmixing (or separating) operator U to these observations, to obtain the output vector

y(n) = U [x(n)]. ( 2
)
The issue is then essentially to make the model U equal to the inverse of the model M , so that y(n) becomes equal to the desired vector s(n) of source values. Within this overall framework, different cases may be defined, especially depending: (i) on the class of functions considered for the mixing model M and (ii) whether the parameter values of this model are known or not. Up to now, most reported investigations [START_REF] Hyvarinen | Independent Component Analysis[END_REF] have been focused on relatively simple mixing models, i.e. linear ones. The development of unmixing models corresponding to a linear model with given parameters has only required limited effort, and the emphasis has been mainly put on the difficulty which arises when the mixing parameters are unknown. One then has to estimate these parameters only from the observations, thus achieving blind source separation (BSS). Many BSS methods have been developed during the last decades, mainly using the concept of Independent Component Analysis [START_REF] Hyvarinen | Independent Component Analysis[END_REF], and more recently by resorting to Sparse Component Analysis or Non-negative Matrix Factorization (see e.g. the overview in [START_REF] Berry | Algorithms and applications for approximate nonnegative matrix factorization[END_REF] and references therein). BSS methods essentially aim at recovering the waveforms of the source signals, i.e. at estimating the source signal values e.g. up to fixed scale factors and additive constants, and with possible permutations between the indices of the original source signals and their restored versions (this corresponds to classical BSS indeterminacies [START_REF] Hyvarinen | Independent Component Analysis[END_REF]).

Our focus in this paper is different, i.e. we aim at providing source separation methods for nonlinear mixtures. Such mixtures have already been considered in the literature (see e.g. the survey in [START_REF] Jutten | Advances in Blind Source Separation (BSS) and Independent Component Analysis (ICA) for Nonlinear Mixtures[END_REF]) but in a much less detailed way than linear mixtures. It is well known [START_REF] Hyvarinen | Nonlinear independent component analysis: Existence and uniqueness results[END_REF], [START_REF] Taleb | Source separation in post-nonlinear mixtures[END_REF] that the independence hypothesis is not sufficient for achieving BSS from general nonlinear mixtures, because of the very large indeterminacies which make the nonlinear BSS problem ill-posed. A natural idea for reducing the indeterminacies is to constrain the structure of mixing and separating models to belong to a certain class of transforms. This additional constraint can be viewed as a regularization of the initially ill-posed problem [START_REF] Taleb | A generic framework for blind source separation in structured nonlinear models[END_REF], [START_REF] Jutten | Three easy ways for separating nonlinear mixtures?[END_REF]. This is the approach that we use in this paper, where we consider a particular class of mixing models. Anyway, nonlinear mixtures are much more difficult to handle than linear ones, even when considering a nonlinear mixing model M corresponding to a fixed function with known parameter values: in such a non-blind configuration, creating a system which implements the inverse of the considered mixing model (and which could then be extended to the blind configuration) is not straightforward for most nonlinear models, as detailed further in this paper. This is therefore the problem that we aim at solving in the first part of this paper, for a large class of nonlinear mixtures that we introduce to this end. We reach this goal thanks to new recurrent artificial neural network structures, that we derive hereafter as extensions of the source separation networks proposed in previous papers for linear mixtures.

More precisely, in the simplest source separation configuration considered in the literature, the observations are Linear Instantaneous (LI) mixtures of the source signals. The above mixing operator M may then be defined by a matrix A, composed of mixing coefficients a ij , and the mixing model (1) may be expressed by the matrix-vector product

x(n) = As(n). (3)
One of the very first reported solutions for this LI source separation problem is the Hérault-Jutten artificial neural network [START_REF] Jutten | Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture[END_REF]. This network has a recurrent (or feedback) structure: each of its outputs y i (n) consists of an LI combination of

• all other outputs y j (n) with j = i, using combination coefficients, i.e. neural network "weights", denoted l ij hereafter,

• and input x i (n), with a weight fixed to 1.

In a blind configuration, the weights l ij are estimated from the output signals y i (n), using the unsupervised adaptation (i.e. "training") algorithm which was first proposed by Hérault and Jutten [START_REF] Jutten | Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture[END_REF] and then analyzed by several authors [START_REF] Comon | Blind separation of sources, Part II: Problems statement[END_REF]- [START_REF] Deville | A unified stability analysis of the Hérault-Jutten source separation neural network[END_REF].

For this class of mixtures, a recurrent structure is not mandatory however, i.e. the same class of LI mappings from the signals x i (n) to the signals y i (n) may also be achieved by a feedforward structure. In the latter structure, each output y i (n) is derived only as an LI combination of all inputs, typically with a weight equal to 1 for the input x i (n) which has the same index i as the considered output y i (n), and with weights again denoted l ij for all other inputs x j (n), i.e. for j = i. This structure was e.g. used by Macchi and Moreau [START_REF] Macchi | Self-adaptive source separation, Part I: convergence analysis of a direct linear network controled by the Hérault-Jutten algorithm[END_REF]- [START_REF] Moreau | Self-adaptive source separation, Part II: comparison of the direct, feedback and mixed linear network[END_REF]. This is the mostly used LI network today, because it is simpler than the LI recurrent structure and it avoids the stability constraints of that structure.

Still for LI mixtures, extended structures were also proposed, with additional weights l ii between each input and output having the same index i. Corresponding adaptation algorithms for the blind context were e.g. introduced by Cichocki et al. in [START_REF] Cichocki | Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals[END_REF] and analyzed by Deville in [16]. Beyond LI mixtures, many investigations reported in the literature are devoted to convolutive mixtures. The same evolution, from recurrent to feedforward separating structures, also occurred for these convolutive mixtures, because feedforward convolutive networks provide the considered convolutive mappings while avoiding some stability constraints of recurrent convolutive structures.

On the contrary, feedforward structures are much less attractive for nonlinear mixtures, which are now receiving increasing attention. This restriction stems from the fact that feedforward structures consist in expressing the desired network outputs, i.e. the source signals, directly with respect to the observed signals. In other words, they require us to derive an explicit analytic expression of the inverse of the mixing (i.e. direct) model M , which is not possible for given arbitrary nonlinear models M : for instance, if M is a higher-order polynomial model, creating an associated feedforward separating structure would mean deriving the analytic solution of the higherorder polynomial mixing equations, which is not feasible. For the sake of clarity, let us contrast this problem with the case of linear mixtures: to derive a feedforward structure for the latter mixtures, we only have to solve linear equations, which is straightforward and yields the linear feedforward structure that we discussed above. While feedforward structures are therefore not suited to nonlinear mixtures, we will show in the next section that the above recurrent structures may be extended to a large class of nonlinear mixtures. Let us stress again that this first part of our investigation is focused on separating structures themselves and therefore on the non-blind configuration, as opposed to the adaptation of the parameters of such structures in a blind context. The latter topic will be addressed in the second part of this paper.

The remainder of this first part of our paper is therefore organized as follows. Section 2 deals with separating structures. Based on the above discussion, we focus on recurrent structures in that section. We first provide a (presumably new) analysis of the operation of existing linear recurrent structures. This then allows us to extend them to new classes of nonlinear mixtures, that we introduce at this stage. We illustrate the proposed general approach by applying it to several specific classes of nonlinear mixtures, especially linear-quadratic mixtures. We conclude this section by defining how this investigation may then be continued, once our approach has been used for creating networks suited to a given type of mixtures. To illustrate the proposed procedure, we then focus on linear-quadratic mixtures in Section 3. We derive the fixed points of the proposed networks and analyze their stability. We thus propose a method for selecting their weights so as to avoid the divergence issue exhibited by the basic version of our networks. Various tests are used in Section 4 to confirm this theoretical analysis and to show the performance of our networks for different types of source signals. Conclusions are drawn from this investigation in Section 5.

We only consider real-valued signals and models hereafter, but several aspects of this paper are directly applicable or extendable to the complex case. When dealing with separating structures, we consider the classical "determined" configuration, i.e. the case when the numbers P and N of observations and sources are equal.

2 Proposed recurrent separating structures for nonlinear mixtures

Analysis of linear recurrent separating structures

Based on the above considerations, we will focus on recurrent separating structures in the remainder of this paper and we will consider increasingly complex mixing models. The simplest case therefore concerns P = 2 LI mixtures of N = 2 sources. The LI mixing model (3) then reads explicitly

x 1 (n) = a 11 s 1 (n) + a 12 s 2 (n) (4) x 2 (n) = a 21 s 1 (n) + a 22 s 2 (n). ( 5
)
It may also be expressed with respect to the "normalized" source signals which are defined as

1 s i (n) = a ii s i (n) ∀ i ∈ {1, 2} (6)
and which are equal to the contribution of the source with index i in the observation with the same index i. Defining

L ij = - a ij a jj ∀ i = j ∈ {1, 2}, (7)
(where we assume all a jj are non-zero), the above mixing equations ( 4)-( 5) may be rewritten as

x 1 (n) = s 1 (n) -L 12 s 2 (n) (8)
x 2 (n) = -L 21 s 1 (n) + s 2 (n). (9)
Source separation then aims at extracting both signals s i (n) from these mixtures (up to classical indeterminacies [START_REF] Hyvarinen | Independent Component Analysis[END_REF]). As stated in the previous section, this may be achieved by using the Hérault-Jutten recurrent structure. We here express the operation of this structure in a way which has not been reported in the literature to our knowledge, and which will then allow us to extend this structure to much more general mixtures. For each time n, this structure receives the couple of observations (x 1 (n), x 2 (n)) and computes the values of its outputs y i by performing a recurrence. We denote as m the index associated to this recurrence and y i (m) the successive values of each output in this recurrence performed at time n 2 . This recurrence reads

y 1 (m + 1) = x 1 (n) + l 12 y 2 (m) (10)
y 2 (m + 1) = x 2 (n) + l 21 y 1 (m) (11)
where l ij are the weights of this neural network 3 . The new values y i (m + 1) are then used as the input data of the next occurence of the loop associated to this recurrence. The corresponding network may then also be considered as a looped structure (see Fig. 1), where the outputs y i (m) are fed back and combined with the inputs x i (n), to create new adder outputs y i (m+1), which then become the new outputs y i (m) for the next round through the loop. Now consider the situation when the network weights are set to [START_REF] Jutten | Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture[END_REF]:

l ij = L ij ∀ i = j ∈ {1, 2}. (12)
+ + x (n)

x (n) y Let us assume that, at step m of the recurrence, we obtained

+ + x (n) x (n) 1 2 2 21 12 1 1 2 s' s' (n) (n) L L s' (n) s' (n)
y i (m) = s i (n) ∀ i ∈ {1, 2} (13)
and let us consider the next occurence of the recurrence loop. Fig. 1 here becomes Fig. 2 and the network operates as follows. The outputs y i (m) are fed back, with the corresponding network weights l ij , and added to the observations. Each observation x i (n) defined by ( 8)-(9) consists of: (i) a "target term", which is the part of this observation that we would like to keep and which is equal to s i (n) and (ii) an "interfering term", which is the part of this observation that we would like to remove and which is equal to -L ij s j (n), with j = i. The major phenomenon is that, in the conditions considered here, each feedback term l ij y j (m) becomes an exact "cancelling term", i.e. it exactly compensates for the above interfering term, so that the output of each adder becomes exactly equal to the target term s i (n). We thus get

y i (m + 1) = s i (n) ∀ i ∈ {1, 2}, (14)
i.e. condition (13) is still met after running one occurence of the loop. It will therefore be met endlessly if the loop is executed again and again in these conditions. In more formal terms, this means that, in the conditions defined by ( 8)-( 9) and ( 12), the point defined by ( 13) is a fixed point (i.e. an equilibrium point) of recurrence ( 10)- [START_REF] Fort | Stabilité de l'algorithme de séparation de sources de Jutten et Hérault[END_REF]. This may be directly checked mathematically from all these equations, instead of considering Fig. 1 and2.

In this paper, we aim at extending this analysis to nonlinear mixtures, and we therefore now introduce a first class of nonlinear mixtures suited to this approach.

Proposed additive-target mixtures

General model

Based on the above results, we here introduce a general class of possibly nonlinear mixtures 4 . We call this model "Additive-Target Mixtures" (ATM), since it corresponds to observations which read

x i (n) = T i [s(n)] -I i [s(n)] ∀ i ∈ {1, . . . , P } (15)
where T i [s(n)] and I i [s(n)] are respectively the target and interfering terms of the considered model, i.e. the components of x i (n) that we aim at keeping and removing in the network outputs.

The target terms T i [s(n)] may take different forms in our approach. In the simplest configuration, they are equal to the source signals s i (n), up to classical scale (or filter) indeterminacies. In other words, they are typically equal to the signals s i (n) which appear in the normalized version of the mixing equations, which was developed above for LI mixtures (see ( 8)-( 9)), and which will be detailed below for specific nonlinear mixtures. The mixing model ( 15) then becomes

x i (n) = s i (n) -I i [s(n)] ∀ i ∈ {1, . . . , P }. ( 16
)
Other configurations involve more general forms for the target terms T i [s(n)]. Anyway, since each target term T i [s(n)] aims at extracting a source, it only depends on one source, which is selected to be s i (n). Each target term may then be expressed as T i [s i (n)]. Each single-input single-output function T i is assumed to be invertible5 (at least for the domain that s i (n) is allowed to span). Its inverse is denoted T -1 i . We now illustrate the above two mixing configurations by means of specific examples, for which we will then also detail associated separating networks.

First application: linear-quadratic mixtures

One of the simplest extensions of the LI model ( 4)-( 5) consists in considering instantaneous mixtures which also include quadratic cross-terms, i.e. terms proportional to the product of both source signals. The observations then read

x 1 (n) = a 11 s 1 (n) + a 12 s 2 (n) + b 1 s 1 (n)s 2 (n) (17) x 2 (n) = a 21 s 1 (n) + a 22 s 2 (n) + b 2 s 1 (n)s 2 (n). ( 18
)
Still using ( 6)-( 7) and now defining

Q i = - b i a 11 a 22 ∀ i ∈ {1, 2}, (19)
(where we still assume all a jj are non-zero), the above mixing equations ( 17)-( 18) may be rewritten as

x 1 (n) = s 1 (n) -L 12 s 2 (n) -Q 1 s 1 (n)s 2 (n) (20) x 2 (n) = -L 21 s 1 (n) + s 2 (n) -Q 2 s 1 (n)s 2 (n). (21)
So, this mixture model is indeed a specific case of the class of models defined in [START_REF] Cichocki | Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals[END_REF], where each target term T i [s(n)] is restricted to s i (n), and each interfering term

I i [s(n)] consists of all other components of x i (n) in (20)-(21) 6 .

Second application: quadratic mixtures

Another simple nonlinear mixing model only consists of quadratic terms, i.e. auto-terms and cross-terms. It reads

x 1 (n) = [a 11 s 1 (n)] 2 + [a 12 s 2 (n)] 2 + b 1 s 1 (n)s 2 (n) (22) x 2 (n) = [a 21 s 1 (n)] 2 + [a 22 s 2 (n)] 2 + b 2 s 1 (n)s 2 (n). ( 23
)
Still using ( 6), ( 19) and now defining

L ij = - a 2 ij a 2 jj ∀ i ∈ {1, 2}, (24)
the above mixing equations ( 22)-( 23) may be rewritten as

x 1 (n) = [s 1 (n)] 2 -L 12 [s 2 (n)] 2 -Q 1 s 1 (n)s 2 (n) (25) x 2 (n) = -L 21 [s 1 (n)] 2 + [s 2 (n)] 2 -Q 2 s 1 (n)s 2 (n). (26)
This mixture model is therefore also a specific case of the class of models defined in [START_REF] Cichocki | Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals[END_REF], where each target term 25)-(26). Both functions T i with i ∈ {1, 2} are then equal to the squaring function. They are therefore invertible, provided the source signals are constrained to have a constant sign.

T i [s(n)] is equal to [s i (n)] 2 , and each interfering term I i [s(n)] consists of all other components of x i (n) in (
Many other specific mixing models belonging to the class defined by ( 15) may also be considered by the reader, depending on his/her needs. To handle them and all ATM, we now propose general separating structures suited to the global model ( 15), and we then detail their specific versions corresponding to the above two applications of this mixing model.

Proposed networks for additive-target mixtures

General structures

We here extend the principles that we introduced in Section 2.1 to the class of mixtures that we defined in Section 2.2. We first consider the basic version ( 16) of these mixtures and we propose to process it with a recurrent structure defined as follows. The behavior of this network is governed by a set of operators C i . These operators may be static or dynamic, depending on the nature of the mixing model itself. For each time n, this structure receives the observation vector x(n) and computes the values of its output vector y by performing a recurrence. We again denote as m the index associated to this recurrence and y(m) the successive values of the output vector in this recurrence performed at time n7 . This recurrence reads

y i (m + 1) = x i (n) + C i [y(m)] ∀ i ∈ {1, . . . , P }. ( 27
)
The structure of this recurrent network is shown in Fig. 3 for the simplest configuration, i.e. P = N = 2. Now consider the case when the network operators C i are "fitted" to the interfering terms of the mixing model, i.e.

C i [s (n)] = I i [s(n)] ∀ i ∈ {1, . . . , P }. ( 28
)
Assume we start step m of the recurrence (27) with

y i (m) = s i (n) ∀ i ∈ {1, . . . , P }. ( 29
)
Then, using the same approach as in Section 2.1, it may be shown easily that, at the end of this step of the recurrence, we still have In other words, the above-defined choice for the operators C i is such that: (i) they exactly cancel the interfering terms of the mixing model, so that these C i may be called "cancellation operators", and (ii) the output state (29) is a fixed point of the recurrence (27) associated to the considered network.

y i (m + 1) = s i (n) ∀ i ∈ {1, . . . , P }. ( 30
) + + x (n) x (n) y (n) y (n) 1 2 1 2 C [.] 1 C [.] 2
+ + x (n) x (n) y (n) y (n) 1 2 1 2 C [.] 1 C [.] 2 1 -1 -1 2 T [.] T [.]
Let us now extend this approach to the general mixing model [START_REF] Cichocki | Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals[END_REF]. The proposed method then essentially makes it possible to extract the target terms T i [s i (n)] from the observations, as "intermediate outputs". To then retrieve the sources s i (n) themselves 8 from the extracted target terms, one just has to transfer each above intermediate output through the corresponding inverse mapping T -1 i . This yields two alternative structures for our recurrent separating systems, depending whether the signals fed back to the inputs are the "final outputs" corresponding to s i (n) or the intermediate outputs associated to T i [s i (n)]. These two separating structures are respectively shown in Fig. 4 and 5 for two sources. One derives in the same way as above the recurrence equations corresponding to these structures (for any number of sources) and shows that the values of the operators C i for which the extracted sources define a fixed point of these recurrences read

C i [.] = I i [.] ∀ i ∈ {1, . . . , P } (31)
for the structure shown in Fig. 4 and

C i [T [.]] = I i [.] ∀ i ∈ {1, . . . , P } (32)
for the network of Fig. 5, where the notation T [.] in (32) represents a vector obtained by transferring each of its entries with index i through the mapping T i .

All the recurrent networks that we thus proposed therefore have the following main features:

• All these structures do possess a fixed point for which they extract the source signals.

• These structures avoid the inversion issue of feedforward networks: the operators C i that they use are essentially "equal" to the direct mixing model, i.e. they are directly related to its interfering part I i , not to its inverse, as shown by ( 28), ( 31) and (32). These structures 8 The following discussion may be straightforwardly modified by considering

s i (n) instead of s i (n). + + x (n) x (n) 1 2 C [.] 1 C [.] 2 1 -1 y (n) 1 2 -1 y (n) 2 T [.] T [.]
Figure 5: Proposed recurrent network for general additive-target mixtures, with adder outputs not inverted before feedback. therefore do not require us to derive an analytic model of the inverse of the considered mixing model.

+ + x (n) x (n) y (n) y (n) 1 2 1 2 C [.] 1 C [.] 2 F [.] 1 F [.] 2
As noted in Section 1, extended structures for linear mixtures have been proposed in the literature by adding feedback loops from each output to each input having the same index i. We may also consider such extensions for our approach, thus introducing self-feedback operators F i in each way i of our above structures. For instance, the structure in Fig. 3 is thus transformed into the network of Fig. 6. This extended approach provides additional flexibility in the design of the proposed structures. This flexibility is indeed required, as shown further in this paper when applying these structures to the specific mixture model that we introduced in Section 2.2.2.

First application: linear-quadratic mixtures

We here consider observations defined by ( 20)- [START_REF] Harris | Handbook of mathematics and computational science[END_REF]. As explained in Section 2.2.2, each target term T i [s(n)] is then restricted to s i (n). The basic approach that we propose in this case is therefore defined by the recurrence equation ( 27) and the structure of Fig. 3. Applying these general results to the mixing model ( 20)-( 21), this recurrence becomes

y 1 (m + 1) = x 1 (n) + l 12 y 2 (m) + q 1 y 1 (m)y 2 (m) ( 33
)
y 2 (m + 1) = x 2 (n) + l 21 y 1 (m) + q 2 y 1 (m)y 2 (m) (34)
where l ij and q i are the weights of the basic neural network that we propose to process this type of mixtures. The structure of this network is shown in Fig. 7.

An extended form of this network, corresponding to the general structure of Fig. 6, may then be derived by introducing linear self-feedback in each way i of the above structure. Denoting l ii the self-feedback weights, the recurrence equations then become: (36)

y 1 (m + 1) = x 1 (n) + l 11 y 1 (m) + l 12 y 2 (m) + x (n) 1 l q q l + x (n) y (n) 1 y (n)
+ x (n) 1 l q q l + x (n) y (n) 1 y (n)
The corresponding extended structure is shown in Fig. 8.

Second application: quadratic mixtures

Now consider observations defined by ( 25)-( 26). Since we showed in Section 2.2.3 that the target terms T i [s(n)] are here equal to [s i (n)] 2 , we may now apply any of the structures of Fig. 4 and5.

For the mixing model considered here, a direct application of the approach of Fig. 4 to positive source signals yields the structure of Fig. 9. The other general structures that we proposed may be applied to this specific mixing model in the same way.

Extension to extractable-target mixtures

The above approach may be further extended by introducing a much wider class of mixing models, that we call "extractable-target mixtures" (ETM) due to their definition that we now provide. These models also involve target and interfering terms T i [s(n)] and I i [s(n)], but which are now combined according to a quite general function f (u, v) to form the observations, i.e. The only constraint that we set on f is that there should exist an associated function9 g(w, z), whose analytic expression may be derived from that of f and which is such that

f (T i [s i (n)], I i [s(n)]) = x i (n) ∀ i ∈ {1, . . . , P }. ( 37
) + x (n) 1 l q q l + x (n) y (n) 1 y (n)
g(f (u, v), v) = u. ( 38
)
The motivation for setting this constraint is that, when fed with w = x i (n) and z = I i [s(n)], the operator g(w, z) provides T i [s i (n)], as shown by ( 37) and (38). The target terms T i [s i (n)] thus become extractable when combining the function f corresponding to the undesired mixing phenomenon and the function g that we introduce in our separating networks to cancel the effect of f . More precisely, we build these networks by starting from those dedicated to ATM and shown in Fig. 4 and 5, and by just replacing their two-input adders by the two-input function g (the approach may then be extended to more than two sources). By applying the ideas developed in the previous sections to this new mixing and separating configuration it may be checked easily that, for adequate operators C i again directly related to the interference operators I i , these new networks have a fixed point which corresponds to the extraction of the source signals.

While the above description aimed at being general and was therefore rather formal, let us now illustrate it briefly with two practical examples. As suggested by the above discussion itself, the ATM that we introduced in Section 2.2.1 are a specific case of ETM. They correspond to

f (u, v) = u -v (39) g(w, z) = w + z, (40)
as may be checked by the reader. Similarly, we can define "multiplicative-target mixtures" (MTM) by

f (u, v) = u/v (41) g(w, z) = w × z. (42)
Concretely, this means that we consider mixtures defined as

x i (n) = T i [s i (n)]/I i [s(n)] ∀ i ∈ {1, . . . , P }, (43)
as shown by (37) and (41). The networks that we propose for separating such mixtures are then derived from those shown in Fig. 4 and5, by just replacing their adders by multipliers10 .

Achievements and remaining tasks

At this stage, we have mainly obtained the following results. We have defined a general procedure which allows us to create several recurrent separating networks dedicated to any given mixing model belonging to the ETM family, especially including ATM. We have also described this procedure in more detail for simple nonlinear mixtures, i.e. especially linear-quadratic mixtures, which result in the basic and extended networks respectively shown in Fig. 7 and8. Before using any such network, one would ideally like to know its properties, especially what are the fixed points of this dynamic structure and which of them are stable. This is of special importance here because, since we are considering nonlinear mixtures, the separating recurrent networks that we created are nonlinear dynamic systems, and it is well-known that such systems may yield unstable or even chaotic behavior [START_REF] Thompson | Nonlinear dynamics and chaos[END_REF]. Deriving all fixed points is only feasible for possibly nonlinear but still relatively simple mixtures, since it is directly related to the derivation of an analytic expression of the inverse of the mixing model 11 . At least, the following result about fixed points holds for any ETM model: we showed above that our structures have a fixed point by construction, which is the point corresponding to the extraction of all sources (also called the separating point), i.e. the main point of interest. One may then analyze the stability of this point.

To illustrate these principles, we will focus again on linear-quadratic mixtures in the next section. We will derive all the fixed points of the proposed basic and extended networks, and we will analyze their stability. As the extended structure of Fig. 8 is a superset of the basic network of Fig. 7, we will analyze the former one and thus obtain results for the latter as a specific case, i.e. by setting l 11 = 0 and l 22 = 0.

(44)

It may be mentioned at this stage that, before proposing the extended structure of Fig. 8 (and considering new mixing models) in this paper, we described in the short communications [START_REF] Deville | Méthode de séparation de sources pour mélanges linéaires-quadratiques[END_REF]- [START_REF] Hosseini | Blind maximum likelihood separation of a linear-quadratic mixture[END_REF] preliminary investigations where we only addressed linear-quadratic mixtures and we only introduced the basic structure of Fig. 7. Despite its attractive features, the experimental tests that we performed with this specific structure showed that it may become unstable, thus confirming our above warning about the general behavior of nonlinear dynamic systems. These limitations of the basic network are a strong motivation for here introducing its extended version of Fig. 8: this new structure has extra parameters, i.e. the linear feedback weights l 11 and l 22 , that we will aim at selecting in order to guarantee the stability of this structure. On the contrary, we will demonstrate that the constraint (44) corresponding to our initial network does not allow it to handle all signal values.

3 Analysis of linear-quadratic networks

Fixed and separating points

We here analyze the fixed points (i.e. equilibrium points) of recurrence ( 35)-(36), for observations expressed as ( 20)- [START_REF] Harris | Handbook of mathematics and computational science[END_REF]. They are defined as the points (y E 1 , y E 2 ) which are such that

y 1 (m + 1) = y 1 (m) = y E 1 and y 2 (m + 1) = y 2 (m) = y E 2 . ( 45
)
We again consider the non-blind configuration, i.e. the case when the mixing coefficients L ij and Q i of ( 20)-( 21) are known. Let us stress that this non-blindness constraint is not highly restrictive: it not only includes the specific case when the mixing coefficients L ij and Q i are known a priori (e.g. thanks to physical knowledge about the considered system), but also the more general situation when they are derived once for all from a set of known source values during a first step of the approach. In the latter case, the proposed procedure consists in using these known source values and measuring the corresponding observed signals. This yields a set of linear equations with unknowns L ij and Q i , as shown by ( 20)- [START_REF] Harris | Handbook of mathematics and computational science[END_REF]. Numerically solving these equations yields the mixing parameter values L ij and Q i . In the second step of our approach, these fixed parameter values are then used for unknown source signal values in the separating networks considered in the remainder of this paper 12 . Note that the same type of procedure may also be defined for other mixing models, such as the quadratic mixtures introduced in Section 2.2.3. So, considering a set of known mixing coefficients, we aim at selecting network weights which meet the condition: "one of the associated fixed points corresponds to source separation without permutation and with possible scale factors", because we aim our network at converging to such a point. We therefore first determine all network weights which meet the above condition. This condition corresponds to the separating point (more precisely, the class of points) defined by

y E 1 = k 1 s 1 (n) and y E 2 = k 2 s 2 (n) ( 46
)
where k 1 and k 2 are two arbitrary scale factors. Combining ( 20)-( 21), ( 35)-( 36) and ( 45)-( 46) yields

[(l 11 -1)k 1 + 1]s 1 (n) + [l 12 k 2 -L 12 ]s 2 (n) (47) +[q 1 k 1 k 2 -Q 1 ]s 1 (n)s 2 (n) = 0 [l 21 k 1 -L 21 ] s 1 (n) + [(l 22 -1)k 2 + 1]s 2 (n) (48) +[q 2 k 1 k 2 -Q 2 ]s 1 (n)s 2 (n) = 0.
For given mixing parameters in ( 20)-( 21), the network weights l ij and q i and scale factors k i should be selected so that (47)-( 48) are met whatever the normalized source values s i (n). All source coefficients in these equations should therefore be zero, which yields

l 11 = - 1 k 1 + 1 (49) l 12 = L 12 k 2 = L 12 l 22 (50)
q 1 = Q 1 k 1 k 2 = Q 1 l 11 l 22 (51)
l 21 = L 21 k 1 = L 21 l 11 ( 52
)
l 22 = - 1 k 2 + 1 ( 53
)
q 2 = Q 2 k 1 k 2 = Q 2 l 11 l 22 (54)
with l 11 = 1 -l 11 and l 22 = 1 -l 22 .

(55)

The first expressions in (50)-( 52) and ( 54) show that we thus obtain an infinite number of solutions, due to the two arbitrary scale factors k i with which the sources appear in the network outputs.

There is a one-to-one correspondence between these factors and the two network weights l 11 and l 22 as shown by ( 49) and (53). One may therefore consider the weights l 11 and l 22 as the primary parameters, select them (freely at this stage), and then assign accordingly the other network weights, using the second expressions in (50)-( 52) and (54).

For these weight values, we know by construction that the network has at least one fixed point, i.e. the point defined by ( 46) with ( 49) and (53). We must then determine all fixed points for these weight values, because the network may converge to any of these points (depending on their 12 There should be no confusion between this approach and alternating estimation techniques, applied to mixture parameters and sources, that one might aim at developing for the blind configuration: in the approach proposed here, we first consider once for all known source values, from which we derive the exact mixing parameter values (i.e. no statistical parameter estimation is required here). Then, the network is always operated with the unknown source values that we aim at restoring, in a non-blind mode (from the mixture point of view), i.e. by using the exact mixing parameter values that we first derived in our first step (therefore without estimating nor adapting the mixing parameter values in that second step).

stability) and we should especially determine whether each of them yields separated sources. To this end, we again consider the equations which define all fixed points, i.e. ( 35)-(36) combined with (45), which yields

x 1 (n) -l 11 y E 1 + l 12 y E 2 + q 1 y E 1 y E 2 = 0 (56) x 2 (n) + l 21 y E 1 -l 22 y E 2 + q 2 y E 1 y E 2 = 0. ( 57
)
These equations may e.g. be solved by first deriving a linear combination of them which cancels their cross-terms y E 1 y E 2 , such as q 2 ×(56) -q 1 ×(57). This yields a linear relationship between y E 1 and y E 2 , from which one then derives the expression of y E 2 vs y E 1 , i.e.

y E 2 = (q 2 l 11 + q 1 l 21 )y E 1 -[q 2 x 1 (n) -q 1 x 2 (n)] q 2 l 12 + q 1 l 22 . ( 58
)
Inserting the latter expression of y E 2 in (56), one gets a second-order polynomial equation for y E 1 , which is then straightforwardly solved, thus yielding

y E 1 = -[q 2 x 1 (n) -q 1 x 2 (n) + l 11 l 22 -l 12 l 21 ] + y1 ∆ 1/2 y1 -2[q 2 l 11 + q 1 l 21 ] ( 59
)
where

y1 = ±1 (60)
defines which solution of the second-order polynomial equation in y E 1 is considered, and ∆ y1 is the discriminant of this equation. The corresponding expressions of y E 2 may then by derived from its above-mentioned expression (58) vs y E 1 . Subsequent expressions of ∆ y1 , y E 1 and y E 2 are then obtained by inserting in the above ones: (i) the expressions (50)-( 52) and ( 54)-( 55) of the considered network weights and (ii) the expressions ( 20)-( 21) of the observations with respect to the source signals. Eq. ( 50)-( 52) and ( 54 with

δ y1 = [Q 2 x 1 (n) -Q 1 x 2 (n) + γ] 2 -4α[x 1 (n) + x 2 (n)L 12 ] (62)
where

α = Q 2 + Q 1 L 21 (63) γ = 1 -L 12 L 21 . (64)
Inserting ( 20)-( 21) in the above equations, (62) becomes

δ y1 = (-αs 1 (n) + βs 2 (n) + γ) 2 (65)
where

β = -(Q 2 L 12 + Q 1 ). ( 66
)
It should be noted that (65), combined with (61), guarantees that ∆ y1 is positive in the considered conditions 13 . The above equations therefore do have real solutions y E 1 and hence y E 2 , i.e. the network does have fixed points. As mentioned above, we already knew by construction that it has at least one fixed point, i.e. the separating point. The result obtained here is more accurate, i.e. it has exactly two (possibly equal) fixed points, defined by the above equations.

The selection of one of the two solutions y E 1 of ( 59) is based on the sign of y1 (see ( 60)) and on the signs of the quantities involved in ∆ 1/2 y1 , i.e. l 11 l 22 and (-αs 1 (n) + βs 2 (n) + γ), due to (61) and (65). It is therefore convenient to introduce the "total sign variable"

T 1 = y1 sgn(l 11 l 22 ) sgn(-αs 1 (n) + βs 2 (n) + γ).
(67)

The two fixed points may then be expressed as

y E 1 = 1 2l 11 α {[αs 1 (n) + βs 2 (n) + γ] -T 1 [-αs 1 (n) + βs 2 (n) + γ]} (68)
y E 2 = 1 βl 22 {-αl 11 y E 1 + [αs 1 (n) + βs 2 (n)]}. ( 69
)
Their explicit expressions, depending on T 1 , are as follows. The first solution, corresponding to

T 1 = 1, reads y E 1 = 1 l 11 s 1 (n) and y E 2 = 1 l 22 s 2 (n). (70)
This solution yields source separation without permutation (and with scale factors). It is nothing but the above solution (46), as shown by ( 49), ( 53) and ( 55). The second solution, corresponding to T 1 = -1, reads

y E 1 = 1 l 11 β α s 2 (n) + γ α , y E 2 = 1 l 22 α β s 1 (n) - γ β . (71)
This additional solution yields source separation with a permutation (and with scale factors and additive constants). Therefore, for these weight values, this network does not yield spurious fixed points, i.e. fixed points such that the outputs are still mixtures of the sources. We now analyze the stability of the above two fixed points.

Stability condition

The considered recurrent network is a two-dimensional nonlinear dynamic system, since the evolution of its state vector [y 1 (m), y 2 (m)] T is defined by the nonlinear equations ( 35)-(36). The linear stability of any such system at a given fixed point may be analyzed by considering a first-order approximation of its evolution equations at that point 14 . The new value of a small disturbance added to the state vector is thus expressed as the product of a matrix H, which defines the first-order approximation of the system, by the former value of that disturbance (see e.g. details in [START_REF] Thompson | Nonlinear dynamics and chaos[END_REF]). The (asymptotic) stability of the system at the considered point is guaranteed by constraining the moduli of both eigenvalues of the corresponding matrix H to be lower than 1. Deriving these eigenvalues and applying the above constraint set on them, it may be shown that this stability condition reads

   T > -D -1 T < D + 1 D < 1 (72)
where D and T are respectively the determinant and trace of H. The stability region thus obtained in the (D, T ) plane is bounded by a triangle which includes the origin (see the figure in [START_REF] Thompson | Nonlinear dynamics and chaos[END_REF], p. 153, which also indirectly confirms (72)).

The above general procedure is here applied by computing a first-order approximation of the recurrence (35)-(36) at any fixed point. We thus get

H = q 1 y E 2 + l 11 q 1 y E 1 + l 12 q 2 y E 2 + l 21 q 2 y E 1 + l 22 . (73)
This directly yields

T = q 1 y E 2 + l 11 + q 2 y E 1 + l 22 . (74)
Additional manipulations using (58)-(59) then show that

D = T -1 + y1 ∆ y1 . (75)
Condition (72) then becomes

2T > -y1 ∆ y1 (76) 0 < y1 ∆ y1 (77)
T + y1 ∆ y1 < 2. ( 78
)
Eq. ( 77) directly shows that the fixed point corresponding to y1 = -1 is then always unstable. We therefore only investigate the fixed point corresponding to y1 = 1 hereafter, and we assume ∆ y1 is strictly positive, so that (77) is met. The overall condition (76)-(78) then becomes

2T + ∆ y1 > 0 (79) T + ∆ y1 < 2. ( 80
)
To use this condition, we then express T and ∆ y1 with respect to known quantities, i.e. the network weights or corresponding mixing coefficients, and the observations x 1 (n) and x 2 (n) (as opposed to the source signals, i.e. we do not use ( 20)-( 21) here). Using (61), we derive

∆ y1 = |l 11 l 22 | δ y1 (81)
with δ y1 defined in (62). T is then obtained by combining ( 74) and ( 58)-( 59), where we set y1 = 1 and we use (50)-( 52), ( 54) and ( 81). This yields

T = -Al 11 -Bl 22 + 2 (82)
with

A = Q 1 2β [-Q 2 x 1 (n) + Q 1 x 2 (n) + γ -sgn(l 11 l 22 ) δ y1 + 1 (83) B = Q 2 2α [-Q 2 x 1 (n) + Q 1 x 2 (n) -γ +sgn(l 11 l 22 ) δ y1 + 1. (84)
Inserting ( 81) and (82) in the stability condition (79)-(80), the latter condition becomes

|l 11 l 22 | δ y1 -2Al 11 -2Bl 22 + 4 > 0 ( 85
)
|l 11 l 22 | δ y1 -Al 11 -Bl 22 < 0. ( 86
)
It should be noted that A and B depend on l 11 and l 22 only because of their factor sgn(l 11 l 22 ). Moreover, δ y1 does not depend on l 11 and l 22 . When l 11 and l 22 are varied, but in such a way that the sign of l 11 l 22 remains constant, we have the following two phenomena, for given mixing parameters and observations: (i) A, B and δ y1 remain constant and (ii) the part of the (l 11 , l 22 ) plane for which condition (85)-( 86) is met may be defined as follows: it is the intersection of two sub-parts of the (l 11 , l 22 ) plane, each of these parts being bounded by a curve defined as:

2c 1 l 11 l 22 + 2c 2 l 11 + 2c 3 l 22 + c 4 = 0, (87)
where all c i are constants which depend on the mixing parameters and observations, and which are specific to each of the two sub-conditions (85) and (86). Each corresponding condition (87) defines an hyperbola [START_REF] Harris | Handbook of mathematics and computational science[END_REF] in the (l 11 , l 22 ) plane.

Limitations of our basic network

The stability condition (85)-( 86), is applicable to any values of l 11 and l 22 . It therefore especially applies to our basic network, which corresponds to (44) and hence to l 11 = 1 and l 22 = 1.

(88)

Inserting these values in (85)-(86) shows that this basic network yields a stable fixed point only for some values of the mixing coefficients and observed signals, and therefore of the source signals.

To clearly illustrate this phenomenon with an example, we consider the simple case

L 12 = 0, L 21 = 0, Q 1 = Q 2 = Q > 0. ( 89
)
Applying this condition and (88) to ( 20)-( 21), ( 63)-( 66) and ( 83)-(84) allows us to rewrite stability condition (85)-( 86). It may thus be easily shown that our basic network then only yields stability for a limited domain of source values, which consists of a strip in the (s 1 (n), s 2 (n)) plane, defined by

-s 1 (n) - 1 Q < s 2 (n) < -s 1 (n) + 3 Q . ( 90
)
3.4 A method for stabilizing our extended network

Analysis of stability condition

Condition (85)-( 86) is of high theoretical interest, because it completely defines the stability of the considered fixed point. However, it does not show easily if and how l 11 and l 22 may be selected in order to ensure stability at the considered fixed point for any given observed signal values. To address that topic, we introduce a method which consists in considering l 11 as the primary variable and l 22 as the secondary variable, and in expressing it as l 22 = λl 11 , where λ is a parameter. For any fixed λ, we first investigate whether there exist values of l 11 such that (85)-( 86) is met, in order to eventually determine if there exist values of l 11 and λ (and therefore l 22 ) such that (85)-( 86 For a given λ, (62) and ( 83)-(84) show that A and B do not depend on l 11 and l 22 . Eq. ( 91)-(92) then yield two inequalities with respect to l 11 , which are solved as follows. We first only derive the solution of (92). It is thus shown easily that, for (92), any (non-zero) λ is suitable and the solutions of (92) for a given λ are

l 11 = µ A + Bλ |λ| δ y1 with 0 < µ < µ max (93)
where µ max = 1. To solve (91)-(92) we then determine, for a given λ, which of the above solutions of (92) are also solutions of (91). We therefore insert the solutions l 11 defined in (93) into (91), which yields

(µ 2 -2µ) (A + Bλ) 2 |λ| δ y1 + 4 > 0 ( 94
)
where we only have to consider 0 < µ < 1, due to the above condition on µ in (93). Eq. ( 94) then defines all the allowed values of µ. By studying the variations of the left-hand term of (94) with respect to µ (for a given λ), it may then be shown that the overall solution of (91)-( 92) is still defined by ( 93), but now with µ max defined as follows. Denoting

C(λ) = (A + Bλ) 2 |λ| δ y1 (95)
we have

µ max = 1 if C(λ) ≤ 4 1 -1 -4 C(λ)
otherwise.

(96)

The above analysis shows that any value of λ yields a non-empty interval of solutions for l 11 . For a given λ, a simple and safe approach therefore consists in selecting for l 11 the value situated in the middle of the allowed range (93), i.e.

l 11 = µ max 2 A + Bλ |λ| δ y1 (97)
with µ max defined by (96). We should eventually propose a method for selecting λ. As stated above, any (non-zero) value of λ is acceptable. A simple solution is λ = ±1, which gives the same "weight" to l 11 and l 22 = λl 11 . Moreover, the sign of λ may be chosen as follows. As explained above, the considered fixed point corresponds to y1 = 1. Since l 22 = λl 11 in addition, (67) here reduces to

T 1 = sgn(λ) sgn(-αs 1 (n) + βs 2 (n) + γ). (98)
Therefore, if λ has the same sign as (-αs 1 (n) + βs 2 (n) + γ), then T 1 = 1, so that the considered stable fixed point yields the non-permuted sources of (70). Otherwise, the permuted sources of (71) are obtained. We thus guarantee that both solutions are obtained by successively applying our approach with two opposite values of λ. Moreover, if the source and mixing coefficients are such that the sign of (-αs 1 (n) + βs 2 (n) + γ) is known, then selecting λ also with that sign guarantees (local) convergence to the non-permuting point15 .

Summary of proposed method

Based on the above analysis, the following procedure is guaranteed to yield local convergence towards a separating point:

1. Select λ as explained above.

2. Set l 11 according to (97), taking into account sgn(l 11 l 22 ) = sgn(λ).

3. Set l 22 = λl 11 .

4. Set all other network parameters according to (50)-( 52) and ( 54)-(55).

Note that this procedure is applied independently at each time n, for different observed values. This yields different values of (l 11 , l 22 ) at each time n, so that when reaching the non-permuting separating point (70), the signals s i (n) are obtained with scale factors 1/l ii which vary with n. In most applications, this should be avoided, in order to obtain the sources s i (n) with constant scales, such as in s i (n) = a ii s i (n). This may be achieved by post-processing the outputs y i (n) of the network of Fig. 8, obtained for each time n after the convergence of the recurrence corresponding to m. Assuming this recurrence converges to the non-permuting separating point (70), this postprocessing consists in multiplying each output signal y i (n) by l ii = 1 -l ii , so as to obtain z i (n) = s i (n) (up to convergence errors). This approach may be extended to the other neural network structures introduced in this paper.

Test results

Tests with artificial sources

We first illustrate the performance of our approach, by considering linear-quadratic mixtures, i.e. observations defined by ( 20)- [START_REF] Harris | Handbook of mathematics and computational science[END_REF], at a single time n. We use s 1 (n) = -1, s 2 (n) = -2 and mixing coefficients defined by (89), with Q = 0.5. We first implement our basic network by running 10 52) and (54), starting from y 1 (1) = 0, y 2 (1) = 0. The resulting trajectory of (y 1 (m), y 2 (m)) is provided in Fig. 10. This shows that the network diverges very rapidly. This is in agreement with the fact that condition (90) is not met here. We then implement our extended network by running the recurrence (35)-(36) as above but with parameters selected as explained in Section 3.4.2 (with λ = 1). The network then converges to the solution (70), as shown in Fig. 11 (we here have l 11 = l 22 = 0.4). Moreover, it converges very rapidly. This shows the effectiveness of the proposed approach.

We then determine the statistical performance of our extended network. To this end, we generate 1000 random samples of each source with a uniform distribution in [-0.5, 0.5]. We mix these sources in the same conditions as above (which guarantees the stability of the non-permuting separating point for λ = 1). For each source vector s(n), we use the initialization y i (1) = x i (n) ∀i ∈ {1, 2}, and then run the recurrence (35)-(36) with parameters selected as above. At each step m of the recurrence performed for s(n), the root-mean-square (RMS) error of the network may be defined as

RM S(n, m) = 1 2 2 i=1 [y i (m)l ii -s i (n)] 2 . ( 99
)
The average accuracy, over all source vectors s(n), of our network after m steps of the recurrence is measured by the mean of RM S(n, m) over n. The evolution of this mean RMS error vs m is shown in Fig. 12. This confirms that our network yields very good accuracy after just a few recurrence steps.

Test with speech sources

Another experiment is performed with two speech signals s 1 (n) and s 2 (n), which are shown in Fig. 13. These source signals are again mixed using the linear-quadratic model ( 20)-( 21), with L 12 = 0.7, L 21 = 0.8, Q 1 = 0.8, Q 2 = 0.9, to generate the two observations x 1 (n) and x 2 (n) shown in Fig. 13. The distributions of the sources and of the observations are provided in Fig. 14.

For each sample n, a recurrent extended network with parameters selected as explained previously is implemented and initialized with y i (1) = x i (n) ∀i ∈ {1, 2}. Then, the recurrence (35)-( 36) is performed and the RMS error of the network is computed at each step m. The mean RMS error vs m is shown in Fig. 15 and confirms the good performance of the proposed method with real one-dimensional signals and a general choice of the mixture parameters.

Test with images

The last experiment concerns artificial mixtures of real images. The two photographic images provided by [START_REF]Web site of ICA Central[END_REF] and shown in Fig. 16 are vectorized, centered, and normalized to generate the two source vectors s 1 and s 2 containing 331 × 438 = 144978 samples. These sources are then mixed using the linear-quadratic model ( 20 used and the RMS error of the network is computed at each step m. The mean RMS error vs m is shown in Fig. 19 and confirms the good performance of the proposed method with images and another choice of the mixture parameters.

Conclusion

In this first part of our paper, we introduced recurrent source separation neural network structures applicable to a large class of new nonlinear mixtures. We analyzed in detail their versions dedicated to linear-quadratic mixtures, and we especially showed how to select their free weights in order to guarantee stability. We experimentally showed that these networks yield fast and accurate convergence for various types of artificial and real sources.

Let us stress again that an attractive feature of the proposed networks is that they are tailored to each considered type of mixtures: given the mixing model, we derive a corresponding separating structure which is such that its equilibrium point of interest exactly restores the source values, and the network converges towards this equilibrium point quickly and with the desired accuracy 16 , as stated above. This should be contrasted with several existing nonlinear optimization/approximation methods, which do not yield such attractive features. For instance, one might consider using a universal approximator (e.g. a feedforward multi-layer perceptron) in order to achieve an approximation of the inverse of the mixing mapping. However, a major drawback of such a structure with given parameter values is that, for a given arbitrary input, it can only provide an approximation of the desired output with bounded accuracy, instead of the exact desired output (i.e. actual source values) that we restore. Note also that our approaches are attractive because they do not require one to manually tune parameters, thanks to the automated procedure that we introduced above.

We plan to further develop the versions of the proposed networks suited to quadratic mixtures and to general polynomial mixtures, which may be considered as Taylor series approximations of general nonlinear models. While we defined the proposed structures for an arbitrary number of sources in this paper, we then did not analyze their properties for more than two sources. An extension of this analysis to any number of sources could be considered in future work, although this is likely to be a difficult task.

More specifically, in the second part of this paper, we will mainly 17 present adaptation algorithms for all these networks, in order to perform blind source separation: as compared to several existing nonlinear optimization methods, the approaches that we proposed are attractive because they yield a natural extension towards the blind configuration, by keeping their structure and introducing a training algorithm for adapting their weights.

Figure 1 :

 1 Figure 1: Loop of linear instantaneous recurrent network.

Figure 2 :

 2 Figure 2: Fixed point of loop of linear instantaneous recurrent network.

Figure 3 :

 3 Figure 3: Proposed recurrent network for basic additive-target mixtures.

Figure 4 :

 4 Figure 4: Proposed recurrent network for general additive-target mixtures, with adder outputs inverted before feedback.

Figure 6 :

 6 Figure 6: Proposed recurrent network for basic additive-target mixtures, with additional selffeedback.

Figure 7 :

 7 Figure 7: Proposed recurrent network for linear-quadratic mixtures: basic version, i.e. without self-feedback.

Figure 8 :

 8 Figure 8: Proposed recurrent network for linear-quadratic mixtures: extended version, i.e. with self-feedback.

Figure 9 :

 9 Figure 9: Proposed recurrent network for quadratic mixtures, with adder outputs inverted before feedback, without self-feedback.

  )-(55) thus first yield ∆ y1 = (l 11 l 22 ) 2 δ y1 (61)

  ) is met. In other words, we first determine the intersection of the part of the (l 11 , l 22 ) plane where (85)-(86) is met and of a given line in that plane, defined by l 22 = λl 11 (with λ = 0). Using the latter expression of l 22 , Eq. (85)-(86) become |λ| δ y1 (l 11 ) 2 -2(A + Bλ)l 11 + 4 > 0 (91) |λ| δ y1 (l 11 ) 2 -(A + Bλ)l 11 < 0. (92)

Figure 10 :

 10 Figure 10: Divergence of basic network.

Figure 11 :

 11 Figure 11: Convergence of extended network.

Figure 12 : 2 Figure 13 :

 12213 Figure 12: Mean of RMS error vs step m of recurrence.

  )-[START_REF] Harris | Handbook of mathematics and computational science[END_REF], with L 12 = -0.8, L 21 = 0.7, Q 1 = -0.4, Q 2 = 0.3, to generate the two observations x 1 and x 2 shown in Fig.17

2 Figure 14 :

 214 Figure 14: Distribution of the speech sources s 1 (n) and s 2 (n) (left) and of their mixtures x 1 (n) and x 2 (n) (right).

Figure 15 :

 15 Figure 15: Mean of RMS error vs step m of recurrence for experiment using real-world speech signals.

Figure 16 :

 16 Figure 16: Photographic real-world images used in our experiment.

Figure 17 :

 17 Figure 17: Mixtures of real-world images used in our experiment.

2 Figure 18 :

 218 Figure 18: Distribution of the sources s 1 (n) and s 2 (n) (left) and of the observations x 1 (n) and x 2 (n) (right).

Figure 19 :

 19 Figure 19: Mean of RMS error vs step m of recurrence for experiment using real-world images.

The dual normalization, i.e.s i (n) = a ij s j (n) ∀ i = j ∈ {1,

2}, may be used for this mixing model and for the other models subsequently considered in this paper. For the sake of brevity, this is not detailed hereafter.[START_REF] Berry | Algorithms and applications for approximate nonnegative matrix factorization[END_REF] These successive output values therefore also depend on n. This index n is omitted in the notations y i (m), in order to improve readability and to focus on the recurrence on outputs for given input values x 1 (n) and x 2 (n).[START_REF] Jutten | Advances in Blind Source Separation (BSS) and Independent Component Analysis (ICA) for Nonlinear Mixtures[END_REF] We use lower-case letters in notations for the weights of the separating network and upper-case letters for the parameters of the mixing model.

This model itself may be defined for arbitrary numbers P and N of observations and sources. However, as stated above, when subsequently developing corresponding separating structures, we will focus on determined mixtures, i.e. P = N .

This assumption is made for the sake of simplicity, i.e. it may even be avoided for specific mixtures, especially when each interfering term I i [s(n)] may be expressed with respect to all target terms T i [s i (n)] instead of all signals s i (n), and provided one only aims at extracting the target terms T i [s i (n)], not at subsequently retrieving the signals s i (n) from them.

Up to the minus sign that we introduced in[START_REF] Cichocki | Multi-layer neural networks with a local adaptive learning rule for blind separation of source signals[END_REF] to simplify the correspondence between mixing and separating parameters. This comment also applies to the mixing model considered in Section 2.2.3.

Again, these successive output values therefore also depend on n, but this index n is omitted in the notations.

One could consider different classes of functions for f and g in each way i of the mixing and separating stages. This case is however also contained in the description provided here, by considering a single but general couple of functions f and g, which each define a class of functions which includes all sub-classes considered in each way i.

Note that various specific ETM may alternatively be transformed into ATM (of transformed signals) by preprocessing the observations. For instance, one may take their logarithms for MTM.

This will appear clearly below for linear-quadratic mixtures, by comparing the resolution of the fixed-point equations (56)-(57) to the inversion of the mixing equations (17)-[START_REF] Deville | Méthode de séparation de sources pour mélanges linéaires-quadratiques[END_REF].

On the contrary, this is not guaranteed for arbitrary network weights.

That approximation therefore only allows us to analyze the local convergence of the system at the considered point, as opposed to its global convergence from any initial state.

Otherwise, this version of our approach may yield permutation, and this could be further investigated.

Acknowledgment

Y. Deville would like to thank Y. Naudet for drawing his attention to linear-quadratic mixtures. This first resulted in the basic network presented in [18]- [20], and this investigation then evolved towards the much more general approach introduced in this paper.