Student Member, IEEE Jin Danqi
email: danqijin@mail.nwpu.edu.cn

Senior Member, IEEE Jie Chen
email: dr.jie.chen@ieee.org

Senior Member, IEEE Jingdong Chen
email: jingdongchen@ieee.org

Fellow, IEEE Ali H Sayed
email: ali.sayed@epfl.ch

D Jin

J Chen

Affine Combination of Diffusion Strategies over Networks

Keywords: Distributed optimization, diffusion strategy, affine combination, adaptive fusion strategy, stochastic performance. I

Diffusion adaptation is a powerful strategy for distributed estimation and learning over networks. Motivated by the concept of combining adaptive filters, this work proposes a combination framework that aggregates the operation of multiple diffusion strategies for enhanced performance. By assigning a combination coefficient to each node, and using an adaptation mechanism to minimize the network error, we obtain a combined diffusion strategy that benefits from the best characteristics of all component strategies simultaneously in terms of excess-mean-square error (EMSE). Analyses of the universality are provided to show the superior performance of affine combination scheme and to characterize its behavior in the mean and mean-square sense. Simulation results are presented to demonstrate the effectiveness of the proposed strategies, as well as the accuracy of theoretical findings.

is possible to combine two distributed strategies and obtain a new strategy whose performance is superior to its individual components. The question was answered in the affirmative for stand-alone adaptive filters in [START_REF] Arenas-Garcia | Mean-square performance of a convex combination of two adaptive filters[END_REF]- [START_REF] Kozat | Steady-state MSE performance analysis of mixture approaches to adaptive filtering[END_REF]. It is more challenging in the context of adaptive networks with a multitude of interacting agents over a graph. We will show nevertheless that this is still possible.

Combination strategies have been successfully used for classical adaptive filters [START_REF] Arenas-Garcia | Mean-square performance of a convex combination of two adaptive filters[END_REF], [START_REF] Arenas-Garcia | Plant identification via adaptive combination of transversal filters[END_REF], multi-kernel learning [START_REF] Alain | Simple MKL[END_REF], as well as modern deep neural network structures [START_REF] Szegedy | Going deeper with convolutions[END_REF]. It is shown in some of these works that using convex combinations [START_REF] Arenas-Garcia | Mean-square performance of a convex combination of two adaptive filters[END_REF], [START_REF] Arenas-Garcia | Plant identification via adaptive combination of transversal filters[END_REF] or affine combinations [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF] of adaptive filters with diversity can lead to filters that combine the advantages of all component filters. Generally, such combination schemes are used to facilitate the selection of filter parameters, to increase robustness against an unknown environment, or to possibly enhance performance beyond the range of each component [START_REF] Arenas-Garcia | Combinations of adaptive filters: Performance and convergence properties[END_REF]. In this paper, we propose affine combination schemes for diffusion strategies. Each agent is designed to run several diffusion strategies in parallel, and to combine their estimates to generate the final estimates. Time-varying affine combination coefficients are set by minimizing the overall squared instantaneous error. Simulation results show that the proposed algorithms endow the networks with a significantly enhanced performance in the learning process. Some related works can be found in [START_REF] Fernandez-Bes | Distributed estimation in diffusion networks using affine least-squares combiners[END_REF] and our previous work [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF]. In [START_REF] Fernandez-Bes | Distributed estimation in diffusion networks using affine least-squares combiners[END_REF], the authors use a useful convex combination scheme for combining two specific fusion strategies albeit without motivating it or formulating a driving optimization problem. Though similar to [START_REF] Fernandez-Bes | Distributed estimation in diffusion networks using affine least-squares combiners[END_REF], in [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF] a convex combination scheme is proposed for two diffusion strategies. However, both works do not examine the theoretical underpinnings of the algorithms. We should also note that the work proposed here is different from the useful formulation in [START_REF] Fernandez-Bes | Adaptive diffusion schemes for heterogeneous networks[END_REF]. This last reference introduces a diffusion scheme for networks with heterogeneous nodes, namely, nodes implementing different adaptive rules or differing in other aspects such as filter structure, length or step-size. In comparison, our work proposes affine combination schemes to combine different diffusion strategies, which can be either homogeneous or heterogeneous. In that sense, the results presented here are more general than [START_REF] Fernandez-Bes | Adaptive diffusion schemes for heterogeneous networks[END_REF].

The main contributions of this work are summarized as follows:

• We introduce a framework for the affine combination of two component diffusion algorithms over networks, and propose two methods for adjusting the combination weights. The framework can be extended to multiple component algorithms as explained in Appendix A.

• We establish the universality of the combined strategy, in a manner that extends a prior universality analysis for the combination of adaptive filters.

• We conduct a theoretical analysis of the performance of combined diffusion LMS strategies under some typical simplifying assumptions and approximations. Although the assumptions are not accurate in general, they are nevertheless typical in the context of studying adaptive systems and tend to lead to performance results that match well with practice for sufficiently small step-sizes with white measurement noise and white regressors [START_REF] Sayed | Adaptive Filters[END_REF].

Notation. Normal font x denotes scalars. Boldface small letters x and capital letters X denote column vectors and matrices, respectively. The superscript (•) denotes the transpose operator. The inverse of a square matrix is denoted by (•)

-1 . The mathematical expectation is denoted by E{•}. The Gaussian distribution with mean µ and variance σ 2 is denoted by N (µ, σ 2 ).

The operators min{•} and max{•} return the minimal or maximal value of their arguments. The operator diag{•} extracts the diagonal elements of its matrix argument, or generates a diagonal matrix from its vector argument. The operator | • | returns the absolute value of its argument. I N and 0 N denote identity matrix and zero matrix of size N × N , respectively. All-one vector of length N is denoted by 1 N . Symbol C i stands for cluster i, i.e., index set of nodes in the i-th cluster. N k denotes the neighbors of node k, including k.

II. NETWORK MODEL AND DIFFUSION LMS

A. Network Model

Consider a connected network consisting of N agents. The problem is to estimate an unknown parameter vector w k of length L × 1 at each agent k. Agent k has access to temporal measurement sequences {d k,n , x k,n }, where d k,n denotes a reference signal, and x k,n is an L × 1 regression vector with positive-definite covariance matrix. The data at agent k and time instant n are driven by the linear model:

d k,n = x k,n w k + z k,n , (1)
where z k,n is an additive noise. For ease of derivation, we introduce the following assumption A11 for z k,n . Although rarely true in practice, A1 simplifies the derivation of algorithm and theoretical analysis, thus it is widely adopted in the context of online learning and adaptation [START_REF] Sayed | Diffusion adaptation over networks[END_REF], [START_REF] Sayed | Adaptive networks[END_REF].

A1:

The additive noise z k,n is zero-mean, stationary, independent and identically distributed (i.i.d.) with variance σ 2 z,k , and independent of any other signal.

To determine the unknown parameter vector w k , we consider the following mean-square-error (MSE) cost at agent k:

J k (w) = E |d k,n -x k,n w| 2 .
(

) 2
Observe in (1) that J k (w) is minimized at w k . For single-task problems, each agent in the network estimates the same parameter vector, while for multi-task problems, agents may estimate distinct parameter vectors [START_REF] Chen | Multitask diffusion adaptation over networks[END_REF].

B. Diffusion LMS Algorithm

The diffusion LMS algorithm is derived to minimize the following aggregate cost function:

J glob (w) = N k=1 J k (w) (3)
in a cooperative manner. The general structure of the diffusion LMS algorithm consists of the following steps:

φ k,n = ∈N k a 1, k w ,n , (4)
ψ k,n+1 = φ k,n + µ k ∈N k c k x ,n (d ,n -x ,n φ k,n ), (5)
w k,n+1 = ∈N k a 2, k ψ ,n+1 , (6)
where w k,n+1 , ψ k,n+1 and φ k,n are estimates for the unknown parameter vectors w k obtained at different operating stages of the diffusion algorithm, with subscripts k and n denoting node index and time index, respectively, and µ k > 0 is the step-size

A 1 1 N = 1 N , A 2 1 N = 1 N , C1 N = 1 N , (7)
a 1, k = 0, a 2, k = 0, c k = 0 if / ∈ N k . (8)
Several adaptive strategies can be obtained as special cases of ( 4) to [START_REF] Sayed | Adaptive networks[END_REF]. Two popular strategies, namely, adapt-then-combine (ATC) and combine-then-adapt (CTA), can be achieved by setting A 1 = I N and A 2 = I N respectively.

III. GENERAL COMBINATION FRAMEWORK

A. Combination framework

Combining two diffusion strategies can be performed with two concurrent adaptive layers: a diffusion strategy layer and a combination layer. The diffusion strategy layer consists of a distributed network running two distinct diffusion strategies, say S (1) and S (2) , individually and simultaneously, with associated quantities denoted by

A (i) 1 , A (i)
2 , C (i) , µ (i) , where the superscript (i) denotes the i-th component diffusion strategy S (i) with i = 1, 2. The scheme is illustrated in Fig. 1. In S (1) and S (2) , the agents have access to identical input and reference signals, and produce each an individual estimate of the optimal weight vector. We associate the combination coefficients γ k,n and 1γ k,n , to S (1) and S (2) , respectively, at each agent k and time instant n. The goal of the combination layer is to learn which diffusion strategy performs better at each time instant, by adjusting γ k,n in order to optimize the overall network performance.

For each S (i) , we define the filter output y

y (i) k,n x k,n w (i) k,n , (9) e
(i) k,n d k,n -x k,n w (i) k,n , (10) e
(i) k,n x k,n (w k -w (i) k,n ). (11)
By combining the estimates of two diffusion strategies S (1) and S (2) at each agent k with coefficients γ k,n and 1γ k,n , we arrive at:

w k,n γ k,n w (1) k,n + (1 -γ k,n )w (2) k,n , (12)
y k,n = γ k,n y (1) k,n + (1 -γ k,n )y (2) k,n , (13)
e k,n = γ k,n e (1)
k,n + (1 -γ k,n )e (2) k,n , (14)
e k,n = γ k,n e (1)
k,n + (1 -γ k,n ) e (2) k,n . (15)
The problem then becomes one of deriving a strategy for adjusting γ k,n based on the minimum mean-square-error (MMSE)

criterion. Convex combination schemes require that γ k,n ∈ [0, 1].
There is no such constraint in affine combination schemes.

It is sufficient in this work to consider affine combination schemes to convey the main ideas.

B. Affine combination schemes

The MSE at time instant n of the entire network at the output of the combination layer is defined by:

J MSE n = 1 2 N k=1 E e 2 k,n . (16)
We suggest to adjust γ k,n by minimizing [START_REF] Kozat | Steady-state MSE performance analysis of mixture approaches to adaptive filtering[END_REF]. Using [START_REF] Arenas-Garcia | Mean-square performance of a convex combination of two adaptive filters[END_REF], setting the derivative of J MSE n with respect to γ k,n to zero, and using the relations:

e (i) k,n = e (i) k,n + z k,n (17)
E{z k,n } = 0, (18)
we obtain the optimal value of γ k,n in the MMSE sense:

γ k,n = E ( e (2)
k,n ) 2 -E e (1) k,n e (2) k,n E ( e (1)
k,n ) 2 +E ( e (2)
k,n ) 2 -2E e (1) k,n e (2) k,n . (19)
However, it is not possible to evaluate γ k,n with [START_REF] Szegedy | Going deeper with convolutions[END_REF] as it requires knowledge of the second-order moments of e

(i)
k,n . We address this problem by introducing adaptive strategies.

1) Affine power-normalized scheme: Using the gradient descent method to minimize [START_REF] Kozat | Steady-state MSE performance analysis of mixture approaches to adaptive filtering[END_REF], and approximating the expectation terms with their instantaneous values, yield the following affine power-normalized LMS iteration [START_REF] Azpicueta-Ruiz | A normalized adaptation scheme for the convex combination of two adaptive filters[END_REF]:

γ k,n+1 = γ k,n - ν γ k ε + p k,n ∂J MSE n ∂γ k,n ≈ γ k,n + ν γ k ε + p k,n e k,n x k,n w (1) k,n -w (2) k,n , (20)
where ε is a small positive parameter, ν γ k is a positive step-size, and p k,n is a low-pass filtered estimate of the power of

x k,n w (1) k,n -w (2)
k,n given by:

p k,n = η p k,n-1 + (1 -η) x k,n (w (1) k,n -w (2) k,n ) 2 , (21)
with 0 < η < 1 a temporal smoothing factor.

2) Affine sign-regressor scheme: Alternatively, we can adopt another normalization scheme for the step-size, leading to the affine sign-regressor LMS iteration [START_REF] Candido | Transient and steady-state analysis of the affine combination of two adaptive filters[END_REF]:

γ k,n+1 = γ k,n -ν γ k ∂J MSE n ∂γ k,n ≈ γ k,n + ν γ k e k,n sgn x k,n w (1) k,n -w (2) k,n (22)
where sgn{x} is the sign function and

ν γ k = νγ k |x k,n (w (1)
k,n -w

(2) k,n )| .
IV. THEORETICAL ANALYSIS OF POWER-NORMALIZED SCHEME

A. Universality at steady state

We first illustrate the universality of the power-normalized scheme at steady state. In other words, we show that the algorithm results in a combined strategy tracks the best performance of each component strategy.

We start the analysis by defining several quantities to be used later. The EMSE at the output of the combination layer, and the EMSE of each component strategy, at node k and time instant n are defined as:

J ex,k,n E{( e k,n ) 2 } = E γ k,n e (1)
k,n + (1 -γ k,n ) e (2) k,n 2 , (23) J (i) ex,k,n E{( e (i)
k,n ) 2 }. (24)
Correspondingly, the EMSE of the whole network, at the output of the combination layer and for each component strategy, are defined by:

J ex,net,n N k=1 J ex,k,n (25)
J (i) ex,net,n N k=1 J (i) ex,k,n (26)
respectively. By taking the limit as n → ∞, we obtain the corresponding values at steady-state: J ex,k,∞ , J

ex,k,∞ , J ex,net,∞ and J (i) ex,net,∞ . In addition, by resorting to [START_REF] Li | Distributed adaptive estimation based on the APA algorithm over diffusion networks with changing topology[END_REF] and ( 15), we have:

x k,n w (1) k,n -w (2) k,n = e (2) k,n -e (1) k,n (27)
and

e k,n = γ k,n e (1) k,n + (1 -γ k,n ) e (2) k,n + z k,n . (28)
Now we introduce some approximations to be used later to simplify the theoretical analysis:

Ap 12 : At steady state, the combination coefficient γ k,n is statistically independent of e

(i)
k,n and p k,n . Ap 2 : For a sufficiently large temporal smoothing factor η, p k,n is statistically independent of x k,n w

(i) k,n , that is, of e (i) k,n .
As indicated in [START_REF] Arenas-Garcia | Mean-square performance of a convex combination of two adaptive filters[END_REF], approximation Ap 1 is reasonable when adopting a decaying step-size ν γ k , and Ap 2 is justified when using a large temporal smoothing factor η. With approximations Ap 1 and Ap 2 , we obtain the following results.

Universality Analysis Result 1: Assume data model [START_REF] Braca | Enforcing consensus while monitoring the environment in wireless sensor networks[END_REF], assumption A1 and approximations Ap 1 , Ap 2 hold. Then for any initial conditions with step-size ν γ k ensuring the stability of power-normalized scheme, the distributed diffusion network with [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF] is universal at steady state, which means that the EMSE of the diffusion network after combination cannot be worse than that of the best component strategies, with

J ex,net,∞ ≤ min J (1) ex,net,∞ , J (2) ex,net,∞ . (29)
Proof: See Appendix B.

B. Mean weight and mean-square behaviors analyses

We shall now examine the mean and mean-square error behavior of the power-normalized scheme on the basis of the two diffusion LMS strategies referred to in Section II-B. Collecting the quantities from the network into block vectors, we have:

w col{w 1 , • • • , w N }, (30)
w n col{w 1,n , • • • , w N,n }, (31)
w (i) n col{w (i) 1,n , • • • , w (i) N,n }, ( 32
)
where w is the block optimum weight vector, w n and w

(i)
n are the block weight estimates of the combination layer and component diffusion strategies S (i) , respectively. Using [START_REF] Nascimento | A transient analysis for the convex combination of adaptive filters[END_REF], [START_REF] Silva | A transient analysis for the convex combination of two adaptive filters with transfer of coefficients[END_REF], and from ( 12) we arrive at:

w n = Γ n w (1) n + (I N L -Γ n )w (2) n , (33)
where Γ n is a diagonal weighting matrix defined by

Γ n diag{γ 1,n , • • • , γ N,n } ⊗ I L , (34)
with symbol ⊗ denoting Kronecker product. The weight error vectors at node k for the component diffusion strategies S (i) and for the combination layer, as well as those of the entire network, are defined by:

v (i) k,n w (i) k,n -w k , (35)
v k,n w k,n -w k , (36)
v (i) n col{v (i) 1,n , • • • , v (i) N,n }, (37)
v n col{v 1,n , • • • , v N,n }. (38)
Using ( 30)- [START_REF] Bertrand | Diffusion bias-compensated RLS estimation over adaptive networks[END_REF], vector v n is given by:

v n = w n -w = Γ n v (1) n + (I N L -Γ n )v (2) n . (39)
To make the theoretical analysis tractable, we introduce the following assumption A2 and approximation Ap 3 :

A2 (Independent Regressors): The regression vector x k,n , generated from a zero-mean random process, is temporally stationary, white (over n) and spatially independent (over k) with covariance matrix R x,k = E{x k,n x k,n } > 0.

Ap 3 : At each time instant n, γ k,n is statistically independent of w (i)
k,n for i = 1, 2. Although not true in general, assumption A2 is usually adopted to simplify the derivation without constraining the conclusions.

Besides, there are several results in the literature showing that performance results obtained under A2 match well with actual performance when the step-sizes µ (i) k of the component diffusion strategies are sufficiently small [START_REF] Sayed | Adaptive Filters[END_REF], [START_REF] Sayed | Adaptation, Learning, and Optimization over Networks[END_REF]. Ap 3 makes the theoretical analysis tractable. Though actually not true, it does not notably affect the theoretical results, as illustrated by the simulation results.

1) Mean weight behavior analysis

For the combination layer, taking expectation of (39) and using approximation Ap 3 , we arrive at:

E v n+1 =E Γ n+1 E v (1) n+1 +E I N L -Γ n+1 E v (2) n+1 . (40)
We now need to evaluate E v (i)

n+1 . Under assumptions A1-A2, and following the derivation from [START_REF] Chen | Diffusion LMS over multitask networks[END_REF], we have:

v (i) n+1 = B (i) n v (i) n + g (i) n -r (i) n , (41)
E v (i) n+1 = B (i) E v (i) n -r (i) , (42)
with

B (i) n = A A A (i) 2 I N L -U (i) H (i) n A A A (i) 1 , (43)
B (i) = A A A (i) 2 I N L -U (i) H (i) A A A (i) 1 , (44)
A A A (i) j = A (i) j ⊗ I L , ∀j = 1, 2, (45)
U (i) = diag µ (i) 1 , • • • , µ (i) N ⊗ I L , (46)
H (i) n = diag ∈N k c (i) k x ,n x ,n N k=1 , (47)
H (i) = diag R (i) 1 , • • • , R (i) N , (48)
R (i) k ∈N k c (i) k R x, , (49)
g (i) n = A A A (i) 2 U (i) p (i) zx,n , (50)
p (i) zx,n = col ∈N k c (i) k x ,n z ,n N k=1 , (51)
h (i) u,n = col ∈N k c (i) k x ,n x ,n w k -w N k=1 , (52)
r (i) n A A A (i) 2
U (i) h (i) u,n r (i) u,n - A A A (i) 2 I N L -U (i) H (i) n A A A (i) 1 -I N L +(A A A (i) 2 -I N L ) w r (i) w,n (53)
r (i) E{r (i) n } = r (i) u -r (i) w , (54)
where the symbols with an overhead bar denote the expectation of the corresponding quantities with subscript n.

Stability Analysis Result 1: (Stability in the mean) Assume data model ( 1), A1-A2 and Ap 3 hold. Then, for any initial condition, the distributed network with power-normalized diffusion scheme [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF] asymptotically converges in the mean if the step-sizes are chosen to satisfy:

0 < µ (i) k < 2 λ max R (i) k , k = 1, • • • , N and i = 1, 2, (55)
where λ max {•} denotes the largest eigenvalue of its matrix argument, and if

0 < ν γ k < 1 -η, ( 56
)
where η is the temporal smoothing factor used in [START_REF] Fernandez-Bes | Distributed estimation in diffusion networks using affine least-squares combiners[END_REF]. The asymptotic bias is given by:

E v ∞ = -Γ ∞ I N L -B (1) -1 r (1)
-

(I N L -Γ ∞ ) I N L -B (2) -1 r (2) . ( 57
)
Proof: See Appendix C.

2) Mean-square behavior analysis

To perform the mean-square behavior analysis, we introduce the following approximation.

Ap 4 : At each time instant n, Γ n+1 is statistically independent of B (i) n , v (i) n , g (i)
n and r

(i) n in (41) for i = 1, 2.
Although this approximation is not true in general, it will make the analysis tractable.

We shall now evaluate the evolution of E v n+1 2 Σ over time, where Σ is an arbitrary positive semi-definite matrix, and x 2

Σ

x Σx. Using (39), we have:

E v n+1 2 Σ = 2E v (1) n+1 Γ n+1 Σ(I N L -Γ n+1 )v (2) n+1 +E (I N L -Γ n+1 )v (2) n+1 2 Σ + E Γ n+1 v (1) n+1 2 Σ . ( 58
) Let Σ (1) n+1 E Γ n+1 ΣΓ n+1 , (59)
Σ (2) n+1 E (I N L -Γ n+1 ) Σ (I N L -Γ n+1 ) , (60)
σ (i) n+1 = vec Σ (i) n+1 , (61)
where vec{•} operator stacks the columns of its matrix argument on top of each other. Under Ap 3 , the last two terms on the RHS of (58) can be written in compact form as E v

(i) n+1 2
Σ (i) n+1 with i = 1, 2.
They can be evaluated compactly as:

E v (i) n+1 2 σ (i) n+1 = E v (i) n 2 K (i) σ (i) n+1 + vec{G (i) } σ (i) n+1 + f r (i) , Σ (i) n+1 , E v (i) n , (62)
where

• 2 Σ (i) n+1 and • 2 σ (i) n+1
are used interchangeably, with:

f (r (i) n , Σ (i) n+1 , v (i) n ) r (i) n 2 Σ (i) n+1 -2r (i) n Σ (i) n+1 B (i) n v (i) n (63)
K (i) ≈ B (i) ⊗ B (i) (64)
G (i) E g (i) n g (i) n . ( 65
)
The derivation of equation ( 62) is provided in Appendix D.

For the first term on RHS of (58), since it has a similar structure to (107), the analysis is omitted. Using [START_REF] Sohrab | Basic Real Analysis[END_REF], following the routine (107)-( 112), and ignoring second-order terms in the step-size, we finally arrive at: 1) , B (2) ,

E v (1) n+1 Σ x,n+1 v (2) n+1 = E v (1) n Σ xc,n+1 v (2) n + vec{G x } σ x,n+1 + f x r (1) , r (2) , Σ x,n+1 , E v (1) n , E v (2) n , B (
where 1) , B

f x r (1) , r (2) , Σ x,n+1 , E v (1) n , E v (2) n , B (
r (1) Σ x,n+1 r (2) -E v (1) n B (1) Σ x,n+1 r (2) -r (1) Σ x,n+1 B (2) E v (2) n ( (2)
) 67
with

Σ x,n+1 E Γ n+1 Σ (I N L -Γ n+1 ) , (68)
Σ xc,n+1 vec -1 K x σ x,n+1 , (69)
σ x,n+1 vec Σ x,n+1 , (70)
K x ≈ B (2) ⊗ B (1) , (71)
G x E g (2) n g (1) n , (72)
and vec -1 {•} is the inverse vectorization operator.

Finally, using (62) for i = 1, 2 and (66) in (58), we obtain the explicit expression of the weighted mean-square behavior of the power-normalized diffusion scheme. k for i = 1, 2 are sufficiently small such that condition (55) is satisfied and approximations (64), ( 71), (112) are justified by ignoring higher powers of the step-size. Furthermore, assume that the step-size ν γ k at the combination layer satisfies the condition:

Stability Analysis

0 < ν γ k < 1 -η 3 ( 73
)
to ensure mean-square stability of the power-normalized scheme. Then, for any initial conditions, the distributed network with doubly stochastic matrices

A (i) 1 , A (i)
2 , of which both columns and rows add up to one, and scheme (20

) is mean-square stable if ρ I N L -U (i) H (i) < 1,
which is further guaranteed for sufficiently small step-sizes that also satisfy condition (55).

Proof: See Appendix E.

Remark 1. (Transient MSD):

We shall adopt the mean-square-deviation (MSD) learning curve of the entire network, defined by

ξ n+1 E v n+1 2 1 N I N L
, as a metric to evaluate the performance of diffusion networks. We observe that although the weighting matrix Σ of E v n+1 2 Σ in (58) may be constant over time, such as choosing Σ = 1 N I N L in evaluating the MSD, matrices Σ (i) n+1 and Σ x,n+1 become time-variant since Γ n+1 varies over time. Thus, it is almost impossible to derive a recursion to relate

E v n+1 2 Σ and E v n 2 Σ directly. To evaluate E v n+1 2 Σ , we must evaluate (62) and (66), in which E v (i) n 2 K (i) σ (i) n+1 and E v (1) n Σ xc,n+1 v (2) n
are calculated iteratively as shown by expressions (121) and (123) in Appendix F, while the remaining terms can be evaluated directly.

Remark 2. (Steady-state MSD):For sufficiently small step-sizes satisfying conditions (55) and (73) to ensure the mean and mean-square stabilities of the power-normalized diffusion scheme, we can obtain the explicit expression of the steady-state MSD as shown by expression (132) in Appendix G.

Proof: See Appendix G.

C. Mean and mean-square behaviors of γ k,n

In order to evaluate the mean and mean-square behavior of the power-normalized diffusion scheme, we must determine the mean and mean-square behavior of Γ n at the combination layer, which are obtained by evaluating those of γ k,n since Γ n is diagonal. To make the analysis tractable, we introduce the following approximations. k,n are jointly Gaussian with zero-mean, which implies [START_REF] Papoulis | Probability, Random Variables, and Stochastic Processes[END_REF]:

E ( e (i) k,n ) 4 = 3 J (i) ex,k,n 2 , ∀ i = 1, 2, (74) E ( e (1) k,n ) 3 ( e (2)
k,n ) 1 = 3J (1) ex,k,n J (1,2) ex,k,n , (75)
E ( e (1) k,n ) 1 ( e (2)
k,n ) 3 = 3J (1,2) ex,k,n J (2) ex,k,n , (76)
E ( e (1)
k,n ) 2 ( e (2)
k,n ) 2 = 2 J (1,2) ex,k,n 2 + J (1) ex,k,n J (2) ex,k,n . (77)
Approximation Ap 5 is commonly adopted in transient analysis of affine combinations of two adaptive filters [START_REF] Candido | Transient and steady-state analysis of the affine combination of two adaptive filters[END_REF], which also coincides with simulation result that γ k,n converges slowly compared to the variations of input signal x k,n , thus to the variations of a priori errors e (i) k,n . Although not true in general, Ap 6 makes the analysis tractable. Although may be violated in general, assumption Ap 7 is frequently adopted to facilitate the transient analysis of adaptive filters [START_REF] Sayed | Adaptive networks[END_REF], [START_REF] Candido | Transient and steady-state analysis of the affine combination of two adaptive filters[END_REF], [START_REF] Chen | Transient performance analysis of zero-attracting LMS[END_REF]- [START_REF] Jin | Model-driven online parameter adjustment for zero-attracting LMS[END_REF]. It becomes more reasonable for small step-sizes and long filters [START_REF] Sayed | Adaptive networks[END_REF].

Starting from [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF], and following the derivation in Appendix H, we arrive at the following stability analysis results.

Stability Analysis Result 3: (Stability in the Mean) Assume data model [START_REF] Braca | Enforcing consensus while monitoring the environment in wireless sensor networks[END_REF], assumption A1 and approximations Ap 2 , Ap 5 , Ap 6 hold. Then for any initial conditions, the power-normalized scheme [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF] asymptotically converges in the mean if the step-sizes ν γ k are chosen to satisfy condition (56). Besides, the mean behavior of γ k,n is evaluated as (136) in Appendix H, with steady-state value E{γ k,∞ } given by equation (95).

Proof: See Appendix H.

Stability Analysis Result 4: (Mean-square Stability) Assume data model [START_REF] Braca | Enforcing consensus while monitoring the environment in wireless sensor networks[END_REF], assumption A1 and approximations Ap 2 , Ap 5 -Ap 7 hold. Then for any initial conditions, the power-normalized scheme [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF] is mean-square stable if the step-sizes ν γ k are chosen to satisfy condition (73). Besides, the transient mean-square behavior of γ k,n is evaluated by (141) in Appendix H, with steady-state value E{γ 2 k,∞ } given by equation (78).

Proof: See Appendix H. E{γ 2 k,∞ }= ν∞ (J (2) ex,k,∞ +σ 2 z,k ) J (1) k,∞ + J (2) k,∞ +2ν ∞ ( J (2) k,∞ ) 2 +2E{γ k,∞ } J (2) k,∞ -3ν ∞ J (2) k,∞ ( J (1) k,∞ + J (2) k,∞ ) 2 J (1) k,∞ + J (2) k,∞ -3ν ∞ J (1) k,∞ + J (2) k,∞ 2 (78)
V. THEORETICAL ANALYSIS OF SIGN-REGRESSOR SCHEME By following the same routine as in Section IV, we conduct theoretical analysis for the sign-regressor diffusion scheme.

A. Universality at steady state Taking expectation of ( 22), and using ( 27), [START_REF] Papoulis | Probability, Random Variables, and Stochastic Processes[END_REF], we obtain:

E{γ k,n+1 } = E γ k,n + ν γ k E e (2)
k,n sgn e

(2)

k,n -e (1) k,n -γ k,n e (2)
k,ne

(1)

k,n . (79)
From (79), the stationary point of γ k,n is reached if

E e (2)
k,n sgn e

(2)

k,n -e (1) k,n -γ k,n e (2)
k,ne

(1)

k,n = 0. ( 80
)
Using the independence approximation Ap 1 , (80) gives a closed-form solution of E{γ k,n } at steady-state as:

E{γ k,∞ } = E e (2) k,∞ sgn e (2) k,∞ -e (1) k,∞ E e (2)
k,∞ -e (1) k,∞ . (81)
We further resort to the joint Gaussian approximation Ap 7 , which leads to approximations (148) and (149) further ahead in Section V-C. Using ( 148) and ( 149), (81) simplifies to (95). With (95), the proof of universality for the sign-regressor diffusion scheme at steady state is identical to that for power-normalized diffusion scheme.

B. Mean weight and mean-square behaviors analyses

Since there is almost no difference in the mean weight and mean-square behaviors of the sign-regressor and power-normalized schemes, we only provide the main conclusions here. Under assumptions A1-A2, approximations Ap 3 -Ap 4 and following the same routine as that in Section IV-B, the mean and mean-square behaviors of the weight error vector v n+1 at combination layer of the sign-regressor diffusion scheme are given by ( 40) and (58), respectively.

Stability Analysis Result 5: (Stability in the Mean) Assume data model (1), A1-A2 and Ap 3 hold. Then for any initial conditions, the distributed network with sign-regressor diffusion scheme [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF] asymptotically converges in the mean if the step-sizes of network are chosen to satisfy (55), and if the step-sizes at the combination layer are chosen to satisfy

0 < ν γ k < π 2 • max n J (1) k,n + J (2) k,n . (82)
The asymptotic bias is given by (57). k for i = 1, 2 are sufficiently small such that condition (55) is satisfied and approximations (64), ( 71), (112) hold. Furthermore, assume that the step-sizes ν γ k at the combination layer satisfy condition to ensure the mean-square stability of the sign-regressor diffusion scheme. Then for any initial conditions, the distributed network with doubly stochastic matrices

Stability Analysis

0 < ν γ k < 2 π max n J (1) k,n + J (2) k,n (83)
E{γ 2 k,∞ } = ν γ k J (2) ex,k,∞ + σ 2 z,k + 2E{γ k,∞ } J (2) k,∞ 2 π J (1) k,∞ + J (2) k,∞ -ν γ k J (2) k,∞ 8 J (1) k,∞ + J (2) k,∞ /π -ν γ k J (1) k,∞ + J (2) k,∞ (84)
A (i) 1 , A (i)
2 and scheme ( 22) is mean-square stable if ρ

I N L -U (i) H (i) < 1, which
is further guaranteed by sufficiently small step-sizes that also satisfy condition (55).

C. Mean and mean-square behaviors of γ k,n

Starting from [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF], and following the derivation in Appendix I, we arrive at the following stability analysis result.

Stability Analysis Result 7: (Stability in the Mean) Assume data model ( 1) and A1, Ap 5 , Ap 7 hold. Then for any initial conditions, the sign-regressor scheme [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF] asymptotically converges in the mean if the step-sizes ν γ k are chosen to satisfy condition (82). Besides, the mean behavior of γ k,n is evaluated by (150) in Appendix I, with steady-state value E{γ k,∞ } given by equation (95).

Proof: See Appendix I.

Stability Analysis Result 8: (Mean-square Stability) Assume data model (1) and A1, Ap 5 , Ap 7 hold. Then for any initial conditions, the sign-regressor scheme ( 22) is mean-square stable if the step-sizes ν γ k are chosen to satisfy condition (83).

Besides, the transient mean-square behavior of γ k,n is evaluated by (154) in Appendix I, with steady-state value E{γ 2 k,∞ } given by equation (84).

Proof: See Appendix I.

VI. SIMULATION RESULTS

In this section, we present simulation results to illustrate the proposed combination schemes and theoretical results. All simulated curves were averaged over 100 Monte Carlo runs.

A. Affine combination schemes validation

Consider a non-stationary system identification scenario with w k varying over time. The distributed network consisted of N = 10 nodes with connection topology depicted in Fig. 2(a). The regressors were generated from a multivariate Gaussian distribution with zero-mean and covariance matrix R x,k = σ 2

x,k I 50 . The noise signals were generated from Gaussian distribution N (0, σ 2 z,k ). Variances σ 2

x,k and σ 2 z,k at each agent were generated randomly as depicted in Fig. 2(b). 1 were set to the identity matrix. As a result, vector φ k,n coincides with w k,n in equations ( 4) and ( 5). We considered two groups of combination matrices A

given in [START_REF] Chen | Diffusion LMS over multitask networks[END_REF] and [START_REF] Zhao | Clustering via diffusion adaptation over networks[END_REF], with ( , k)-th entries at time instant n + 1 given by:

a (1) 2, k (n+1) = ψ k,n+1 + µ k q k,n -ψ ,n+1 -2 j∈N k ψ k,n+1 +µ k q k,n -ψ j,n+1 -2 (85) a (2) 2, k (n+1) = ζ -2 k,n+1 j∈N k ζ -2 jk,n+1 (86)
for ∈ N k . We use the index (n + 1) in ( 85) and (86) to highlight the time-variant nature of adaptive combination matrices, and quantities q k,n and ζ k,n+1 are evaluated as:

q k,n = [d k,n -x k,n ψ k,n+1 ] • x k,n (87)
ζ 2 k,n+1 = (1 -τ k ) • ζ 2 k,n + τ k • ψ ,n+1 -w k,n 2 (88)
where 0 < τ k 1 are forgetting factors. The evolution of the coefficient vectors w k was divided into four stationary stages and three transient episodes. For stationary stages, the vectors w k were generated randomly from a standard Gaussian distribution.

During stationary stages, we set w k at each agent so that, from time instant n = 1 to 1000 and n = 4501 to 7000, the whole network tracked the same target, while from instant n = 1501 to 2500 and n = 3001 to 4000, the network started to track 2 and 3 targets, respectively. The transient episodes were designed by using linear interpolation over 500 time instants by a + b-a 500 (nn c ), with a and b denoting an element of the weight of the previous and next stationary stages respectively, and n c denoting the starting instant of the current transient episode -see [START_REF] Chen | Diffusion LMS over multitask networks[END_REF]. Besides, we set ε = 0.05 and η = 0.95 for power-normalized diffusion scheme, with ν γ k = 0.01 for static combination matrices and ν γ k = 0.04 for adaptive combination matrices. For sign-regressor diffusion, we set ν γ k to 0.015 and 0.03 for static and adaptive combination matrices, respectively.

The results are plotted in Figs. 3 and4 for static and adaptive fusion matrices, respectively. In Fig. 3(a), as expected, the power-normalized and sign-regressor diffusion schemes led to a MSD learning curve approaching the best of each component strategies at the different stages. This coincides with the theoretical results that these schemes are universal at steady state.

The evolutions of affine combination coefficients in Fig. 3(b) ensure the effectiveness of proposed schemes.

The results for adaptive fusion matrices plotted in Fig. 4 lead to similar conclusions as Fig. 3. Interestingly, from time instant n = 1901 to 2500, the combined result of the power-normalized diffusion scheme outperformed each individual one, driven by combination coefficients γ k,n such as at node 4 in Fig. 4(b). This confirms the theoretical result in ( 29) that shows that the combined strategy can outperform each component strategy in certain situations. The power-normalized diffusion performs better than the sign-regressor diffusion due to its faster convergence rate. All of the results in Fig. 3 and Fig. 4 illustrate the effectiveness of the proposed schemes.

2) Affine combination of other strategies: Consider the diffusion strategies for clustered multi-task networks proposed in [START_REF] Chen | Multitask diffusion adaptation over networks[END_REF] and [START_REF] Nassif | Multitask diffusion LMS with sparsity-based regularization[END_REF]. The former uses squared 2 -norm co-regularizer to promote cooperation within clusters, while the latter uses an 1 -norm co-regularizer. The simulation setting is similar to that of Section VI-A1, except that the 10 nodes were grouped into 3 clusters, to track three groups of different but related targets. For stationary stages, the coefficient vectors w Ci were generated as w Ci = w o + δ Ci w Ci , with w Ci drawn from standard Gaussian distribution. When δ Ci for i = 1, 2, 3 are the same or similar, [START_REF] Nassif | Multitask diffusion LMS with sparsity-based regularization[END_REF] with 1 norm co-regularizer works better, otherwise the method in [START_REF] Chen | Multitask diffusion adaptation over networks[END_REF] is better. The regularization strength of the two co-regularizers were both set to 0.1, and a uniform A (i) 2 was used such that a

(i) 2, k = |N k ∩ C(k)| -1 .
For the power-normalized diffusion scheme, we set ν γ k to 0.01, ε to 0.05, and η to 0.95. For sign-regressor diffusion, we set ν γ k to 0.1.

The results are plotted in Fig. 5 and Fig. 6. Both power-normalized and sign-regressor diffusion not only led to the best of each component strategies at different stages, they also outperformed each component strategies at some instants.

3) Influence of parameters: Since there are several parameters in the power-normalized and sign-regressor schemes, such as step-size ν γ k , temporal smoothing factor η and parameter ε, we examine their influence on the performance. Based on various experiments, we find that the performance of the power-normalized scheme is not sensitive to η and to small-valued ε. We therefore suggest setting η to a typical value of 0.95 and ε to 0.05. We also examine the influence of the step-size ν γ k . The simulation settings are identical to those used in the first experiment, and we use static combination matrices. Fig. 6. Simulation results of the sign-regressor scheme with the two diffusion strategies in [START_REF] Chen | Multitask diffusion adaptation over networks[END_REF] and [START_REF] Nassif | Multitask diffusion LMS with sparsity-based regularization[END_REF].

The results are plotted in Figs. 7 and8. For both power-normalized and sign-regressor diffusion, small ν γ k lead to weak ability in tracking the best component and slow convergence toward the best component at steady state, such as ν γ k = 0.001, while a large ν γ k results in biases from the best component at steady state, though a large step-size ensures good tracking for the best component. The step-size parameter ν γ k needs to be fine-tuned to ensure good tracking ability and a lower bias from the best component at steady state. In this simulation setting, the values of ν γ k are set to 0.01 and 0.015 for power-normalized and sign-regressor schemes, respectively.

B. Theoretical models

To illustrate the theoretical results as well as challenge the assumptions and approximations adopted in the theoretical analysis, we considered three networks with different connectivity parameters as described in Table I. Net1 consisted of 10 nodes with the network topology given in Fig. 2(a). Net2 was a more complicated network consisting of 20 nodes. Net3 was generated by dividing 20 nodes into seven fully connected clusters, with 3 nodes in each of the first six clusters and 2 nodes in the last cluster. These seven clusters were connected in chain, with a single edge connecting adjacent clusters: agent 3 (in cluster 1) was connected with agent 4 (in cluster 2), and agent 6 (in cluster 2) was connected with agent 7 (in cluster

3), and so on until agent 18 (in cluster 6) was connected to agent 19 (in cluster 7). The unknown coefficient vectors to be estimated were of length L = 2. We first considered regressors drawn from a zero-mean

Gaussian distribution with covariance matrix R x,k = σ 2 x,k I L . Next we considered colored regressors x k,n = [x k,n x k,n-1 ] generated from a first-order AR model: x k,n = 0.5x k,n-1 + 0.75σ 2
x,k w k,n . The input signal w k,n was i.i.d and drawn from a zero-mean Gaussian distribution, with variance σ 2 w = 1, so that: The noise signals were generated from Gaussian distributions N (0, σ 2 z,k ). Variances σ 2 x,k and σ 2 z,k at each agent were generated randomly. By varying σ 2 z,k , we changed the signal-to-noise ratio (SNR) [START_REF] Das | Distributed state estimation in multi-agent networks[END_REF] to three levels as described in Table II. Both combination schemes were run with network step-sizes being set to {0.01, 0.002}, and with ν γ k correspondingly being set to {0.01, 0.002} and {0.015, 0.001} for power-normalized diffusion and sign-regressor diffusion, respectively. Besides, we set ε to 0.05 and η to 0.95 for power-normalized diffusion. We first validated the theoretical results related to the mean and mean-square behaviors of γ k,n , the theoretical MSD of each component strategy, as well as cross-MSD of the whole network defined by MSD cross 1

R x,k = σ 2 x,k   1
N E v (1) n v (2) n
. Then we evaluated the theoretical MSD behavior of the combined strategy. The results for white Gaussian inputs with Net1 and SNR1 are plotted in Fig. 9-Fig. 12. In both Fig. 9 and Fig. 11, we observe that the simulated transient values and theoretical transient values accurately matched, especially for small step-sizes, since the approximations adopted in theoretical analyses are more reasonable for small step-sizes. Further, we observe that a larger ν γ k results in a faster convergence rate, which coincidences with equations ( 138) and (151) in theoretical analyses.

In Fig. 10 and Fig. 12, besides the accurate matching of the simulated and theoretical MSD learning curves, the almost superposition of theoretical steady-state MSDs for combined strategy and that of the best component validates the conclusion again that the combination schemes are universal at steady state. Further, the gaps of these two theoretical values becomes smaller as the step-sizes decrease, since the analyses of the universality are based on the approximations that are more valid for small step-sizes.

The results of the power-normalized diffusion for white Gaussian inputs with Net2 and Net3 in SNR1 are plotted in Fig. 13 and Fig. 14, respectively. And the results with Net1 in SNR2 and SNR3 are plotted in Fig. 15 and Fig. 16, respectively. Together with Fig. 10, all these validate the accuracy of theoretical analyses under different SNR conditions and network connectivity parameters.

The results of the power-normalized diffusion for correlated input with Net1 and SNR1 are plotted in Fig. 17. Though assumption A2 is violated, the superimposition of simulated and theoretical curves validates the accuracy of the theoretical analyses for sufficiently small step-sizes with moderately correlated regressors.

VII. CONCLUSION AND PERSPECTIVES

Combining diffusion strategies enables a network to reach better performance. In this paper, we proposed two schemes for affine combination of two diffusion strategies. By using the proposed combination schemes, we obtained a combined diffusion strategy ensuring the advantages of both component strategies simultaneously, sometimes even better, in terms of EMSE. We conducted theoretical analyses in the mean and mean-square sense, and analyzed the universality of each approach. Simulation results illustrate the interesting properties of affine combination schemes, as well as the accuracy of the theoretical results.

Several open problems still have to be addressed. For instance, it would be interesting to design combination schemes and conduct theoretical analysis for colored measurement noise and correlated regressors. Some works focusing on adaptive networks with colored measurement noise include [START_REF] Bertrand | Diffusion bias-compensated RLS estimation over adaptive networks[END_REF]- [START_REF] Piggott | Diffusion LMS with correlated regressors II: Performance[END_REF]. It would also be interesting to explore other combination frameworks and schemes.

APPENDIX A COMBINATION SCHEME FOR MULTIPLE STRATEGIES

The general scheme follows the description from Section III except that we now have M component diffusion strategies.

We introduce M affine combination coefficients γ

k,n , • • • , γ (M ) k,n for node k at time instant n, satisfying the constraint M i=1 γ (i) k,n = 1. (1) k,n , γ (2)
By combining the estimates of M component strategies at each agent k, we obtain the overall system coefficients w k,n and estimation error e k,n at combination layer, with

e k,n = M i=1 γ (i) k,n e (i) k,n . In order to keep γ (i)
k,n satisfying the sum-to-one constraint, we calculate γ (i) k,n via the mapping:

γ (i) k,n = α (i) k,n + δ M j=1 α (j) k,n + M δ , (90) where α (i)
k,n are newly introduced auxiliary parameters, and δ is a small positive constant to avoid zero-division. We shall update α

(i) k,n instead of updating γ (i)
k,n directly. By minimizing the MSE ( 16) over the entire network, and using stochastic gradient approximation, we obtain:

α (i) k,n+1 ≈ α (i) k,n + ν α k e k,n sgn e k,n -e (i) k,n M j=1 α (j) k,n + M δ . ( 91
)
APPENDIX B

PROOF OF UNIVERSALITY ANALYSIS RESULT 1

Starting from the update equation of γ k,n in (20), we know that the stationary point of γ k,n is reached for:

E ν γ k ε + p k,n e k,n x k,n w (1) k,n -w (2) k,n = 0. (92)
Using ( 27) and ( 28), we obtain:

E ν γ k ε + p k,n γ k,n e (1) k,n e (2) k,n -( e (1)
k,n ) 2 + ν γ k ε + p k,n 1 -γ k,n ( e (2) k,n ) 2 -e (1)
k,n e

(2) k,n = 0, (93)
where we used the zero-mean property of z k,n under assumption A1. Though we cannot obtain a closed-form solution for

E{γ k,n } at each time instant, by resorting to approximations Ap 1 , Ap 2 and introducing the quantity

J (i) k,n J (i) ex,k,n -J (1,2) ex,k,n , with J (1,2)
ex,k,n denoting the cross-EMSE defined by

J (1,2) ex,k,n E{ e (1) k,n e (2)
k,n }, equation (93) at steady-state becomes:

-γ k,∞ J (1) k,∞ + 1 -γk,∞ J (2) k,∞ = 0 (94)
with γk,∞ E{γ k,∞ }. This leads to a closed-form expression for the stationary point at steady-state:

γk,∞ = J (2) k,∞ J (1)
k,∞ + J (2) k,∞ . (95)
We can now establish the universality of the power-normalized scheme (20) at steady-state by using (95). Under approximation Ap 1 , and assuming further that the variance of γ k,∞ is small enough so that the approximation E{γ 2 k,∞ } ≈ γ2 k,∞ holds, the EMSE at node k after combination at steady-state can be written as:

J ex,k,∞ ≈ γ2 k,∞ J (1) ex,k,∞ + (1 -γk,∞ ) 2 J (2) ex,k,∞ + 2γ k,∞ (1 -γk,∞ ) J (1,2) ex,k,∞ . (96)
By substituting γk,∞ in (95) into (96), and after some algebraic manipulations, we arrive at:

J ex,k,∞ = J (1,2) ex,k,∞ + J (1) k,∞ J (2) k,∞ J (1) k,∞ + J (2) k,∞ . (97)
From Cauchy-Schwartz inequality, we know that J

(1,2) ex,k,∞ cannot be simultaneously larger than J

ex,k,∞ and J

ex,k,∞ . Therefore, to evaluate (97), we split the problem into three cases according to the relations between

J (1,2) ex,k,∞ and J (i) ex,k,∞ : • Case 1: J (i) ex,k,∞ ≥ J (1,2) ex,k,∞ for i = 1, 2, which means J (1) k,∞ ≥ 0 and J (2)
k,∞ ≥ 0 at the same time. Since

J (1) k,∞ J (2) k,∞ J (1) k,∞ + J (2) k,∞ ≤ J (1)
k,∞ and

J (1) k,∞ J (2) k,∞ J (1) k,∞ + J (2) k,∞ ≤ J (2)
k,∞ , we conclude from (97) that:

J ex,k,∞ ≤ J (1) ex,k,∞ and J ex,k,∞ ≤ J (2) ex,k,∞ . (98)
• Case 2:

J (1) ex,k,∞ < J (1,2) ex,k,∞ < J (2)
ex,k,∞ , which corresponds to J

k,∞ < 0 and J

k,∞ > 0. Then (97) leads to:

J ex,k,∞ = J (1) ex,k,∞ - J (1) k,∞ 2 J (1) k,∞ + J (2) k,∞ . (99)
Since J

(1)

k,∞ + J (2)
k,∞ > 0, we have J ex,k,∞ < J

ex,k,∞ . As a result, J ex,k,∞ < min J

ex,k,∞ , J

ex,k,∞ .

• Case 3:

J (1) ex,k,∞ > J (1,2) ex,k,∞ > J (2)
ex,k,∞ . Due to the symmetry of (97) for J

k,∞ and J

k,∞ , we obtain the same conclusion as Case 2.

Therefore, at steady-state when the combination coefficient (95) is applied, the resulting EMSE after combination cannot be worse than that of the best component, that is,

J ex,k,∞ ≤ min J (1) ex,k,∞ , J (2) ex,k,∞ . (100)
We also conclude that equation ( 29) holds. It means that the EMSE of the diffusion network after combination cannot be worse than that of the best component strategies, leading to the universality property of the power-normalized scheme at steady state.

APPENDIX C

PROOF OF STABILITY ANALYSIS RESULT 1 E v n+1 converges as n → ∞ if, and only if, both terms on the right-hand side (RHS) of ( 40) converge to finite values.

Using [START_REF] Lorenzo | Sparse distributed learning based on diffusion adaptation[END_REF] and iterating the first term from E v

(1) 0

, we obtain:

E Γ n+1 E v (1) n+1 = E Γ n+1 B (1) n+1 E v (1) 0 - n j=0 E Γ n+1 B
(1) j r (1) .

To prove convergence of (101), it is sufficient to prove that the sequence

s 1,m E Γ n+1 B (1) n+1 E v (1) 0 m
and series

n j=0 E Γ n+1 B (1) j r (1)
m converge for m = 1, • • • , N L.
It is known that a sequence is convergent if it is upper bounded and lower bounded by two sequences with the same limit [START_REF] Sohrab | Basic Real Analysis[END_REF]. A series is absolutely convergent if each of its terms is bounded by a term of an absolutely convergent series [START_REF] Lorenzo | Sparse distributed learning based on diffusion adaptation[END_REF], [START_REF] Nassif | Proximal multitask learning over networks with sparsity-inducing coregularization[END_REF]. Define s 2,m E Γ n+1 B

(1) j r (1) m . Since the largest absolute value of the entries in a block vector is smaller than or equal to the block maximum norm of this vector, we have:

|s 1,m | ≤ E Γ n+1 b,∞ • B (1) n+1 b,∞ • E v (1) 0 b,∞ (44)
≤ E Γ n+1 b,∞ • M (1) n+1 b,∞ • E v (1) 0 b,∞ = ρ E Γ n+1 • ρ M (1) n+1 E v (1) 0 b,∞ , (102)
|s 2,m | ≤ E Γ n+1 b,∞ • B (1) j b,∞ • r (1) b,∞ (44)
≤

ρ E Γ n+1 • ρ M (1) j • r (1) b,∞ , (103)
where

M (1) I N L -U (1) H (1) , (104)
• b,∞ is the block maximum norm [START_REF] Sayed | Diffusion adaptation over networks[END_REF], and ρ(•) is the spectral radius of a given matrix. Since E v

(1) 0 and r (1) are bounded, it is sufficient to require that ρ M (1) < 1 and ρ E Γ n+1 is uniformly bounded. It is similar for the second term on the RHS of [START_REF] Piggott | Diffusion LMS with correlated regressors II: Performance[END_REF]. As discussed later in Section IV-C, condition (56) ensures the convergence of E Γ n+1 and E I N L -Γ n+1 .

Given that ν γ k satisfies condition (56), the convergence of (40) requires only ρ I N L -U (i) H (i) < 1, which is ensured by step-sizes satisfying (55).

For the asymptotic bias, from ( 40) and ( 42), we obtain:

E v ∞ = E Γ ∞ E v (1) ∞ +(I N L -E Γ ∞ ) E v (2) ∞ (105) E v (i) ∞ = -I N L -B (i) -1 r (i) , (106)
which leads to expression (57).

APPENDIX D DERIVATION OF EQUATION (62)

Using ( 41), ( 63), (65), and approximation Ap 4 , we have:

E v (i) n+1 2
Σ (i) n+1 = E v (i) n 2 Σ (i) B,n+1 + trace Σ (i) n+1 G (i) + E f (r (i) n , Σ (i) n+1 , v (i) n ) (107)
with

Σ (i) B,n+1 E B (i) n Σ (i) n+1 B (i) n . (108)
In the derivation of expression (107), we used relation E{g

(i)
n } = 0, which is derived from ( 50) and (51) directly by using the zero-mean property of z ,n .

Let σ

(i) B,n+1 = vec Σ (i) B,n+1
. Under approximation Ap 4 , and using the properties of vec{•} operator, we have:

σ (i) B,n+1 = K (i) σ (i) n+1 , (109)
where

K (i) E B (i) n ⊗ B (i) n ≈ B (i) ⊗ B (i) (110)
for sufficiently small step-sizes. We have:

trace

Σ (i) n+1 G (i) = vec{G (i) } σ (i) n+1 . (111)
For sufficiently small step-sizes, we have:

E f (r (i) n , Σ (i) n+1 , v (i) n ) ≈ f r (i) , Σ (i) n+1 , E v (i) n . (112)
Substituting ( 111) and ( 112) into (107) yields expression (62).

APPENDIX E PROOF OF STABILITY ANALYSIS RESULT 2

The convergence of E v n+1 2 Σ requires the convergence of the terms on the RHS of (58). For the last term, using (62) and iterating from n = 0, we find that

E v (1) n+1 2 σ (1) n+1 = v (1) 0 2 (K (1) ) n+1 σ (1) n+1 + vec{G (1) } n t=0 (K (1) ) t σ (1) n+1 + n t=0 f r (1) , (K (1) ) t σ (1) n+1 , E v (1) n-t (113) for n ≥ 0 with initial condition v (1) 0 = w (1) 0 -w . Since σ (1) n+1 = E Γ n+1 ⊗ Γ n+1 σ (114)
where σ vec{Σ}, the convergence of (113) requires that the sequence s 3,m (K (1) )

n+1 E Γ n+1 ⊗ Γ n+1 σ m and series n t=0 (K (1) ) t E Γ n+1 ⊗ Γ n+1 σ m be convergent for m = 1, • • • , (N L) 2 . Define s 4,m (K (1) ) t E Γ n+1 ⊗ Γ n+1 σ m .
Similar to (102) and (103), we have:

|s 3,m | ≤ K (1) n+1 b,∞ • E Γ n+1 ⊗ Γ n+1 b,∞ σ b,∞ = K (1) n+1 b,∞ • ρ E Γ n+1 ⊗ Γ n+1 σ b,∞ , (115)
|s 4,m | ≤ K (1) t b,∞ • E Γ n+1 ⊗ Γ n+1 b,∞ σ b,∞ = K (1) t b,∞ • ρ E Γ n+1 ⊗ Γ n+1 • σ b,∞ . (116)
According to assumption A2, we have:

E{γ k,n+1 γ ,n+1 } = 0, ∀k = . (117)
Thus, ρ E Γ n+1 ⊗ Γ n+1 is bounded as long as E{Γ n+1 Γ n+1 } is convergent. We further have:

K (1) b,∞ (64)
= B (1) ⊗ B (1) b,∞ (44)
≤ A A A

(1)

1 ⊗ A A A (1) 1 b,∞ M (1) ⊗ M (1) b,∞ A A A (1) 2 ⊗ A A A (1) 2 b,∞ . (118)
Since σ b,∞ is bounded,

M (1) ⊗ M (1) b,∞ (104)
= ρ M (1) ⊗ M (1) = ρ M (1) 2 (119) and A A A

(1)

j ⊗ A A A (1) j b,∞ = A A A (1) j ⊗ A (1) j ∞ = max ,p N k=1 a (1) j, k N q=1 a (1) j
,pq = 1, (120)
where the notation • ∞ denotes the maximum absolute row sum of its argument, the convergence of (115) and (116) requires that ρ M (1) < 1 and the convergence of E{Γ n+1 Γ n+1 }. It is similar for the first two terms on the RHS of (58). As derived later in Section IV-C, condition (73) guarantees the stabilities of E{Γ n+1 } and E{Γ n+1 Γ n+1 }. Then, for any given weighting matrix Σ, condition (55) ensures the mean-square stability of the power-normalized diffusion scheme.

APPENDIX F RECURSIONS FOR EVALUATING TRANSIENT MSD

Recursion for evaluating E v

(i) n 2 K (i) σ (i) n+1
is given in [START_REF] Chen | Diffusion LMS over multitask networks[END_REF] as:

ξ (i) n+1 = ξ (i) n + vec{G (i) } (K (i) ) n σ (i) + r (i) 2 (K (i) ) n σ (i) -v (i) 0 2 (I-K (i) )(K (i) ) n σ (i) -2 Λ (i) n + (B (i) E{v (i) n }) ⊗ r (i) σ (i) , (121)
and

Λ (i) n+1 = Λ (i) n K (i) + (B (i) E{v (i) n }) ⊗r (i) K (i) -I (122)
obtain:

lim n→∞ 1 N E v n+1 2 = 1 N E v (1) ∞ 2 Γ ∞ Γ∞ + 1 N E v (2) ∞ 2 (I N L -Γ∞) (I N L -Γ∞) + 2 N E v (1) ∞ Γ ∞ (I N L -Γ ∞ ) v (2) ∞ , (126)
which is the summation of three steady-state values. For the first two terms, recursing (62) with n → ∞ yields:

lim n→∞ E v (i) n+1 (I (N L) 2 -K (i) )σ (i) n+1 = vec{G (i) } σ (i) ∞ + f r (i) , Σ (i) ∞ , E v (i) ∞ . (127)
In order to use (127) in (126), we select σ

∞ and σ

∞ to satisfy:

(I (N L) 2 -K (1) )σ (1) ∞ = 1 N vec E{Γ ∞ Γ ∞ } , (128)
(I (N L) 2 -K (2) )σ (2) ∞ = 1 N vec E{(I N L -Γ ∞ ) (I N L -Γ ∞ )} . (129)
Similarly for the last term of (126), recursing (66) with n → ∞ yields: 1) , B 2) ,

E v (1) ∞ vec -1 {(I (N L) 2 -K x )σ x,∞ }v (2) ∞ = vec{G x } σ x,∞ +f x r (1) , r (2) , Σ x,∞ , E v (1) ∞ , E v (2) ∞ , B (
and we select σ x,∞ to satisfy:

(I (N L) 2 -K x )σ x,∞ = 2 N vec E{Γ ∞ (I N L -Γ ∞ )} . (131)
These expressions lead to the MSD at steady state as: 1) ,B

MSD steady = vec{G (1) } σ (1) ∞ +f r (1) , Σ (1) ∞ , E v (1) ∞ + vec{G (2) } σ (2) ∞ +f r (2) , Σ (2) ∞ , E v (2) ∞ + vec{G x } σ x,∞ + f x r (1) ,r (2) ,Σ x,∞ , E v (1) ∞ , E v (2) ∞ ,B (
(132) with

Σ (i) ∞ = vec -1 {σ (i) ∞ }, ∀ i = 1, 2 (133)
Σ x,∞ = vec -1 {σ x,∞ }. ( 134
) APPENDIX H PROOF OF STABILITY ANALYSIS RESULTS 3 & 4
By substituting ( 27), ( 28) into [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF], and taking expectation, we obtain:

E{γ k,n+1 } = E γ k,n +E ν γ k ε + p k,n γ k,n e (1)
k,n e

k,n -( e

k,n ) 2 + ν γ k ε + p k,n 1 -γ k,n ( e (1)
k,n ) 2 -e (2)
.

By resorting to EMSEs J

ex,k,n and cross-EMSE J

ex,k,n defined in Section IV-A, as well as Ap 2 , Ap 5 , Ap 6 , expression (135) becomes

E{γ k,n+1 } = E γ k,n 1 - ν γ k ε + pk,n J (1) k,n + J (2) k,n + ν γ k ε + pk,n J (2) ex,k,n -J (1,2) ex,k,n , (136)
where we adopt the following approximation to simplify the derivation:

E ν γ k ε + p k,n ≈ ν γ k ε + pk,n with pk,n = η pk,n-1 + (1 -η)( J (1)
k,n + J (2) k,n ). (137)
From (136) a sufficient condition can be derived for the step-size ν γ k to ensure the mean stability of the power-normalized scheme [START_REF] Sayed | Adaptive Filters[END_REF], [START_REF] Candido | Transient and steady-state analysis of the affine combination of two adaptive filters[END_REF]:

1 - ν γ k ε + pk,n J (1)
k,n + J

(2)

k,n < 1 -φ, ∀n, (138)
with a positive constant φ, i.e., the left term is uniformly bounded away from one. A sufficient condition to ensure (138) is given by:

0 < ν γ k < min n ε + pk,n J (1)
k,n + J

Substituting pk,n of (137) into (139), and after some mathematical manipulations, we obtain condition (56). By further taking the limit of (136) as n → ∞, and solving for E{γ k,∞ }, we arrive at equation (95).

1 + 3 ν γ k ε + pk,n 2 J (1) k,n + J (2) k,n 2 - 2ν γ k ε + pk,n J (1)
k,n + J

(2)

k,n < 1 -φ, ∀n (146)
Next, we evaluate the mean-square behavior of γ k,n . To simplify the notation, define ν γ k,n νγ k ε+p k,n . By substituting ( 27) and ( 28) into [START_REF] Bershad | An affine combination of two LMS adaptive filters-transient mean-square analysis[END_REF], we obtain:

γ k,n+1 = γ k,n 1 -ν γ k,n e (2)
k,ne (140)

As a consequence, we have:

E{γ 2 k,n+1 } = E α 2 1 +E α 2 2 +E α 2 3 +2E α 1 α 2 . ( 141
)
By resorting to approximations Ap 2 , Ap 5 -Ap 7 , and using expression (140), we obtain:

E α 2 1 = E{γ 2 k,n } 1 + 3E ν 2 γ k,n J (1)
k,n + J (2) k,n 2 -2E ν γ k,n J (1)
k,n + J

(2)

k,n , (142)
E α 2 2 = E ν 2 γ k,n J (2) ex,k,n J (1)
k,n + J

(2) k,n

+ 2E ν 2 γ k,n J (2) k,n 2 , ( 143
)
E α 2 3 = σ 2 z,k E ν 2 γ k,n J (1)
k,n + J

(2)

k,n , (144)
E α 1 α 2 = E γ k,n E ν γ k,n J (2) k,n -3E ν 2 γ k,n J (1)
k,n + J From (141) and the term E α 2 1 , the range of step-sizes to ensure the mean-square stability is given by ( 146). With the use of approximation (137), then for a constant step-size, a sufficient condition to ensure (138) is given by:

0 < ν γ k < min n ε + pk,n 3 J (1) k,n + J (2) k,n . (147)
Substituting pk,n of (137) into (147), and after some mathematical manipulations, we arrive at condition (73).

APPENDIX I PROOF OF STABILITY ANALYSIS RESULTS 7 & 8

We first evaluate the mean behavior of γ k,n for the sign-regressor diffusion scheme in (79). To make the analysis tractable, we introduce approximation Ap 5 and joint Gaussian assumption Ap 7 in the same way. Furthermore, by resorting to Price's theorem [START_REF] Sayed | Adaptive Filters[END_REF], the following approximations hold:

E e (2)
To ensure the mean stability of γ k,n , the step-size parameter ν γ k must satisfy:

1 -ν γ k 2 J (1) k,n + J (2) k,n /π < 1 -φ, ∀n (151)
with a positive φ. A sufficient condition is then given by:

0 < ν γ k < 1 max n 2 J (1) k,n + J (2) k,n /π , (152)
which leads to condition (82) directly. In addition, by taking the limit of (150) as n → ∞, we obtain the same steady-state value for E{γ k,n } as in (95).

To evaluate the mean-square behavior of γ k,n , by substituting ( 27), ( 28) into [START_REF] Jin | Convex combination of diffusion strategies over distributed networks[END_REF] and rearranging terms, we obtain:

γ k,n+1 = 1 -ν γ k e (2)
k,ne

To obtain an explicit expression for the mean-square behavior of γ k,n , we have to evaluate the terms on RHS of (154) one by one under the joint Gaussian assumption Ap 7 . By resorting to Ap 5 , Ap 7 and utilizing (153), we obtain:

E{α 4 α 5 }= E{γ k,n } ν γ k E e (2)
k,n sgn{ e

k,n } -ν 2 γ k J (2) k,n ≈ E{γ k,n } 2 π ν γ k J (2) k,n J (1) k,n + J (2) k,n -ν 2 γ k J (2) k,n , (155)
E{α 2 4 } ≈ E{γ 2 k,n } 1 + ν 2 γ k J (1)
k,n + J

(2) k,n -2ν γ k 2 J

(1)

k,n + J (2)
k,n /π , (156)

E{α 2 5 } ≈ ν 2 γ k J (2) ex,k,n , (157)
E{α 2 6 } ≈ ν 2 γ k σ 2 z,k , (158)
and we used sgn e

k,ne

(1) k,n 2 ≈ 1 in the derivations of ( 157) and (158). Equations ( 154) and ( 155)-(158) constitute the iteration of the mean-square behavior of γ k,n . From (154) and the term E{α 2 4 }, we obtain the condition to ensure the mean-square stability of γ k,n as:

1 + ν 2 γ k J (1) k,n + J (2) k,n -ν γ k 8 J (1) k,n + J (2) k,n /π <1 -φ (159)
for all n. For a constant step-size, a sufficient condition to ensure (159) is given by equation (83). In addition, by taking the limit of (154) as n → ∞, we obtain the steady-state value of E{γ 2 k,n } as (84).

Fig. 1 .

 1 Fig. 1. Illustration of the combination framework for two diffusion strategies.

  , the estimation error e (i) k,n , and the a priori output estimation error e (i) k,n :

Result 2 :

 2 (Mean-square Stability) Assume data model (1) and A1-A2, Ap 3 -Ap 4 hold. Assume further that the step-sizes µ (i)

Ap 5 :

 5 The combination coefficient γ k,n varies slowly enough so that the correlation between γ k,n and e(m) k,n e (n)k,n for m, n = 1, 2 can be ignored.

Ap 6 :

 6 For a large enough temporal smoothing factor η, p k,n is statistically independent of γ k,n .Ap 7 : The a priori errors e (1) k,n and e (2)

Result 6 :

 6 (Mean-square Stability) Assume data model (1) and A1-A2, Ap 3 -Ap 4 hold. Assume further that the step-sizes µ (i)

Fig. 2 .

 2 Fig. 2. Network topology and associated input variances and noise variances. (a) Network topology; (b) Agent input and noise variances.

Fig. 3 . 1 )

 31 Fig. 3. Simulation results with static fusion matrices. (a) Network MSD learning curves; (b) Evolution of combination coefficient γ k,n for power-normalized scheme.

2 =

 2 I N and averaging rule[START_REF] Sayed | Diffusion adaptation over networks[END_REF] for A

Fig. 4 .

 4 Fig. 4. Simulation results with adaptive fusion matrices.

Fig. 5 .

 5 Fig. 5. Simulation results of the power-normalized scheme with the two diffusion strategies in [8] and [34]. (a) Network MSD learning curves; (b) Evolution of the affine combination coefficients γ k,n .

Fig. 7 .Fig. 8 .

 78 Fig. 7. Simulation results of the power-normalized scheme with different step-sizes νγ k .

Fig. 9 .

 9 Fig. 9. Illustration of simulation results (model vs. Monte Carlo) for the power-normalized scheme in Net1 and SNR1. Transient and steady-state values of E{γ k,n } derived in (136) and (95) (top), as well as these of E{γ 2 k,n } derived in (141) and (78) (bottom) for network step-size 0.01 (a) and 0.002 (c); (b) Transient and steady-state cross-MSDs derived in (123) and (130).

Fig. 10 .

 10 Fig. 10. Network performance with the power-normalized scheme (model vs. Monte Carlo) with two different network step-sizes in Net1 and SNR1. For each step-size, the results plotted with a same color stand for same diffusion strategy. For each component diffusion strategy, theoretical transient MSD and theoretical steady-state MSD are derived in (121) and (127). The combination results are derived in (58) and (132).

Fig. 11 .

 11 Fig. 11. Illustration of simulation results (model vs. Monte Carlo) of the sign-regressor scheme in Net1 and SNR1. Transient and steady-state values of E{γ k,n } derived in (150) and (95) (top), as well as E{γ 2 k,n } derived in (154) and (78) (bottom) for network step-size 0.01 (a) and 0.002 (c); (b) Transient and steady-state cross-MSDs derived in (123) and (130).

Fig. 12 .

 12 Fig. 12. Network performance with the sign-regressor scheme in Net1 and SNR1.

Fig. 13 .

 13 Fig. 13. Network performance with the power-normalized scheme in Net2 and SNR1, and the network step-sizes are 0.01.

Fig. 14 ..Fig. 15 .

 1415 Fig. 14. Network performance with the power-normalized scheme in Net3 and SNR1, and the network step-sizes are 0.01 .

Fig. 16 .

 16 Fig. 16. Network performance with the power-normalized scheme in Net1 and SNR3, and the network step-sizes are 0.01.

Fig. 17 .

 17 Fig.[START_REF] Arenas-Garcia | Plant identification via adaptive combination of transversal filters[END_REF]. Network performance with the power-normalized scheme using correlated inputs in Net1 and SNR1.

E ν 2 γ

 2 k,n ≈ E ν γ k,n2 in (142)-(145), we obtain the expression of E{γ 2 k,n+1 }. Taking the limit of (141) with n → ∞, and solving for E{γ 2 k,∞ } lim n→∞ E{γ 2 k,n }, we obtain the steady-state value (78), with ν∞ E{ν γ k,∞ } = lim n→∞ E{ν γ k,n }.

+ 4 } + E{α 2 5 } + E{α 2 6 }

 456 ν γ k z k,n sgn e squaring (153) and taking expectation, we haveE{γ 2 k,n+1 } = E{α 2 + 2E{α 4 α 5 }.

TABLE I NETWORK

 I

	Network Size Density	λ 2 (L) Diameter
	Net1	10	44%	0.7962	3
	Net2	20	38%	0.9549	3
	Net3	20	17.25%	0.0439	13

STATISTICS FOR THEORETICAL MODELS VALIDATION. L IS THE LAPLACIAN MATRIX ASSOCIATED WITH THE GRAPH (NETWORK), λ 2 (L) IS THE ALGEBRAIC CONNECTIVITY [35] OF GRAPH, S I Z E IS THE NUMBER OF NODES, D E N S I T Y IS THE NUMBER OF NON-ZERO ENTRIES OF THE ADJACENCY MATRIX OF GRAPH, AND D I A M E T E R IS THE MAXIMUM DISTANCE BETWEEN ANY TWO NODES

[START_REF] Simões | FADE: Fast and asymptotically efficient distributed estimator for dynamic networks[END_REF]

.

TABLE II THREE

 II SNR LEVELS IN DECIBEL (DB) FOR THEORETICAL MODELS VALIDATION. SINCE SNRS VARY FROM NODE TO NODE, WE ENUMERATE THE MAXIMUM, MINIMUM AND MEAN VALUES.

	SNR Level Maximum Minimum	Mean
	SNR1	3.5724	1.6673	2.7946
	SNR2	-9.4379	-11.343	-10.2157
	SNR3	-18.9803	-20.8855	-19.7581

  It is noted that approximation (148) is used in evaluating the transient and steady-state behaviors of E γ k,n and E{γ 2 k,n }. Then, under Ap 5 and Ap 7 , and with the use of approximation (149), iteration (79) becomes:E{γ k,n+1 } ≈ E γ k,n + ν γ k E e

	k,n sgn e	(2) k,n -e	(1) k,n	≈	2 π	(2) k,n J (1) k,n + J J	k,n (2)	(148)
	E e	(2) k,n -e	(1) k,n	≈ 2		J	(1) k,n + J k,n /π. (2)	(149)

In this paper, we adopt the acronym "A" for "assumption".

In this paper, we adopt the acronym "Ap" for "approximation".

The work of C. Richard was funded in part by ANR under grant ANR-19-CE48-0002.

with

Besides, [START_REF] Chen | Chapter # -Multitask learning over adaptive networks with grouping strategies[END_REF] gives an alternative expression for (121), which greatly saves computations. Following the same routine, E v

can be evaluated as follows:

(1) 1) E{v (1) n })

with

.

APPENDIX G PROOF OF REMARK 2

The steady-state MSD is given by the limit MSD steady = lim n→∞ 1 N E v n+1 2 . By resorting to expression (58), we


Zine Eddine Cherif

Christophe Poilâne
email: christophe.poilane@unicaen.fr

Ladan Momayez

Jun Chen

FLAX /EPOXY PREPREGS: INFLUENCE OF ELABORATION PARAMETERS ON THE MECHANICAL PROPERTIES

Keywords: flax, prepreg, textile treatment, mechanical properties

The experienced prepreg flax/epoxy consists of twisted flax yarns coming from textile industry. It is mainly used in sports and leisure industry. The required added-value is the good capacity of vibration damping. A first study showed that the behavior of the flax/ epoxy is not just elastic and that its resistance is not optimal. We reached at best case 350 MPa where the mixtures law look forward to the double! The objective in the long term of the work consists in the optimization of composite prepreg. We studied the influence of the torsion of the streak by comparing a WOVEN comprising a weft yarn for 19 warp yarn , with an UD, an UD SEWN with 45° and a MAT having a privileged orientation. The best result is obtained for WOVEN. It is believed that the wet spinning optimizes the separation of elementary fibers. We studies the influence of textile treatment by comparing an UNBLEACHED 2/2 twill with the same LEACHED (WASHED), BLEACHED and MERCERIZED fabric. The best result is obtained for MERCERIZED fabric.

INTRODUCTION

The industrial employment of vegetable fibers as a support reinforcement in composite materials necessities to answer several requirements. Among these, figurant the capacity of the resin to adhere to fibers, the control of the water absorption and the user friendliness of the semi-manufactures. The tested flax/epoxy prepregs, provided by the Lineo NV Company, respond to these criteria favorably. They are used in the composite industry for several years (MUSEEUW bicycle, ARTENGO racket).

The Prepregs consist of flax yarns with controlled weight and an epoxide resin. The yarn contains a large number of fibers of unspecified origin, twisted according to the traditional techniques of spinning. The flax fibers intended for the textile may undergo various pre-treatment before or after weaving: bleaching, leaching and mercerizing. Usually, the yarn intended for prepreg comes from the wet spinning and thus undergoes leaching. Then the fabrics undergo a treatment making it possible to ensure the adhesion of the resin to the fiber. This stage, which can be compared to the sizing of glass fibers, constitutes the know-how of the elaboration of prepregs.

The primary study [START_REF] Poilâne | Propriétés mécaniques de préimprégnés lin/époxyde[END_REF] bearing on laminated plates with 0° showed that, the behavior of the flax/epoxy is not linear, it comprises a rather short elastic zone then a viscous-elastic-plastic zone; the mechanical properties of prepregs are proportional to the percentage of fibers; the mechanical properties of prepregs are inversely proportional to the yarn titration; the threshold of elasticity is a function of the percentage of fibers but independent of the yarn titration; the resistance of the flax/epoxy is not as high as the resistance of elementary fibers.

The objective of the work consists of understanding the origin of the nonelastic phenomenon and analyzing the influence of numerous parameters that participate in the development and elaboration of prepregs the flax/epoxy to improve the properties of these prepregs. Our studies showed the influence of the torsion of the streak by comparing the product reference, a WOVEN comprising a weft yarn for 19 warp yarns, with a UD, a SEWN UD at 45° and a MAT having a privileged orientation. We observed the influence of the textile treatment by comparing the reference product, an UNBLEACHED 2/2 twill made of elaborated yarn by semi-wet spinning with the same LEACHED, BLEACHED, MERCERIZED fabric and with a 2/2 twill made of LEACHED YARN in wet spinning.

ELABORATION OF THE MATERIAL a. THE FLAX FIBER

The utilized fibers in the processing of tested prepregs are not subject to any particular selection. It should be noted that some studies have shown the utility, for the composite industry sector, of selecting the flax varieties and preserving only the central part of the plant [START_REF] Charlet | Contribution à l'étude de composites unidirectionnels renforcés par des fibres de lin : relation entre la microstructure de la fibre et ses propriétés mécaniques[END_REF]. This approach, that certain industrialists try to put into practice, is not utilized here. Indeed, the objective of the provider of the prepreg is to penetrate into the composite market without deeply modifying the work tools of the flax professionals. The preparation of flax fibers is thus carried out according to four usual stages: retting, scutching, heckling, and folding/drawing. Retting is a natural process carried out in Normandy by the climate. After ripping out, the stems of the plant are left on the ground, in the form of swathe, in order to facilitate the action of natural micro-organisms. Degradation of pectin cements, favorite by the alternation of heat and moisture, makes it possible to obtain the first separation of fibers and wood. Scutching consists of extracting fibers contained in the stems by rippling and picking of the straws. The mechanical actions carried out on fibers include sorting, picking, rippling between two rib rollers, drawing by differential rolling and drafting. Hacking consists of carding the bout of fibers in a succession of increasingly fine hacks (sort of comb). Folding/ drawing of the flax make it possible to conglomerate fibers in order to obtain a several-meter long ribbon exploitable by the spinning mills. The ribbons are standardized by mixture of fibers from different origins.

In the world of vegetable fibers, the flax fibers offer top-of-the-range performances. We often balance the properties of fibers by their density in order to compare the flax and conventional fibers (table 1). D is the density, E is the Young's modulus, σ R is the ultimate stress (or failure stress), ε R is the ultimate strain (or failure strain), E/D is the specific Young's modulus and σ R / D is the specific ultimate stress. Concerning the origin of the product, the median values found in the literature [START_REF] Baley | Analysis of the flax fibres tensile behaviour and analysis of the tensile stiffness increase[END_REF] were selected and not those of the most performing flax fibers [START_REF] Charlet | Characteristics of Hermès flax fibres as a function of their location in the stem and properties of the derived unidirectional composites[END_REF]. We see that the specific properties of flax fibers are better than those of E-glass fibers. b. PREPREGS The tested prepreg are detailed in tables 2 and 3. They are: a UD resulting from scutching ribbons (long fibers), a UD sewn at 45° resulting from scutching ribbons which we name SEWN; a MAT resulting from tow agglomerated by water jet, comprising a preferential direction; a WOVEN comprising 1 weft yarn for 19 warp yarn coming from textile industry and a 2/2 twill having undergone leaching, bleaching or mercerizing.

The transformations of flax fibers into flax/epoxy prepreg may consist of several stages consist of: spinning, weaving, textile pretreatment or treatment, treatment (that can be compared to sizing of glass fibers), impregnation.

Stage 1 can be carried out in a semi-wet or a wet spinning. The yarns coming from the semi-wet spinning undergo only one plugging in water; we consider them as untreated. The yarns coming from the wet spinning are leached. The MAT, the UD and SEWN, undergo neither stage 1 nor stage 2. Stage 3 is not necessary for the elaboration of a prepreg. An untreated fabric is called UNBLEACHED. It is called LEACHED YARNS if the constitutive yarn comes from the wet spinning. Stage 4 and 5 are carried out on all the products analyzed in our study. The parameters of development and elaboration internal to these stages are fixed and thus do not represent variables of the study. c. SPECIMENS The plates are laminated in hot pressing using a mould according to the following stages: cutting folds of 200 mm × 110 mm according to the chosen orientation; waxing the moulds; stacking the folds in the male mould; putting adjusted metallic plates that make it possible to choose the air gap in the mould; placing and tightening of the female mould by screw; cross-linking in the enclosure equipped with a thermostat in atmospheric pressure; removing the composite plate from the mould after returning to room temperature.

The relationship between number of folds and thickness of the plates makes it possible to act slightly on the percentage of the fibers V f . Consequently, the resin surplus is more or less important according to this ration.

Curing is in conformity with the cycle of the supplier: a 30 minute landing at 80°C followed by a 60 minute landing at 130°C.

The laminated plates are cut out with a water-cooled diamonds saw then dried in the drying oven. The blank is cut and then polished with the strip. The samples intended for traction are equipped with 4 aluminum heels. The samples intended for optical microscopy are prepared according to the usual protocol of metallurgists. The samples intended for electronic microscopy receive a gold deposit.

TESTS AND RESULTS

The measurements consist of five tensile tests per product supplemented by observation under optical microscope and electronic microscope.

a. INFLUENCE OF THE OF ROVING TORSION

The four compared prepregs, a UD, a MAT, a SWEN, a WOVEN, are intended for the unidirectional reinforcement. The properties and the results in traction are synthesized on figure 1 and in table 4. We notice from the start that the behavior of SWEN is singular. The sewing generates a crimp of the fiber roving. The product is very difficult to handle without deformation because of the sewing at 45°. Another non-identified cause affects perhaps the behavior too!

The behavior of the MAT and WOVEN are similar. They are in conformity with the behavior usually observed in flax/ epoxy in the yarn direction: a rather short elastic strain followed by a non-elastic strain. The behavior of the UD, on the other hand, clearly shows a resumption of stiffness in the last part of the test. This is visible on the stiffness/strain curve in which we can follow the evolution of the secant module. However, the strength does not reach that of the WOVEN.

b. INFLUENCE OF TEXTILE TREATMENT

The 5 twills tested, an UNBLEACHED, a LEACHED, a LEACHED YARNS, a MERCERIZED and a BLEACHED, have the same volume percentage of fibers. The results are represented in figure 2 andtable 5.

The behavior of the fabric in traction is almost preserved whatever the treatment. The loss of stiffness of BLEACHED is however slight to which we cannot provide any explanation. Concerning the failure It is noticed, that the increase in strength is accompanied by a reduction in the strain. The treatments improve overall stiffness and strength of the flax /epoxy. A particular case is obtained by elaborated fabric with leached yarn. The final properties are not as good as those of untreated fabric! It should be reminded that it consists of wet spun short fibers.

DISCUSSION

The comparative WOVEN, MATE, SEWN, UD, confirmed the nonlinear of flax/epoxy behavior. The properties of the resin alone are not out issue because the intend is a standard formulation for prepregs. This resin may used in prepreg glass E/epoxy whose behavior is elastic linear. This phenomenon thus originates from flax fibers.

The resumption of rigidity observed is the UD provides us an important information. It enables us to correlate the behavior of the composite to that of elementary flax fiber. The results of the literature show that the resumption of rigidity is observed in certain flax fibers. Figure 3, drawn from K. Charlet's dissertation [START_REF] Charlet | Contribution à l'étude de composites unidirectionnels renforcés par des fibres de lin : relation entre la microstructure de la fibre et ses propriétés mécaniques[END_REF], show the behavior in three zones. The phenomenon is attributed to the reorientation of the fibril of the secondary wall of the initial fiber positioned approximately at 9° [START_REF] Baley | Analysis of the flax fibres tensile behaviour and analysis of the tensile stiffness increase[END_REF]. This result enables us to propose that the general of flax/epoxy behavior comprises three zones, more or less marked according to the arrangement and the quality of the reinforcement consist of the linear elastic strain; the strain non-elastic probably viscous-elastic-plastic due to the collective reorientation of the fibril of the secondary wall of flax fibers (we showed the irreversible character of this zone by a creep /recovery test [START_REF] Poilâne | Optimisation d'un pré-imprégnés lin/époxy industriel[END_REF]), and the resumption of stiffness.

The three zones are perfectly identifiable on figure 4 where the behavior of the UD is deferred. The third zone is rarely observed in practice. The actual position of our actual knowledge does not allow us to know if it is elastic or not.

The influence of the torsion of the roving is not as harmful as what we had expected. The WOVEN, which is the only product of figure 2.a. comparative containing flax yarns, gives the best results. It is thought that the most important reason comes from a good separation of elementary flax fibers that cause of by wet spinning. In the plant, the flax fibers are gathered in packages called "technical fiber" by the professionals of flax. The chemical links which connect fibers between them, provided by pectins, are difficult to break. This affects the quality of the fiber/resin interface by limiting the access of the resin to the walls of the elementary fibers. Our assumption on poor performance of the UD is that the elementary fibers are rarely separated and that there are too much technical fibers. This is confirmed by the observations of figure 5 obtained on the topographies of failure appearance of WOVEN and the UD. The twisted yarn shows that elementary fibers are well separated and coated with resin. On the other hand the UD shows that the technical fibers are well coated with resin but in which the resin does not penetrate. It is advisable in all the cases to facilitate access of the resin to the external walls of the elementary fibers. It is the object of comparative between the textile treatments. The three treatments tested, leaching, bleaching and mercerizing, in general improved the properties of the composite. The MERCERIZED, that already identified for improve the interface flax fibers/resin [START_REF] Van De Weyenberg | Flax fibers as reinforcement for epoxy composites[END_REF], which offers the best performances. The textile treatments are used to facilitate the ennobled (dry filling), like the dyeing for example. Their role is to support the penetration of the products of ennobled by removing pectins which have a hydrophobic effect. In addition, the cellulose chains are also attacked by these treatments. A compromise is thus to find in the parameters of each treatment to separate elementary fibers without degrading them. The comparison between the properties of LEACHING, which are higher of 20% than those of the UNBLEACHED, and of the LEACHING WOVEN, which are lower to him by 20%, makes it possible to confirm this idea of compromise. The leaching made it possible to separate elementary fibers, can observe it on figure 6, but this one was too destructive. It should be added, to be complete, that the LEACHING WOVEN consists of short fibers, which means that the tow, and nothing sure that the properties of these fibers which have statistically identical properties to that of the long fibers used in the other products. In addition we observe on the photograph of right-hand side a zone dries on the weft yarn. This phenomenon is current in the case of the studied twill and does not seem correlated with the mechanical properties obtained. However, the dry zones, just as the cavities, constitute an indicator of performance of the impregnation and elaboration of the laminated plates. We are not currently to be in a position to say if the treatments of fabrics act on the number of cavities within the composite. It is a track to be studied undoubtedly.

CONCLUSION

The objective of work consists in understanding the origin of the non-elastic phenomenon observed in the behavior of flax/epoxy prepreg and analyzing the influence of the many parameters occurring in their elaboration. The products tested are elaborated and treated industrially.

We study the influence of the torsion of the roving by comparing the product reference, A WOVEN, with a UD, a SWEN and a MAT having a privileged orientation. These four products are intended for the uni-directionnel reinforcement. The study confirms the non-elastic behavior of the composite flax/epoxy. We propose to generalize this behavior in three zones: a linear elastic strain, a non-elastic strain associated with a loss of stiffness, a resumption of stiffness. WOVEN offers the best results. We postulate that the elimination of pectins, facilitated by the wet spinning, allows in spite of the morphology of the reinforcement in twisted roving, a better effectiveness of the interface fibers/resin than for the other products tested.

We studies the influence of the textile treatment by comparing the reference product, a 2/2 twill UNBLEACHED made up of yarn worked out in semi-wet spinning with the same WASHED fabric, BLEACHED, MERCERIZED and with a 2/2 twill constituted by leached yarns. The MERCERIZED offers the best performances and the leaching weave fabric presents of the performances lower than fabrics untreated. We think that beyond the physicochemical action of the treatment, the important thing is to optimize the multiple parameters concerned in the treatments (proportioning, temperature, duration, containment…). A compromise is to be found between the elimination of pectic cements, which obstruct the impregnation of the resin, and cellulose attacks it, which deteriorates the intrinsic performances of elementary fibers. A treatment ensymatic to the pectinase will be implemented to confirm this assumption. This treatment is primarily witness of pectins without attacking to cellulose.

Figure 1 .

 1 Figure 1. Influence of the roving torsion, stress/strain and evolution of the module, average test.

Figure. 2 .

 2 Figure. 2. Influence of treatment, stress /strain curve and evolution of the modulus, average test.

Fig. 3 .

 3 Fig. 3. Behavior of elementary fiber of flax Figure. 4. Behavior of the flax/ epoxy composite

Figure 5 .

 5 Figure 5. Observation on the failure topography, a WOVEN yarn on the left, a UD technical fiber on the right.

Figure 6 .

 6 Figure 6. Observation of a section of specimen after traction, LEACHING on the left, LEACHIN WOVEN on the right.

Table 1 .

 1 Properties of flax fibers compared with those of current fibers.

		Flax	Glass E	Aramide	Carbon HR
	D (s.u)	1.53	2.54	1.45	1.94
	E (GPa)	58	72	135	588
	σ R (MPa) ε R (%)	1340 3.3	2200 3	3000 4.5	3920 0.7
	E / D (GPa)	38	28	93	303
	σ R / D (MPa)	875	866	2100	2020

Table 2 .

 2 Products to test the influence of the torsion of the roving; the reference material is WOVEN.

	Name	weight of	fibres	textile treatment	warp/weft ratio	model of the texture
		fabric	type			aspect
		(g/m²)
						²
	UD	78	long fibres	nothing	nothing
	SEWN	125	long fibres	nothing	?
	MAT	151	short fibers	nothing	nothing
	WOVEN	117	long fibers	yarn leaching (wet spinning) before weaving	1/19

Table 3 .

 3 The 2/2 twill prepregs to test the influence of the textile treatment; the reference is the UNBLEACHED twill.

	Name	weight of	Fibers type	textile treatment warp/weft	model of the
		fabric			ratio	texture
		(g/m²)				aspect
	UNBLEACHED	300	longue fibers	nothing	1
	YARNS LEACHED	300	short fibers	leaching of yarns (wet spinning) before weaving	1	idem
	LEACHED	300	longue fibers	leaching of fabric	1	idem
	MERCERIZED	300	longue fibers	mercerization of fabric	1	idem
	BLEACHED	300	longue fibers	bleaching of fabric	1	idem

Table 4 .

 4 Torsion Influence of the roving, properties and results.

	type/properties	UD	TISSÉ	COUSU	MAT
	Weight of fabric (g/m²)	78	125	151	117
	Number of plies	14	11	10	12
	V F (%)	44	50	56	49
	D	1,27	1.28	1.3	1,28
	E (GPa)	26.3	28	27.9	16
	finale stiffness (MPa)	19,5	14.1	-	6.4
	σ R (MPa) ε R (%)	259 1.4	311 2	156 1.2	146 1.7

Table .

 .

			5. Influence treatment, properties and results.
	type/properties	WOVEN	LEACHED	BLEACHING MERCERIZE UNBLEACHED
						YARNS
	weight of fabric	300	300	300	300	300
	(g/m²)
	folds numbers	7	7	7	7	7
	thickness (mm)	2.66	2.61	2.65	2.65	2.72
	V F (%)	63	63	66	66	67
	E (GPa)	124	142	141	143	108
	finale stiffness	4.19	6.14	6.85	7.24	3.52
	(MPa)
	ε R (MPa)	98	117	124	127	81
	ε R (%)	2.05	1.75	1.7	1.64	1.98