Nisrine Ghadban

Paul Honeine

Farah Mourad-Chehade

Joumana Farah

Clovis Francis

GOSSIP ALGORITHMS FOR PRINCIPAL COMPONENT ANALYSIS IN NETWORKS

Keywords: Gossip averaging, principal component analysis, in-network processing, adaptive learning, distributed processing

This paper deals with the issues of the dimensionality reduction and the extraction of the structure of data using principal component analysis for the multivariable data in large-scale networks. In order to overcome the high computational complexity of this technique, we derive several in-network strategies to estimate the principal axes without the need for computing the sample covariance matrix. To this aim, we propose to combine Oja's iterative rule with average gossiping algorithms. Gossiping is used as a solution for communication between asynchronous nodes. The performance of the proposed approach is illustrated on time series acquisition in wireless sensor networks.

INTRODUCTION

Extracting information from multivariable data is a difficult task. For a better understanding of data compression and denoising, one may identify relevant patterns. Among many existing techniques, Principal Component Analysis (PCA) [START_REF] Jolliffe | Principal Component Analysis[END_REF][START_REF] Abdi | Principal component analysis[END_REF] is probably the most widely used. It is also called the discrete Karhunen-Loève transform in the signal processing literature and the Hotelling transform in multivariate analysis. While the scope of the PCA is very wide, it provides a powerful tool in the context of sensor networks. It is investigated to extract features from noisy samples [START_REF] Chitradevi | Designing an efficient PCA based data model for wireless sensor networks[END_REF], compress and denoise time series measurements [START_REF] Rooshenas | Reducing the data transmission in wireless sensor networks using the principal component analysis[END_REF], as well as for the detection of intrusion [START_REF] Ahmadi Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF] or anomaly [START_REF] Huang | In-network pca and anomaly detection[END_REF].

The PCA consists in determining a subspace that retains the largest variance of the data. This subspace is spanned by the most relevant principal axes. The conventional PCA algorithm requires the eigen-decomposition of the sample covariance matrix. In network context, sending all time series to a fusion center is not scalable. Moreover, the computational complexity is cubic with the size of the dataset. Several techniques have been proposed to overcome these drawbacks, such as [START_REF] Gill | Methods for modifying matrix factorizations[END_REF][START_REF] Bunch | Updating the singular value decomposition[END_REF], however these techniques still present a This work is supported by Université de Technologie de Troyes, Université Libanaise, and Région Champagne-Ardenne (grant "WiDiD"). high computational cost. In [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF], the authors used the power iteration method to estimate the most relevant principal axis. This method requires the computation of the sample covariance matrix, which is inappropriate for networks.

On the other hand, synchronization in a decentralized network is a very crucial problem, more specifically for the above algorithms. Indeed, nodes communicate by exchanging each other's information. Therefore time synchronization is required to insure the same time scale for the nodes' local clocks. It demands oscillators to emit signals at the same frequency. Designing synchronization algorithms has been investigated in many research works [START_REF] Lu | A time-varying complex dynamical network model and its controlled synchronization criteria[END_REF][START_REF] Sivrikaya | Time synchronization in sensor networks: a survey[END_REF]. However, these techniques necessitate a significant consumption of resources in the network which are very often limited. To overcome this problem, distributed and asynchronous algorithms have been investigated. In such algorithms, each node exchanges information with only one neighbor in a time slot. The gossip algorithms, also called epidemic algorithms, are based on an asynchronous information exchange [START_REF] Boyd | Randomized gossip algorithms[END_REF][START_REF] Asensio-Marco | Fast average gossiping under asymmetric links in wsns[END_REF]. They allow a distribution of the computational burden since at each time instance, a node communicates with only one randomly chosen neighbor. Gossip algorithms are simple, scalable, and robust to node failures, message loss, and transient network disruptions.

In this paper, we propose to estimate the principal axis using the gossip approach, without the need of computing the sample covariance matrix. To this end, we revisit Oja's rule, initially described in [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF] and studied more recently in [START_REF] Honeine | Online kernel principal component analysis: a reduced-order model[END_REF] for nonlinear PCA with kernel-based machines. Within the network settings, we combine this rule with the averaging gossip where sensors cooperate between each other to estimate the principal axis. Communication between nodes is governed by the gossip concept in order to relax constraints on the reception synchronism. The relevance of the proposed algorithms is shown in the context of Wireless Sensor Networks without restricting the large spectrum of applications that can take advantage of the presented study in this paper.

The rest of this paper is organized as follows. Next section describes the gossip algorithm for averaging applications. In Section 3, we present strategies for PCA. We describe the gossip algorithm for PCA in Section 4. Section 5 provides experimental results and discussions, whereas Section 6 concludes the paper.

AVERAGING GOSSIP

Consider a network of N nodes. In a centralized network, the nodes communicate with a fusion center (FC); in a decentralized network, the nodes provide in-network processing, either in a noncooperative way by a given routing path, or in a cooperative way where each node communicates with its neighboring nodes. Let V k denote the set of indices of the neighboring nodes to agent k, i.e., the nodes that are directly connected to k. We consider that k is not adjacent to itself, that is to say k / ∈ V k . Let x k be the (p × 1) vector of measures collected by the node k, p being the measurements' dimension, and let X ⊂ R p be the space of these collected data (assumed to be zero-mean), with the conventional inner product x ⊤ k x l for any x k , x l ∈ X. Let w k be some feature extracted from the measurements and corresponding to the node k. We are interested, in this section, in the case where the nodes estimate the average of

w k , k = 1, • • • , N .
In a centralized network, the nodes send their features/states w k , k = 1, • • • , N to a FC where the average is computed via the formula 1 N N k=1 w k . In a noncooperative strategy, nodes communicate according to a routing process. Each node k receives an estimate from the previous node, adds 1 N w k to it, and transmits it to the following node. This technique requires a well-defined routing scheme and a synchronous transmission/reception of the information exchange. Gossip algorithms provide an elegant asynchronous solution to overcome these issues, without the need for a FC.

Gossip algorithms use an asynchronous protocol with low computational complexity, described as follows. At iteration t, an arbitrarily chosen node k picks out randomly one of its neighbors, designed by node l. Node l sends its state to node k which produces a new state according to

w k,t = (1 -ǫ)w k,t-1 + ǫw l,t-1 , (1)
for a mixing parameter ǫ ∈ [0, 1]. The states of the other nodes remain unchanged, namely w j,t = w j,t-1 for all j = k. This is the asymmetric gossip, described in details in [START_REF] Fagnani | Asymmetric randomized gossip algorithms for consensus[END_REF]. Unlike the asymmetric gossip, the symmetric gossip described in [START_REF] Boyd | Randomized gossip algorithms[END_REF] uses a symmetric communication protocol between nodes. In such case, both nodes k and l exchange their states to compute the update according to the mixing parameter ǫ ∈ [0, 1], as follows:

w k,t = (1 -ǫ)w k,t-1 + ǫw l,t-1 , (2)
w l,t = ǫw k,t-1 + (1 -ǫ)w l,t-1 . (3)
The states of the other nodes remain unchanged. We note that this update preserves the total sum, and hence the mean of the nodes' estimates [START_REF]Randomized consensus algorithms over large scale networks[END_REF], since we have N k=1 w k,t = N k=1 w k,t-1 . Note that this characteristic does not apply to the asymmetric gossip. This is the main cause why symmetric gossip outperforms asymmetric gossip, but with the cost of a much higher energy consumption for communication and computing.

PRINCIPAL COMPONENT ANALYSIS

In this section, we review previous works for PCA in networks. We denote by the scalar value y w,k = w ⊤ x k the inner product associated with the orthogonal projection of any x k ∈ X onto the vector w ∈ X and by y w the scalar random variable taking the values y w,k . The ultimate goal would be to compute w, optimally for some given cost function and according to the network topology, and as a consequence to keep afterwards only y w,k (and w), for k = 1, ..., N , which allows to represent efficiently the data x k .

Centralized strategy

Here, the network is assumed to be centralized, with nodes connected to the FC, to which they send their data without any in-networks processing. In order to extract the first principal axis, the FC maximizes the variance of the projected data, namely max w E(y 2 w ), where E(•) is the expectation over the density of the input data. By taking the empirical estimation of the variance of the projected data with respect to the available samples, x 1 , x 2 , . . . , x N , we get max w w ⊤ Cw, where

C = 1 N N k=1 x k x ⊤
k is the covariance of the measured data. The problem takes the form Cw = λw. This is the wellknown eigen-decomposition problem where w is the eigenvector associated with the eigenvalue λ of C.

Having the data, x 1 , . . . , x N , the FC solves the eigendecomposition problem and associates the eigenvector with the largest eigenvalue to the principal axis, denoted w * in the following. Resolving of the eigen-decomposition problem has a computational complexity O(p 3 ). There is also the communication complexity, which is O(N p) over a distance O(1). Such configuration is not scalable and inappropriate for network applications. In the following, we propose several strategies that allow reducing the computational complexity by avoiding the covariance matrix computation and the eigendecomposition.

PCA-based distributed approach

The PCA-based distributed approach (PCADID) proposed in [START_REF] Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF] is outlined as follows. The N nodes are divided into d clusters of N i nodes, for i = 1, . . . , d, where

d i=1 N i = N .
Each cluster i deals the p i features (i.e., dimensions) where d i=1 p i = p. In this setting, the PCADID algorithm operates in several steps, as follows. First, for each cluster i, data associated to the cluster are normalized to the range [0 , 1] and then one computes the column-centered matrix X c i . The principal axes of the data in the matrix X c i are extracted using a singular value decomposition X c i = U i Σ i V ⊤ i , or equivalently the eigen-decomposition of the corresponding covariance matrix. Each cluster sends its principal axis to its neighbor cluster according to a routing process. Finally, the principal axes of all the data are given by the union of the principal axes of all the clusters. The PCADID operates through a "divide-to-conquer" scheme in order to reduce the computational complexity of the eigen-decomposition problem. However, it still requires the eigen-decomposition of several matrices, making it inappropriate for large-scale networks.

Noncooperative strategy

We propose to adaptively learn the first principal axis, drawing inspiration from Oja's rule [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF] which is a single-neuron special case of the generalized Hebbian learning [START_REF] Dony | Neural network approaches to image compression[END_REF]. Each node k receives, according to a routing process, an estimate w t-1 from another node, and adjusts it using its own data x k by maximizing y 2 w,k . The quadratic reconstruction error is minimized, namely

J k (w) = 1 4 x k -y w,k w 2 . ( 4
)
Applying the gradient descent technique on J(w) gives the adaptive update rule:

w t = w t-1 + η t (x k y wt-1,k -y 2 wt-1,k w t-1
), (5) where η t is the learning rate.

The noncooperative strategy requires an "instantaneous" estimation of the principal axis at each node and necessitates a synchronous communication protocol. In our previous work, we proposed two cooperative strategies, adapt-then-combine and combine-then-adapt [START_REF] Ghadban | Diffusion strategies for in-network principal component analysis[END_REF][START_REF] Ghadban | Strategies for principal component analysis in wireless sensor networks[END_REF]. Although these strategies do not require a routing process, they use a synchronous communication protocol with a broadcast scheme to diffuse information. Next, we propose to solve this issue by using the gossip for PCA.

GOSSIP FOR PCA

In this cooperative strategy, each node k has access to the information of its neighborhood V k , which is the subset of nodes currently connected to node k. We aim to minimize the cost function N k=1 J k (w), where J k (w) is the cost function at node k, for instance J k (w) = 1 4 x k -y w,k w 2 . The global cost function of node k can be decomposed as

N l=1 J l (w) = J k (w) + N l=1 l =k J l (w). (6)
The first term in the right-hand-side is known at the node under scrutiny, while the second one should be estimated using information from the neighbors of node k; thus, the above summation is restricted to its neighborhood. Moreover, we relax it by constraining the norm between the estimated vector w and the optimal (global) principal axis w * within the neighborhood of node k, with the following regularization:

N l=1 l =k J l (w) ≈ j∈V k b jk w -w * 2 ,
where the parameters b lk control the tradeoff between the accuracy and the smoothness of the solution. Such regularization is also motivated by investigating the second-order Taylor expansion of J l (w), as described in [START_REF] Chen | Diffusion adaptation strategies for distributed optimization and learning over networks[END_REF]. Since gossiping is used between nodes, at each iteration, node k communicates with only one of its neighbors l. Therefore, b lk = ν where ν is a constant and b jk = 0 for all j ∈ V k \{l}. Thus, we have:

N l=1 l =k J l (w) ≈ ν w -w * 2 .
Note that we use w * knowing that we do not have access to its value. We will show in the following how to overcome this problem. Injecting this approximation into (6) allows us to rewrite the global cost function at node k as follows

J glob k (w) = J k (w) + ν w -w * 2 . ( 7
)
In order to minimize the above cost function, the node k applies the gradient descent to J glob k (w) with:

w k,t = w k,t-1 -η k,t ∇ w J glob k (w k,t-1 ). (8)
Here, η k,t is the learning rate for node k at iteration t. Replacing J glob k (w) by its expression in [START_REF] Gill | Methods for modifying matrix factorizations[END_REF], we get:

w k,t = w k,t-1 -η k,t ∇ w J k (w t-1 ) + η k,t ν(w * -w k,t-1 ).
In this expression, ∇ w J k (w t-1 ) is the gradient of the PCAbased cost function. In this case, we get

w k,t = w k,t-1 + η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 + η k,t ν(w * -w k,t-1 ). ( 9
)
This update from w k,t-1 to w k,t involves adding two correction terms to w k,t-1 . We decompose this update rule in two successive steps by including one correction term at a time.

In the following, we show two possibilities of the update.

First possibility

We express the update rule (9) as follows:

φ k,t = w k,t-1 + η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 , w k,t = φ k,t + η k,t ν(w * -w k,t-1 ).
The first step uses local gradient vectors from the neighborhood of the node k in order to update w k,t-1 to the intermediate estimate φ k,t . The second step updates φ k,t to w k,t . This second step is not realizable since the nodes do not know w * . However, each node l has its own approximation for w * , which is its local intermediate estimate φ l,t , and since the parameter ν refers to the neighboring node l, we replace w * by φ l,t . On the other hand, the intermediate value φ k,t at node k is obtained by adapting the information as given by the first step. Thus, it is generally a better estimate for w * than w k,t-1 . Therefore, we further replace w k,t-1 by φ k,t in the second step. Hence, the second step is replaced by:

w k,t = φ k,t + η k,t l∈V k \{k} b lk (φ l,t -φ k,t ) = 1 -η k,t ν φ k,t + η k,t ν φ l,t = 1 -ǫ φ k,t + ǫ φ l,t .
Finally, the update rule for the adapt-then-asymmetric-gossip (ATAG) strategy is:

φ k,t = w k,t-1 + η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 , w k,t = (1 -ǫ)φ k,t + ǫφ l,t .
Motivated by the symmetric averaging gossip where the update is applied for the node k and its neighbor l, we apply the equations of ATAG on the two nodes. Therefore, we obtain the adapt-then-symmetric-gossip (ATSG)technique with the following update rule:

           φ k,t = w k,t-1 + η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 , φ l,t = w l,t-1 + η l,t x l y w t-1 ,l -y 2 wt-1,l w l,t-1 , w k,t = (1 -ǫ)φ k,t + ǫφ l,t , w l,t = ǫφ k,t + (1 -ǫ)φ l,t .

Second possibility

In this strategy, we express the update rule (9) as follows:

φ k,t = w k,t-1 + η k,t ν(w * -w k,t-1 ) w k,t = φ k,t + η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 .
For the same reasons shown in the first possibility, we replace in the first step w * by w l,t-1 and in the second step we replace w l,t-1 by φ l,t . We obtain the update rule for the asymmetric-gossip-then-adapt (AGTA) strategy:

φ k,t = (1 -ǫ)w k,t-1 + ǫw l,t-1 , w k,t = φ k,t + η k,t x k y w t-1 ,k -y 2 wt-1,k φ k,t . (10)
and the symmetric-gossip-then-adapt (SGTA) strategy:

           φ k,t = (1 -ǫ)w k,t-1 + ǫw l,t-1 , φ l,t = ǫw k,t-1 + (1 -ǫ)w l,t-1 , w k,t = φ k,t + η k,t x k y w t-1 ,k -y 2 wt-1,k φ k,t , w l,t = φ l,t + η l,t x l y w t-1 ,l -y 2
wt-1,l φ l,t .

Convergence

All proposed learning rules converge to the first principal axis w * . This can be verified by considering, for instance, the AGTA strategy given in [START_REF] Lu | A time-varying complex dynamical network model and its controlled synchronization criteria[END_REF]. When w k,t for k = 1, • • • , N converges to some state w, in the first equation, we have for

k = 1, • • • , N : φ k,t = (1 -ǫ)w + ǫw = w.
As for the second equation, it comes down to

x k y w,k = y 2 w,k w, namely x k x ⊤ k w = y 2 w,k w.
Averaging over the whole data, we get the well-known eigendecomposition problem of the covariance matrix Cw = w ⊤ Cw w,where the eigenvalue w ⊤ Cw corresponds to the squared output y i that one wishes to maximize. Therefore, the update rule of AGTA converges to the largest eigenvector of the covariance matrix, without the need to compute C. The same demonstration can be applied to the other strategies.

EXPERIMENTATIONS

In this section, we illustrate the performance of the proposed approach and compare it to the algorithm presented in [START_REF] Korada | Gossip pca[END_REF] as well as the aforementioned PCADID algorithm [START_REF] Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF]. In order to provide a fair comparative study, we use the same initial random estimate for all strategies.

As an application to the proposed study, we consider the problem of tracking a gas spread using a wireless sensor network (WSN) [START_REF] Akyildiz | A survey on sensor networks[END_REF]. The sensor k at a position denoted by z k ∈ Z measures, at time t, a gas quantity denoted by x k,t . The region under scrutiny Z = [-0.5, 0.5] × [-0.5, 0.5] is a two-dimensional unit-area. We aim to reduce the time series order of the measurements. The gas diffusion within this region is governed by the following differential equation:

∂G(z, θ) ∂θ -c ∇ 2 z G(z, θ) = Q(z, θ),
where G(z, θ) is the density of gas at the position z and time θ, ∇ 2 z is the Laplace operator, c the conductivity of the medium, and Q(z, θ) corresponds to the added quantity of gas. A gas source placed at the origin is activated from θ = 1 to θ = 15. We use N = 100 sensors uniformly deployed in the region Z, each acquiring a time series of 15 measurements, between θ = 1 and θ = 15.

We consider a predetermined range of communication in the WSN: two nodes are considered as being connected when they are less than s units of distance apart, that is V k = {l : z kz l < s}. In our experiments, we set this threshold to s = 0.38. For the stepsize parameters, we consider η = 0.0025 and ǫ = 0.5. The performance is measured in terms of the angle between the principal axis w * , obtained from the centralized strategy with the eigen-decomposition of the covariance matrix, and the estimate w * ,l at node l, namely Θ i = arccos w ⊤ * ,l w * w * ,l w * . Results are shown in terms of the angle averaged over all the nodes. Figure 1 shows the convergence of the proposed strategies for WSN. It shows that all the proposed strategies outperform the PCADID algorithm [START_REF] Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF]. These learning curves show the benefits of implementing the gossip. The symmetric gossip outperform the asymmetric gossip as expected. Moreover, we note that the order of gossip (AGTA/ATAG or SGTA/ATSG) has almost the same performance. Note that the noncooperative strategy presents fluctuations mainly because of the routing system and it requires a synchronous protocol of communication. Figure 1 shows the stability of the gossip strategies. Although using gossip makes the convergence slow, but it solves the problem of synchronization and communication between nodes.

The algorithm Korada et al. [START_REF] Korada | Gossip pca[END_REF]is a centralized algorithm based on gossiping the PCA. By using a sparsification 1 of C, it is multiplied at each iteration by the last estimate w t , then normalized. The final result is the gossip average of all estimates. Figure 1 shows that the proposed approach outperforms Korada et al. algorithm.

CONCLUSION

In this paper, we studied the issue of estimating the principal axis from PCA in networks. In order to relax constraints on reception synchronism, we proposed to apply gossip in PCA. Experiments are done taking into account the constraints imposed in WSNs. The results showed the relevance of the proposed strategies. Future works will include the extraction of multiple axes.

Fig. 1 .

 1 Fig. 1. Convergence of the proposed strategies for WSN, measured by the average angle.


Nisrine Ghadban

Paul Honeine

Clovis Francis

Farah Mourad-Chehade

Joumana Farah

Strategies for principal component analysis in wireless sensor networks

Keywords: Principal component analysis, wireless sensor network, adaptive learning, distributed processing

This paper deals with the issue of monitoring physical phenomena using wireless sensor networks. It provides principal component analysis for the time series of sensors' measurements. Without the need to compute the sample covariance matrix, we derive several in-network strategies to estimate the principal axis, including noncooperative and diffusion strategies. The performance of the proposed strategies is illustrated in the issue of monitoring gas diffusion.

I. INTRODUCTION

A Wireless Sensor Network (WSN) involves a large number of wireless inexpensive devices, called sensor nodes [START_REF] Akyildiz | A survey on sensor networks[END_REF]. Working in a stand-alone manner, a sensor node has several modules, such as a microprocessor, a radio unit for communication and a sub-system for measurement, linking it to the physical world. All these modules are powered by a system of energy supply, very often a battery. However, with its low price, a sensor node has limited amounts of memory, reduced processing capabilities and low communication capacities. Also, with a non-exchangeable battery, its autonomy imposes constraints of energy saving. In order to reduce energy consumption, sensors only communicate with their neighbors and hence, communication is focused on short distances. Many applications of WSNs exist in several domains, such as military, environmental domain, health [START_REF] Conner | Workplace applications of sensor networks[END_REF], [START_REF] Czubak | On applications of wireless sensor networks[END_REF].

WSNs provide an inexpensive way to monitor physical phenomena such as the temperature field, the atmospheric pressure, the emission of a pollutant or a gas [START_REF] Polastre | Analysis of wireless sensor networks for habitat monitoring[END_REF], [START_REF] Ma | Air pollution monitoring and mining based on sensor grid in london[END_REF]. Estimating fields measured by a WSN has received considerable attention. Using learning methods such as kernel machines, the measurements are combined leading to a regression model which links the measured quantity to the positions where measurements are made [START_REF] Predd | Distributed kernel regression: An algorithm for training collaboratively[END_REF], [START_REF] Honeine | Distributed regression in sensor networks with a reduced-order kernel model[END_REF]. The authors in [START_REF] Bergamo | Scalar field estimation using adaptive networks[END_REF] propose an estimation of the scalar field using adaptive networks. Diffusion strategies, i.e., adapt-then-combine and combinethen-adapt strategies [START_REF] Chen | Diffusion adaptation strategies for distributed optimization and learning over networks[END_REF] are used for this purpose. These strategies have been investigated to solve linear supervised learning problems, where the models are often estimated by least-squares approach. In this paper, we take advantage of these strategies to estimate the principal axis in principal component analysis (PCA) in order to provide unsupervised learning of the time series of sensors' measurements.

The PCA is one of the most popular unsupervised learning techniques, with applications in statistical analysis, data compression and feature extraction [START_REF] Abdi | Principal component analysis[END_REF]. This statistical method defines a set of principal axes that transforms a number of correlated variables into uncorrelated ones, the so-called principal components. The most relevant principal axes retain the largest variance of the data. This technique has been investigated in many applications involving measurements of a sensor network, e.g., to extract features from noisy samples [START_REF] Chitradevi | Designing an efficient PCA based data model for wireless sensor networks[END_REF], or to compress and denoise high-dimensional datasets (time series of measurement) [START_REF] Chen | Algorithm of data compression based on multiple principal component analysis over the wsn[END_REF], [START_REF] Rooshenas | Reducing the data transmission in wireless sensor networks using the principal component analysis[END_REF]. See also [START_REF] Ahmadi Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF] for intrusion detection and [START_REF] Huang | Innetwork pca and anomaly detection[END_REF] for anomaly detection in network traffic with PCA.

The conventional PCA requires the eigen-decomposition of the sample covariance matrix. Sending all time series to a fusion center is not scalable, thus inappropriate for WSNs. Several techniques have been proposed to overcome these drawbacks. In [START_REF] Kargupta | Collective principal component analysis from distributed, heterogeneous data[END_REF], each node sends the estimated principal components, instead of the time series, to the fusion center. In [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF], the authors use the power iteration method to estimate the most relevant principal axe. This method requires the computation of the sample covariance matrix, which is illsuited for WSNs.

In this paper, we propose to estimate the principal axis without the need to compute the sample covariance matrix. To this end, we revisit Oja's rule, initially described in [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF] and studied more recently in [START_REF] Honeine | Online kernel principal component analysis: a reducedorder model[END_REF] for nonlinear PCA with kernelbased machines. Within the WSN settings, we derive several strategies in order to respect WSN constraints, including scalability, energy consumption, as well as computational and memory cost. Starting from the centralized scheme, we first provide a noncooperative strategy for PCA. Then, we study diffusion strategies, where sensors cooperate between each other to estimate the principal axis. Two diffusion strategies are derived, the so-called combine-then-adapt and adapt-thencombine strategies, have been recently investigated in adaptive filtering literature [START_REF] Chen | An adaptive learning algorithm for principal component analysis[END_REF].

The rest of this paper is organized as follows: Next section describes the WSNs and their topologies. In Section III, we describe the proposed strategies for PCA in WSNs, and study their convergence in Section IV. Section V provides experimentation results and discussions, whereas Section VI concludes the paper. II. WIRELESS SENSOR NETWORKS Let N sensors be deployed in a region X, where X ⊂ R 2 or X ⊂ R 3 , for a two-or three-dimensional space. We denote by x i ∈ X the position of sensor i, and y i,θ ∈ R its measurement at time θ of some physical quantity (e.g., temperature or gas concentration measurements). Each sensor collects measurements from θ = 1 to θ = Θ. Let Y ⊂ R Θ be the measurement vector space, with the conventional inner product y ⊤ i y j for any y i , y j ∈ Y, where y i is the vector of entries y i,θ , for 1 ≤ θ ≤ Θ. We denote by the realvalue z = w ⊤ y the inner product associated to the orthogonal projection of any y ∈ Y onto some vector w ∈ Y.

In the following, we present several strategies to estimate the principal axis for PCA in the WSN. Potential applications include feature extraction, compression and denoising these time series (as illustrated in Section I). Let V i denote the index set of the nearest neighboring sensors to sensor i. We consider that x i is adjacent to itself, that is to say i ∈ V i .

III. PCA FOR WSNS

We derive several strategies to extract the principal axis. The communication between sensors and the information interchange depend on the considered strategy. See Fig. 1.

A. Centralized strategy

Here, the network is assumed to be centralized, with sensors connected to a fusion center, to which they send their measured data without any local processing. Having the measurements, y 1 , . . . , y N , the fusion center solves the eigen-decomposition problem Cw = λw, where

C = 1 N N i=1 y i y ⊤
i is the covariance of the data (assumed centered). The eigenvector associated with the largest eigenvalue corresponds to the principal axis, denoted w * in the following.

Since the covariance matrix C is a Θ-by-Θ matrix, the computational complexity of such operation is O(Θ 3 ). Furthermore, the communication of N sensors with a fusion center costs O(N Θ) on an average distance O(1). In order to overcome these communication and computational complexities, we propose next strategies that satisfy the scalability constraint in WSNs, namely without the need of a fusion center and without resorting to the covariance matrix.

B. Noncooperative strategy

Considering sensor-to-sensor scheme [START_REF] Honeine | A decentralized approach for non-linear prediction of time series data in sensor networks[END_REF], we propose to adaptively learn the principal axis, drawing inspiration from Oja's rule [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF]. According to a routing process, a sensor i receives from another sensor the estimation w t-1 , and adjusts it with the following update rule:

w t = w t-1 + η t (y i z i -z 2 i w t-1 ), (1)
where η t is the learning rate which can also change with iteration and z i = w ⊤ t-1 y i . This learning rule converges to the first principal axis w * . In fact, when w t converges to some state w, we have y i z i = z 2 i w, or equivalently y i y ⊤ i w = w ⊤ y i y ⊤ i ww. Averaging over the whole data, we get the well-known eigen-decomposition problem of the covariance matrix Cw = w ⊤ Cw w, where the eigenvalue w ⊤ Cw corresponds to the squared output z i that one wishes to maximize. Therefore, the update rule (1) converges to the largest eigenvector of the covariance matrix, without the need to compute C. Communication between sensors in this strategy costs O(N ) on an average distance O(1/ √ N ).

C. Cooperative strategies

In a cooperative scheme, each sensor k has access to the information of its neighborhood V k , which is the subset of nodes currently connected to node k, including itself. In the following, we study two diffusion strategies.

Combine-then-adapt strategy: At every iteration t, each sensor k combines into φ k,t-1 its estimate with the estimates from its neighborhood, namely {w l,t-1 } l∈V k . Then it updates its own estimate using an update rule similar to (1) where w k,t-1 is substituted by φ k,t-1 . These two steps are described as follows:

φ k,t-1 = l∈V k a kl w l,t-1 w k,t = φ k,t-1 + µ k (y k z k -z 2 k φ k,t-1 ).
Here, µ k is the stepsize for node k, and a kl are convex combiner coefficients satisfying the conditions:

a kl ≥ 0, l∈V k a kl = 1. (2)
Adapt-then-combine strategy: In this strategy, at every iteration t, each sensor k updates its own estimate and then fuses it with those from its neighbors into φ k,t-1 . These two steps are described by the following update rule:

φ k,t = w k,t-1 + µ k (y k z k -z 2 k w k,t-1 ) w k,t = l∈V k a kl φ l,t .
Here, µ k denotes the stepsize for node k, and a kl are convex combiner coefficients satisfying the conditions (2).

IV. CONVERGENCE ANALYSIS AND PARAMETER STUDY

Both adapt-then-combine and combine-then-adapt strategies are "diffusion" strategies since the combination step allows information to diffuse through the network. These strategies have fundamentally the same structure. The difference lies in which variable we choose to correspond to the updated estimate. In the adapt-then-combine case, the estimate is the result of the combination step, whereas in the combine-thenadapt case the estimate is the result of the adaptation.

In the following, we provide a convergence analysis and study the choice of the parameters for the strategies proposed in the previous section.

A. Stepsize in the noncooperative strategy

In the noncooperative strategy, the stepsize parameter η t is chosen to be sufficiently small to ensure convergence. It should be smaller than the inverse of the largest eigenvalue [START_REF] Chen | An adaptive learning algorithm for principal component analysis[END_REF]. Unfortunately, the eigenvalues are usually unknown. For appropriate convergence, the stepsize η t should decrease at each iteration such as η t = η 0 /t, where η 0 is a positive constant parameter. An alternative approach is the "searchthen-converge" scheme [START_REF] Honeine | Online kernel principal component analysis: a reducedorder model[END_REF], with η t = η0 1+t/τ . In this case, the delay parameter τ determines the duration of the initial search phase, with η t ≃ η 0 when t ≪ τ , before a converge phase where η t decreases as η 0 /t when t ≫ τ .

B. Stepsize for the adaptation in diffusion strategies

In the diffusion strategies, the stepsize parameter µ k is chosen small to ensure the convergence. It can also decreases with iterations, such as with µ k,t = µ k,0 1+t/τ . We notice that this stepsize converges toward zero as t -→ ∞. Therefore it turns off the adaptation. This choice is not suitable for continuous learning. For this reason, we choose a constant stepsize to ensure continuous adaptation.

C. Coefficients for the combination in diffusion strategies

The combiner coefficients a kl used in the diffusion strategies determine the weight that each node k assigns to the estimates from its neighbors l ∈ V k . These coefficients need to satisfy the convexity conditions [START_REF] Conner | Workplace applications of sensor networks[END_REF]. By revisiting the suggestions given in the adaptive filtering literature [START_REF] Sayed | Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior[END_REF], we consider the following rules to define these coefficients:

1) Averaging rule:

a kl = 1/n k , when l ∈ V k ; 0, otherwise .
2) Laplacian rule:

a kl = ⎧ ⎨ ⎩ 1/n max , if l ∈ V k \{k}; 1 -(n k -1)/n max , if k = l; 0, otherwise .
3) Metropolis rule:

a kl = ⎧ ⎨ ⎩ 1/ max{n k , n l }, if l ∈ V k \{k}; 1 -j∈V k \{k} a kj , if k = l; 0, otherwise.
In these expressions, n k denotes the degree of node k, i.e., the size of its neighborhood, and n max denotes the maximum degree across the network, i.e., n max = max 1≤k≤N n k .

V. EXPERIMENTS

In order to illustrate the results of this work, we consider the framework proposed in [START_REF] Ghadban | Mobility using first and second derivatives for kernel-based regression in wireless sensor networks[END_REF], where we have the diffusion of a gas in a two-dimensional space X = [-0.5, 0.5] × [-0.5, 0.5]. This distribution is governed by the differential equation

∂G(x, θ) ∂θ -c ∇ 2 x G(x, θ) = Q(x, θ),
where G(x, θ) is the density of gas depending on the position x and time θ, ∇ 2

x is the Laplace operator, and Q(x, θ) corresponds to the added quantity of gas and c the conductivity of the medium. A gas source placed at position (-0.3, 0.3) is activated from θ = 1 to θ = 15. We use N = 100 sensors deployed uniformly in the region under scrutiny X, each acquiring a time series of 15 measurements, from θ = 1 to θ = 15.

We consider a predetermined range of communication in the WSN. In this case, two nodes are connected when they are less than s units of distance apart, that is

V k = {l : ∥x k -x l ∥ < s}.
In our experiments, we set this threshold to s = 0.15. For the stepsize parameters (see Section IV), we consider as a reference the constant stepsize η 0 = 0.005. We compare the results with the stepsize given by η t = η0 1+t/τ , where the delay parameter τ is given by the third of the total number of nodes. We also use the same stepsize value for the cooperative strategies, namely µ k = 0.005.

In order to provide a comparative study, we use the same initial random estimate for all strategies. The performance is measured with the angle between the principal axis w * , obtained from the centralized strategy with the eigen-decomposition of the covariance matrix, and the estimate w l at node l, namely

arccos w ⊤ l w * ∥w l ∥∥w * ∥ . (3)
Fig. 2 shows the convergence of each strategy, with the angles averaged over the N nodes. These learning curves show that the noncooperative strategy is more suitable when a variable stepsize is used, as opposed to the constant one where convergence is not guarantied. This strategy is outperformed by any diffusion strategy, independently of the combination rule. The analysis of the diffusion strategies shows that the adapt-then-combine strategy performs slightly better that the combine-then-adapt strategy. In terms of combination rules, the averaging rule provides a slight refinement. The overall efficiency of the diffusion strategies is shown in terms stability.

VI. CONCLUSION

In this paper, we studied the issue of estimating the principal axis from PCA in WSNs. Under constraints imposed in WSNs, we proposed several strategies including noncooperative and diffusion strategies. Experimental results showed the relevance of these strategies, and illustrated that the adapt-then-combine strategy with the averaging rule outperforms the other strategies. As a futur work, we will extend this technique to the case of multiple principal axis by using a generalized Hebbian approach. We will also study the use of the spatial information.

Fig. 1 .

 1 Fig.1. Illustration of information processing within the proposed strategies for PCA in WSNs. The right-hand scheme illustrates both diffusion strategies: combine-then-adapt and adapt-then-combine.

Fig. 2 .

 2 Fig.2. Convergence over 200 iterations of the proposed strategies, measured by the angle given in (3) and averaged over all the nodes. The confronted strategies are the noncooperative strategy, with either a constant stepsize η 0 or a decreasing one ηt = η 0 1+t/τ , and both combine-the-adapt (CTA) and adaptthen-combine (ATC) diffusion strategies with several rules for combination defined in Section IV-C.

ACKNOWLEDGMENT

This work is supported by Université de technologie de Troyes, Université Libanaise, and Région Champagne-Ardenne (grant "WiDiD").


Sandy Mahfouz

Farah Mourad-Chehade

Paul Honeine

Joumana Farah

Hichem Snoussi

RIDGE REGRESSION AND KALMAN FILTERING FOR TARGET TRACKING IN WIRELESS SENSOR NETWORKS

Keywords: radio-fingerprinting, Kalman filter, ridge regression, RSSI, tracking, WSN

This paper introduces an original method for target tracking in wireless sensor networks that combines machine learning and Kalman filtering. A database of radio-fingerprints is used, along with the ridge regression learning method, to compute a model that takes as input RSSI information, and yields, as output, the positions where the RSSIs are measured. This model leads to a position estimate for each target. The Kalman filter is used afterwards to combine the model's estimates with predictions of the target's positions based on acceleration information, leading to more accurate ones.

I. INTRODUCTION

Recently, wireless sensor networks (WSNs) have become a major research field, since they can be used for many everyday applications [START_REF] Gomez | Wireless home automation networks: A survey of architectures and technologies[END_REF], [START_REF] Honeine | Wireless sensor networks in biomedical: body area networks[END_REF]. One interesting application of WSNs is target tracking [START_REF] Li | Detection, classification, and tracking of targets[END_REF], that consists of estimating instantly the position of a moving target. Target tracking can be viewed as a sequential location estimation problem, that requires a real-time location estimation algorithm. Typically, sensors broadcast signals in the network, while targets collect these signals for location estimation. Several types of measurements can be considered, such as received signal strength indicators (RSSIs) [START_REF] Lau | Enhanced rssi-based real-time user location tracking system for indoor and outdoor environments[END_REF], that has proven to achieve acceptable performance, with no extra hardware.

Many RSSI-based tracking algorithms have been proposed in the literature. For instance, the authors of [START_REF] Dias | Cooperative target tracking using decentralized particle filtering and rss sensors[END_REF] provide a tracking technique using a particle filter with the exact RSSI channel model. Using also signals powers, the authors of [START_REF] Mourad | Robust boundederror tracking in wireless sensor networks[END_REF] propose a target tracking method based on the interval theory and connectivity measurements. These methods are more robust than the ones using the exact channel model. However, the number and the positions of the sensors in the network highly affect the performance of such methods.

In other contexts, RSSI-based methods have been proposed for nodes localization in WSNs. Such methods investigate observation information, but do not take advantage This work is supported by the Champagne-Ardenne Region in France. of nodes mobility. Radio-fingerprinting [START_REF] Lin | Performance comparison of indoor positioning techniques based on location fingerprinting in wireless networks[END_REF] is an interesting RSSI-based localization approach, since it avoids estimating the RSSI channel model. We have recently proposed in [START_REF] Mahfouz | Kernel-based localization using fingerprinting in wireless sensor networks[END_REF] and [START_REF]Decentralized localization using fingerprinting and kernel methods in wireless sensor networks[END_REF] two robust localization methods using radiofingerprinting in WSNs, by taking advantage of kernel methods in machine learning. In [START_REF] Chan | Using wi-fi signal strength to localize in wireless sensor networks[END_REF], the authors perform tracking by combining estimates obtained using the weighted K-nearest neighbor (WKNN) algorithm, which is a wellknown RSSI-based fingerprinting algorithm, with acceleration information using the Kalman filter with a second-order state-space model.

In this paper, we provide a new method for target tracking in WSNs by combining radio-fingerprinting and accelerometer information. Our method consists of constructing a database of radio-fingerprints used with the ridge regression learning method [START_REF] Saunders | Ridge regression learning algorithm in dual variables[END_REF] to define a kernel-based model, whose input is the RSSI vector and whose output is the corresponding position. Then, RSSI measures collected by a moving target are used with the computed kernel-based model to get a position estimate. This estimate is combined with the acceleration information, by means of a Kalman filter [START_REF] Welch | An Introduction to the Kalman Filter[END_REF], to achieve better accuracy. The rest of the paper is organized as follows. The proposed tracking method is described in Section II. Section III describes the kernel-based model's definition, while Section IV shows how the Kalman filter is applied to make use of the accelerometer information. In Section V, we examine the performance of the proposed method. Finally, Section VI concludes the paper.

II. DESCRIPTION OF THE TRACKING METHOD

Consider an environment of D dimensions, with D = 2 or 3, and N s stationary sensors having known locations, denoted by s i , i ∈ {1, . . . , N s }. In the following, all coordinates are 1-by-D vectors. For the sake of clarity, only one target with the unknown position x(k) is considered in this paper, k being the current time step.

The first step in the proposed method consists of setting reference positions along the network where RSSI measurements are collected, to construct a database of radiofingerprints. To this end, N p reference positions, denoted by p ℓ , ℓ ∈ {1, . . . , N p }, are generated uniformly or randomly in the studied region. The stationary sensors broadcast signals with the same initial power, and signals are assumed to be received at all reference positions. A sensor is temporarily placed at each reference position p ℓ to measure the RSSI of the broadcasted signals. Let ρ ℓ = (ρ s1,p ℓ . . . ρ sN s ,p ℓ ) ⊤ be the vector of RSSIs sent by all N s sensors and received at the position p ℓ . This way, a database of N p pairs (ρ ℓ , p ℓ ) is obtained, where ℓ ∈ {1, . . . , N p }. The objective now is to define a function ψ : IR Ns → IR D , that associates to each RSSI vector ρ ℓ the corresponding position p ℓ . Kernel methods [START_REF] Hofmann | Kernel methods in machine learning[END_REF] provide an elegant framework to find ψ (•), as it will be shown in the following section.

Note that the database construction and the computation of ψ(•) are performed only once, before the tracking phase. Once ψ(•) is defined, the target is able to perform all tracking computations and determine its own position. Indeed, consider a target in the network. This target collects, while moving, the RSSIs of the signals received from the N s sensors, at each time step k, and stores them into a vector ρ(k). An estimate of the target's position is obtained using the defined model ψ (•), and the estimated coordinates are then given by:

z(k) = ψ(ρ(k)). ( 1
)
Each target is assumed to be equipped with an accelerometer that yields its instantaneous accelerations over the D coordinates. The estimated position z(k) is then combined with the accelerometer information to obtain a more accurate estimate using the Kalman filter [START_REF] Welch | An Introduction to the Kalman Filter[END_REF], as will be explained in Section IV.

III. COMPUTATION OF ψ (•)

Based on the information gathered by the target from the stationary sensors (i.e., the constructed database), the objective is to define the model ψ (•), that associates to each RSSI vector ρ ℓ the corresponding position p ℓ , ℓ ∈ {1, . . . , N p }. Determining ψ (•) requires solving a nonlinear regression problem, as we show in the following.

Let

ψ(•) = (ψ 1 (•) . . . ψ D (•))
, where each ψ d (•), d ∈ {1, . . . , D}, estimates the d-th coordinate in p ℓ = (p ℓ,1 . . . p ℓ,D ), for an input ρ ℓ . Let P = (p ⊤ 1 . . . p ⊤ Np ) ⊤ ; the matrix P is then of size N p -by-D, having p ℓ,d for the (ℓ, d)-th entry. In the following, we denote p ℓ by P ℓ, * , and the d-th column of P by P * ,d . The vector P * ,d holds now all N p points for a fixed coordinate d. The classic ridge regression is considered in this section, where D optimization problems are set separately to estimate the D models ψ d (•) [START_REF] Saunders | Ridge regression learning algorithm in dual variables[END_REF]. Indeed, each function ψ d (•) must minimize the mean quadratic error between the model's outputs ψ d (ρ ℓ ) and the desired outputs p ℓ,d :

min ψ d ∈H 1 N p Np ℓ=1 (p ℓ,d -ψ d (ρ ℓ )) 2 + η ψ d 2 H , (2)
where η is a regularization parameter that controls the tradeoff between the training error and the complexity of the solution. According to the representer theorem [START_REF] Hofmann | Kernel methods in machine learning[END_REF], the optimal function can be written as follows:

ψ d (•) = Np ℓ=1 α ℓ,d κ(ρ ℓ , •), (3)
where κ : IR Ns × IR Ns → IR is a reproducing kernel, and α ℓ,d , ℓ ∈ {1, . . . , N p }, are parameters to be determined. Let us denote by α the N p -by-D matrix whose (ℓ, d)-th entry is α ℓ,d . Let then α * ,d denote its d-th column, and α ℓ, * its ℓ-th row. By injecting (3) in ( 2), we get a dual optimization problem in terms of α * ,d , whose solution is given by taking its derivative with respect to α * ,d and setting it to zero. One can easily find the following form of the solution:

α * ,d = (K + ηN p I Np ) -1 P * ,d ,
where I Np is the N p -by-N p identity matrix, and K is the N p -by-N p matrix whose (i, j)-th entry is κ(ρ i , ρ j ), for i, j ∈ {1, ..., N p }. For an appropriate value of the regularization parameter η, the matrix between parenthesis is always nonsingular.

It is easy to see that the same matrix (K + ηN p I Np ) needs to be inverted in order to estimate each coordinate. Therefore, to reduce the computational complexity, all D estimations are collected into a single matrix inversion problem, as follows:

α = (K + ηN p I Np ) -1 P . ( 4
)
We then define a model that allows us to estimate all D coordinates at once, using equation ( 3) and the definition of the vector of functions ψ(•), as follows:

ψ(•) = Np ℓ=1 α ℓ, * κ(ρ ℓ , •). (5)

IV. TRACKING ENHANCEMENT USING THE KALMAN FILTER

In this section, we propose to combine the position estimates obtained using the already-defined kernel-based model with accelerometer information by means of a Kalman filter [START_REF] Welch | An Introduction to the Kalman Filter[END_REF]. Challenges arise when the target does not follow a predictable path, making it difficult to find a model to describe its motion. We propose here to use a third-order state-space model to describe the target's motion. Such model considers that the target's accelerations vary linearly between any two consecutive time steps k -1 and k with a slope equal to γ(k)-γ(k-1) ∆t , where γ(k) = (γ 1 (k) . . . γ D (k)) is the acceleration vector at time step k, and ∆t the time period separating two consecutive time steps. According to this assumption, the velocity vector ν(k) of the target at time step k is estimated recursively by:

ν(k) = ν(k-1)+γ(k-1) ∆t+ γ(k) -γ(k -1) ∆t ∆t 2 2 . (6)
Then, by taking the primitive integral of ( 6), the position of the target can be written as follows:

x(k) = x(k -1) + ν(k -1) ∆t + γ(k -1) ∆t 2 2 + γ(k) -γ(k -1) ∆t ∆t 3 6 . (7)
For small values of ∆t, one can find an accurate approximation of x(k) using this model. The target is assumed to be at a fixed known position x(0) at the beginning of the tracking, having ν(0) and γ(0) null. We add a random D-vector noise θ(k) to the model of ( 7), to take into account the inaccuracies in the measured accelerations. Let the acceleration noises be independent with zero-mean normal distributions, having known variances σ 2 γ d , d = 1, ..., D. Then, θ(k) would have a normal probability distribution, with zero mean and diagonal covariance matrix Q(k), given by the following:

Q(k) = Q(k -1) + Q ν (k -1) ∆t 2 + 11 36 ∆t 4 Diag σ 2 γ
, where Q(0) is null, since there is no uncertainty at time step k = 0, and Q ν (k) is given by:

Q ν (k) = Q ν (k -1) + 3 2 ∆t 2 Diag σ 2 γ ,
where Q ν (0) is also null, and Diag σ 2 γ is the D-by-D diagonal matrix with entries σ 2 γ d , d = 1, ..., D. From [START_REF] Lin | Performance comparison of indoor positioning techniques based on location fingerprinting in wireless networks[END_REF], one can see that the relation between x(k) and x(k -1) is linear, and can be written as follows:

x(k) = x(k -1) + B(k) + θ(k), (8)
where B(k) is a control-input vector depending on the accelerations given by ν(k -1) ∆t + γ(k -1) ∆t 2 2 + γ(k)-γ(k-1) ∆t ∆t 3

6 . Now, to consider the observations, instead of taking the RSSI values, we take the position estimate z(k) given by the kernel-based model, to which we add an additive noise n(k) due to measurements, as follows:

z(k) = x(k) + n(k). (9)
The noise n(k) ∼ N (0, R) is assumed to have a normal distribution, with zero mean and covariance matrix R. To approximate R, a new set of reference pairs is generated, and the positions are estimated using ψ(•). The error on this set is computed and stored into a vector. The matrix R, assumed to be constant over time and for all targets, is then determined by computing the covariance of the error vector. Finally, having the two linear equations ( 8) and ( 9), the Kalman filter can be applied to enhance the solution provided by the technique of Section III. Indeed, the Kalman filter first predicts the unknown position using the previous estimated position and the state-space equation ( 8), as follows:

x-(k) = x(k -1) + B(k),
where x(k-1) denotes the target's position estimated by the Kalman filter at time step k -1, with x(0) assumed to be known. The D-by-D predicted estimation covariance is also updated by T -(k) = T (k -1) + Q(k), where T (k -1) is the final covariance estimation at time step k -1, and T (0) is null since the initial state is known. Finally, the predicted quantities x-(k) and T -(k) are corrected using the observation equation ( 9) as follows:

x(k) = x-(k) + (z(k) -x-(k)) G K (k) T (k) = (I D -G K (k)) T -(k),
where I D is the D-by-D identity matrix, and G K (k) is the optimal Kalman gain given by:

G K (k) = T -(k) (T -(k) + R) -1 .
In summary, a target's position is obtained by combining the position estimated using the model ψ(•) with the one predicted by the accelerometer information.

V. PRACTICAL SIMULATIONS AND RESULTS

In this section, we evaluate the performance of our method on simulated data. We consider a 100m × 100m area, and generate 16 stationary sensors and 100 reference positions uniformly distributed over the area. RSSI values are computed using the known Okumura-Hata model [START_REF] Medeisis | On the use of the universal Okumura-Hata propagation prediction model in rural areas[END_REF] given by:

ρ s i ,p ℓ = ρ 0 -10 n P log 10 s i -p ℓ + ε i,ℓ , (10)
where ρ si,p ℓ (in dBm) is the power received from the sensor at position s i by the node at position p ℓ , that is the i-th entry of the vector ρ ℓ , ρ 0 is the initial power (in dBm) set to 100, n P is the path-loss exponent set to 4, s ip ℓ is the Euclidian distance between the position p ℓ of the considered node and the sensor position s i , and ε i,ℓ is the noise affecting the RSSI measures with σ ρ its standard deviation. For the definition of ψ(•), we consider the Gaussian kernel given by κ(ρ u ,

ρ u ′ ) = exp -ρ u -ρ u ′ 2 2σ 2
, where σ is its bandwidth. The choice of the parameters η and σ is done using the 10-fold cross-validation technique [START_REF] Stone | Cross-validatory choice and assessment of statistical predictions[END_REF], considering

ηN p = 2 v with v ∈ {-20, -19, • • • , -1} and σ = 2 v ′ with v ′ ∈ {1, 2, • • • , 10}.
We consider cosine functions for the accelerations. Then, the trajectory shown on the left plot of Fig. 1 is generated by taking twice the primitive integral of the accelerations. The RSSI values collected by the moving target are calculated using [START_REF] Chan | Using wi-fi signal strength to localize in wireless sensor networks[END_REF]. In order to simulate a noisy environment, both components of σ γ are taken equal to 0.01m/s 2 , and σ ρ is taken equal to 1dB. The left plot of Fig. 1 shows the environment setup, the real trajectory, and the estimated trajectory using the proposed method. Let the estimation error be evaluated by the root mean squared distance between the exact positions and the estimated ones, then the error is equal to 1.07m.

We now compare our method to the one proposed in [START_REF] Chan | Using wi-fi signal strength to localize in wireless sensor networks[END_REF], where the authors perform tracking by combining the fingerprinting-based WKNN algorithm with a Kalman filter. This method consists of finding a position estimate using Fig. 1: Estimation of the trajectory using the proposed method on the left plot, estimation error as a function of the noise on the accelerations on the middle plot and as a function of the noise on the RSSI on the right plot.

WKNN, then enhances it by applying the Kalman filter. The number of neighbors K is taken equal to 8, and a secondorder state-space model is used as in [START_REF] Chan | Using wi-fi signal strength to localize in wireless sensor networks[END_REF]. The algorithm is tested for the same setup as in the left plot of Fig. 1, and using the same values for σ γ and σ ρ . The estimation error obtained is equal to 2.88m. One can see that this error is significantly higher than the one obtained with our method.

We now propose to study the impact of the noises standard deviations σ γ and σ ρ on the estimation error. All results are averaged over 50 Monte-Carlo simulations. First, different percentages of the standard deviation of the acceleration are taken, going from 1% to 10%, along with a fixed σ ρ equal to 5% of standard deviation of the RSSI measures. The standard deviation of the RSSI measured at the reference positions is equal to 10.79dBm, leading to σ ρ = 0.54dBm. The results in the middle plot of Fig. 1 show that the ridge regression combined with the Kalman filter yields the best results, even when the noise on the accelerations is high. We then fix the value of σ γ to 1% of the standard deviation of the acceleration, and consider several percentages of the standard deviation of the RSSI measures, going from 0% to 50%; in other words, σ ρ varies from 0 to 5.40dBm. The right plot of Fig. 1 shows that the ridge regression is highly affected by the noise variations, since RSSI measurements are used for the estimation, while tracking using only accelerometer information is not. However, the ridge regression combined with the Kalman filter yields the best results here as well, proving the effectiveness of using the Kalman filter.

VI. CONCLUSION

In this paper, we proposed a new method for target tracking in WSNs that combines ridge regression and Kalman filtering. Simulation results showed that our method allows accurate tracking, and is proved to be robust in the case of noisy data. It also outperforms tracking using a recently developed method based on the WKNN method and a Kalman filter. Future work will handle further improvements of our method, such as introducing a decentralized version.


Nisrine Ghadban

Paul Honeine

Farah Mourad-Chehade

Clovis Francis

Joumana Farah

DIFFUSION STRATEGIES FOR IN-NETWORK PRINCIPAL COMPONENT ANALYSIS

Keywords: Principal component analysis, network, adaptive learning, distributed processing

This paper deals with the principal component analysis in networks, where it is improper to compute the sample covariance matrix. To this end, we derive several in-network strategies to estimate the principal axes, including noncooperative and cooperative (diffusion-based) strategies. The performance of the proposed strategies is illustrated on diverse applications, including image processing and dimensionality reduction of time series in wireless sensor networks.

INTRODUCTION

The principal component analysis (PCA) is one of the most popular unsupervised learning techniques, with applications in statistical analysis, data compression, and feature extraction [START_REF] Abdi | Principal component analysis[END_REF]. It defines a set of principal axes that transforms a number of correlated variables into uncorrelated ones, the so-called principal components. The most relevant principal axes retain the largest variance of the data. The PCA technique has been investigated in many applications involving multivariate analysis, e.g., data validation and fault detection [START_REF] Wise | Upset and sensor failure detection in multivariable processes[END_REF] and quality control [START_REF] Macgregor | Multivariate statistical methods for monitoring large data sets from chemical processes[END_REF]. It is very useful in a sensor network, where it helps in extracting features from noisy samples [START_REF] Chitradevi | Designing an efficient PCA based data model for wireless sensor networks[END_REF], compressing and denoising times series measurements [START_REF] Chen | Algorithm of data compression based on multiple principal component analysis over the wsn[END_REF][START_REF] Rooshenas | Reducing the data transmission in wireless sensor networks using the principal component analysis[END_REF], as well as for intrusion detection [START_REF] Ahmadi Livani | A PCA-based distributed approach for intrusion detection in wireless sensor networks[END_REF] and anomaly detection [START_REF] Huang | Innetwork pca and anomaly detection[END_REF].

The conventional PCA requires the eigen-decomposition of the sample covariance matrix. Such calculation is inappropriate for networks since sending all data to a fusion center (FC) is not scalable. Moreover, the computational complexity is cubic with the size of the dataset. Several attempts have been made to alleviate this problem, such as in [START_REF] Bunch | Updating the singular value decomposition[END_REF][START_REF] Hall | Merging and splitting eigenspace models[END_REF]; however, these techniques remain computationally inefficient. More recently, in [START_REF] Kargupta | Collective principal component analysis from distributed, heterogeneous data[END_REF], only the principal components have been transmitted to the FC, instead of the whole time series. In [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF], the power iteration method is investigated to estimate This work is supported by the Région Champagne-Ardenne (grant "WiDiD") and the Lebanese University.

the most relevant principal axis. This method requires the computation of the sample covariance matrix, making it inappropriate for in-network processing.

In this paper, we propose to estimate the principal axis without computing the sample covariance matrix. To this end, we investigate two principles, the Oja's neural-based rule [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF], which has been recently investigated in [START_REF] Honeine | Online kernel principal component analysis: a reducedorder model[END_REF] for nonlinear PCA with kernel-based machines, and the information theoretical criterion derived in [START_REF] Plumbley | Lyapunov functions for convergence of principal component algorithms[END_REF]. Within a network setting, we derive several in-network algorithms, including noncooperative and cooperative strategies. In the latter, sensors cooperate by information diffusion in order to estimate the principal axis. Two diffusion strategies are derived, the so-called combine-then-adapt and adapt-then-combine strategies, which have been recently investigated in linear adaptive filtering literature by A. Sayed et al. in [START_REF] Sayed | Diffusion strategies for adaptation and learning over networks: an examination of distributed strategies and network behavior[END_REF]. To the best of our knowledge, the present study is the first work that investigates these adaptation strategies for unsupervised learning, as given here with the PCA.

The rest of this paper is organized as follows. Section 2 presents the network topology and Section 3 describes the proposed strategies for estimating the first principal axis. The strategies are extended in Section 4 for multiple axes extraction. Section 5 provides experimentation results and discussions.

NETWORK MODEL

We consider a connected network of N nodes (agents), where any two nodes are either directly linked if they are neighbors, or through other intermediate nodes. We differentiate between a centralized network where the nodes are connected to a FC, and a decentralized network where the nodes are connected in a noncooperative way by a routing system or in a cooperative way where each node communicates with its neighbors. Figure 1 illustrates these network topologies. Let V k denote the indices set of the neighboring nodes to node k, with k = 1, ..., N , i.e., the nodes that are directly connected to it. We consider that the node k is adjacent to itself, that is to say k ∈ V k . The notation V k would be useful in the following for cooperative networks. In this paper, the nodes estimate the same principal axis w based on some measured information corresponding to a common phenomenon. Let x k be the (p × 1) vector collected by the node k, for k = 1 • • • N , p being the measurements dimension, and let X ⊂ R p be the space of these collected data (assumed to be zero-mean), with the conventional inner product x ⊤ k x l for any x k , x l ∈ X. We denote by the scalar value y w,k = w ⊤ x k the inner product associated with the orthogonal projection of any x k ∈ X onto the vector w ∈ X and by y w the scalar random variable taking the values y w,k , with k = 1, ..., N . The ultimate goal would be to compute w, according to the networks topologies, to keep afterwards only y w,k and w, k = 1, ..., N , allowing us to rebuild optimally the data x k when needed.

IN-NETWORK PCA

In this section, we first expose the conventional strategy for PCA where the network is assumed to be centralized. We derive then our new strategies to extract the first principal axis.

Centralized strategy

Here, the nodes are assumed to be connected to a FC, to which they send their collected data without any local processing. The FC extracts the first principal axis that maximizes the variance of the projected data, namely max w E(y 2 w ), where E(•) is the expectation over the density of input data. By taking the empirical estimation of the latter with respect to the available samples, x 1 , x 2 , . . . , x N , we get max w w ⊤ Cw, where

C = 1 N N k=1 x k x ⊤
k is the covariance of the measured data. It is well known that the problem takes the form Cw = λw. This is an eigen-decomposition problem where w is the eigenvector associated with the eigenvalue λ of C. One can easily see that the latter is the corresponding variance, since

w ⊤ Cw = w ⊤ λw = λw ⊤ w = λ.
Therefore, to maximize the variance of the projected data, the eigenvector associated with the largest eigenvalue must be considered. This eigenvector, denoted by w * , corresponds to the first principal axis and is determined by the FC using the data x 1 , . . . , x N . Since the covariance matrix C is a pby-p matrix, the computational complexity1 of such operation is O(p 3 ). In the following, we propose strategies that allow reducing the computational complexity by avoiding the calculation of the covariance matrix.

Noncooperative strategy

Instead of explicitly investigating the covariance matrix, we propose to adaptively learn the first principal axis, by considering an in-network scheme. To this end, instead of directly maximizing the overall projected variance E(y 2 w ), we consider an "instantaneous" estimation at each node. According to a routing process, each node k receives an estimate w t-1 from another node, and adjusts it using its own data x k by maximizing y 2 w,k . In the following, two different cost functions are explored.

• In the first alternative, the "instantaneous" quadratic reconstruction error is minimized, namely

J k (w) = 1 4 x k -y w,k w 2 . ( 1
)
The gradient of this cost function with respect to w is

∇ w J k (w) = y w,k (y w,k w -x k ), (2)
which leads to the gradient descent formulation

w t = w t-1 + η t (x k y wt-1,k -y 2 wt-1,k w t-1 ), (3)
where η t is the learning rate. This update rule is essentially Oja's rule [START_REF] Oja | Simplified neuron model as a principal component analyzer[END_REF], which is a single-neuron special case of the generalized Hebbian learning, applied here within the in-network setting described above.

• Drawing inspiration from the information theoretical framework studied in [START_REF] Plumbley | Lyapunov functions for convergence of principal component algorithms[END_REF][START_REF] Miao | Fast subspace tracking and neural network learning by a novel information criterion[END_REF], we minimize the following cost function associated with node k:

J k (w) = 1 2 w ⊤ w -1 2 ln(y 2 w,k ). (4)
whose gradient with respect to w is

∇ w J k (w) = w - x k y w,k . (5)
By applying the gradient descent technique on this cost function, this leads to the following update rule:

w t = w t-1 + η t x k y wt-1,k -w t-1 . (6)
The update rules (3) and ( 6) are intimately connected, since the latter can be derived from the former by using the normalized learning rate η t /y 2 wt-1,k . For this reason, we will provide throughout this paper the general expressions, while we study the effect of such normalization in experiments.

These learning rules converge to the same equilibrium state, which is the first principal axis w * . In fact, when w t converges to some state w, we have

x k y w,k = y 2 w,k w (from (3)) or x k /y w,k = w (from (6)), namely x k x ⊤ k w = y 2 w,k w.
Averaging the latter expression over the whole data, we get the well-known eigen-decomposition problem of the covariance matrix Cw = E(y 2 w ) w, where the eigenvalue is the squared output y w,k to be maximized. Therefore, the two update rules (3) and ( 6) converge to the eigenvector associated with the largest eigenvalue of the covariance matrix.

Cooperative strategies

Now, each node k has access to the information of its neighborhood V k . The optimization problem can be presented as a minimization of the cost function N k=1 J k (w), where J k (w) is the cost function at node k, for instance either (1) or (4). Since each node k communicates only with its neighbors, we introduce the nonnegative coefficients c kl that relate the node k to its neighbors. They satisfy the following conditions:

c kl ≥ 0, l∈V k c kl = 1, and c kl = 0 if l / ∈ V k . ( 7
)
It is worth noting that if l / ∈ V k , k / ∈ V l and thus c lk is also null. These coefficients allow defining, for each node l, a local cost that consists of a weighted combination of the individual costs of the neighbors of node l (including l itself):

J loc l (w) = k∈V l c kl J k (w) = N k=1 c kl J k (w). (8)
It turns out that the cumulative sum of these local cost functions is equal to the global cost function since we have

N k=1 J k (w) = N k=1 N l=1 c kl J k (w) = N l=1 N k=1 c kl J k (w) = N l=1 J loc l (w).
For node k, the global cost function can be decomposed as

N l=1 J l (w) = J loc k (w) + N l=1 l =k J loc l (w). (9)
While the first term in the right-hand-side is known at the node under scrutiny, the second one should be estimated using information from the neighbors of node k; thus, the above summation is restricted to its neighborhood. Moreover, we relax it by constraining the norm between the estimated vector w and the optimal (global) principal axis w * within the neighborhood of node k, with the following regularization:

N l=1 l =k J loc l (w) ≈ l∈V k \{k} b lk w -w * 2 ,
where the parameters b lk control the tradeoff between the accuracy and the smoothness of the solution. Such regularization is also motivated by investigating the second-order Taylor expansion of J loc l (w), as described in [START_REF] Chen | Diffusion adaptation strategies for distributed optimization and learning over networks[END_REF]. Note that we use w * knowing that we do not have access to its value. We will show in the following how to overcome this problem.

By injecting this approximation into (9), and using ( 8), the minimization of the latter is equivalent to minimizing

J glob k (w) = l∈V k c lk J l (w) + l∈V k \{k} b lk w -w * 2 . ( 10
)
In order to minimize the above cost function, the node k applies the gradient descent on J glob k (w) with:

w k,t = w k,t-1 -η k,t ∇ w J glob k (w k,t-1 ). (11)
Here, η k,t is the learning rate for node k at iteration t. Replacing J glob k (w) by its expression in [START_REF] Hall | Merging and splitting eigenspace models[END_REF], we get:

w k,t = w k,t-1 -η k,t l∈V k c lk ∇ w J l (w t-1 ) + η k,t l∈V k \{k} b lk (w * -w k,t-1 ).
In this expression, ∇ w J l (w t-1 ) is the gradient of the PCAbased cost function, presented earlier in (2) and ( 5). In the former case, we get

w k,t = w k,t-1 + η k,t l∈V k c lk x l y w t-1 ,l -y 2 wt-1,l w l,t-1 + η k,t l∈V k \{k} b lk (w * -w k,t-1 ), (12)
while the latter corresponds to substituting the above learning rate by η k,t /y 2 wt-1,k . Without loss of generality, we will consider the update rule [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF] in the following. This update rule from w k,t-1 to w k,t involves adding two correction terms: the first one operates in the maximum variance direction and the second one associates neighborhood regularization. By decomposing this expression into two successive steps, , we get two possible strategies, depending on the order of adding the correction terms, and that differ essentially in the approximation of the unknown w * in [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF], as shown next.

Adapt-then-combine strategy

We express the update rule [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF] as follows:

φ k,t = w k,t-1 + η k,t l∈V k c lk x l y w t-1 ,l -y 2 wt-1,l w l,t-1 , w k,t = φ k,t + η k,t l∈V k \{k} b lk (w * -φ k,t ).
The first step uses local gradient vectors from the neighborhood of the node k in order to update w k,t-1 to the intermediate estimate φ k,t . By incorporating information from the neighbors, this intermediate estimate is generally a better estimate for w * than w k,t-1 , as used in the second step, which becomes

w k,t = φ k,t + η k,t l∈V k \{k} b lk (φ l,t -φ k,t ),
or, equivalently w k,t = l∈V k a kl φ l,t where we have used the following nonnegative weighting coefficients:

a kl =    1 -η k,t i∈V k \{k} b ik , if l = k; η k,t b lk , if l ∈ V k \{k} ; 0,
otherwise.

Combine-then-adapt strategy

In this strategy, we express the update rule [START_REF] Le Borgne | Distributed principal component analysis for wireless sensor networks[END_REF] as follows:

φ k,t = w k,t-1 + η k,t l∈V k \{k} b lk (w * -w k,t-1 ) w k,t = φ k,t + η k,t l∈V k c lk x l y w t-1 ,l -y 2 wt-1,l φ l,t .
By approximating w * by w l,t-1 and introducing the same coefficients a kl , we obtain the update rule

φ k,t = l∈V k a kl w l,t-1 , w k,t = φ k,t + η k,t l∈V k c lk x l y w t-1 ,l -y 2 wt-1,l φ l,t .

Connections to the consensus strategies

The consensus-type implementation is a class of distributed strategies that takes the following form:

w k,t = l∈V k a kl w k,t-1 +η k,t x k y w t-1 ,k -y 2 wt-1,k w k,t-1 .
We notice that the consensus strategy has the same computational complexity as for the adapt-then-combine and the combine-then-adapt diffusion strategies. However, these diffusion strategies outperform the consensus implementation. Indeed, we can express the adapt-then-combine and the combine-then-adapt diffusion strategies respectively by:

w k,t = l∈V k a kl w k,t-1 + η k,t x k y w,k -y 2 w,k w k,t-1 , w k,t = l∈V k a kl w k,t-1 + η k,t x k y w,k -y 2 w,k l∈V k a kl w k,t-1 .
Considering for instance the combine-then-adapt strategy, we note that the combination coefficients a kl appear in the correction term, while the consensus uses only w k,t-1 to correct the error, which is less effective, as will be shown in the experimental results.

MULTIPLE PRINCIPAL AXES

To extend the derivation to multiple principal axes, we denote by

W t = [w 1,t w 2,t • • • w r,t
] ⊤ , the r-by-p matrix of the first r principal axes estimated at iteration t, listed in descending order of their eigenvalues. Let

y W ,k = [y w 1 ,k y w 2 ,k • • • y w r ,k ] ⊤
, where y w j ,k = w ⊤ j,t x k . Next, we combine the update rules given in Sections 3.2 and 3.3 with the Gram-Schmidt orthogonalization process. 2In the noncooperative strategy, we get the update rule of the j-th principal axis as following:

w j,t = w j,t-1 + η t x k y wj ,k -y wj ,k j l=1 y wj ,l w l,t-1 , (13)
which is essentially similar to a Sanger's generalized Hebbian algorithm [START_REF] Sanger | Optimal unsupervised learning in a single-layer linear feedforward neural network[END_REF]. In matrix form, we obtain

W t = W t-1+ηt y W t-1 ,k x ⊤ k -LT(y W t-1 ,k y ⊤ W t-1 ,k ) W t-1 , (14)
where LT(•) makes its argument lower triangular by setting to zero the entries above its diagonal. Note that the learning rate does not needs to be the same for all the principal axes.

In the cooperative strategies, we denote by

Φ k,t = [φ 1,k,t φ 2,k,t • • • φ r,k,t ] ⊤ the
r-by-p matrix of the r intermediate estimates. Written in a matrix form, the update rule associated with the adapt-then-combine strategy becomes

Φ k,t = W k,t-1 + η k,t y W t-1 ,k x ⊤ k -LT(y W t-1 ,k y ⊤ W t-1 ,k )W k,t-1 , W k,t = l∈V k a kl Φ l,t .
For the combine-then-adapt strategy, we get the steps

Φ k,t = l∈V k a kl W l,t-1 , W k,kt = Φ k,t + η k,t y W t-1 ,k x ⊤ k -LT(y W t-1 ,k y ⊤ W t-1 ,k )Φ k,t .

EXPERIMENTATIONS

In this section, we illustrate the performance of the proposed approach. In order to provide a fair comparative study, we use the same initial random estimate for all strategies. The performance is measured in terms of the angle between the principal axis w * , obtained from the centralized strategy with the eigen-decomposition of the covariance matrix, and the estimate w * ,l at node l, namely arccos w ⊤ * ,l w * w * ,l w * .

Time series measurements in a WSN

In this section, we consider the problem of tracking a gas spread using a wireless sensor network (WSN) [START_REF] Akyildiz | A survey on sensor networks[END_REF]. Here, x k,t denotes the gas measurement of the k-th sensor at time t. The position of a sensor k is denoted by z k ∈ Z. The region under scrutiny Z = [-0.5, 0.5] × [-0.5, 0.5] is a twodimensional unit-area. Our goal is to reduce the order of the time series of the measurements. The gas diffusion within this region is governed by the following differential equation:

∂G(z, θ) ∂θ -c ∇ 2 z G(z, θ) = Q(z, θ),
where G(z, θ) is the density of gas depending on the position z and time θ, ∇ 2 z is the Laplace operator, c the conductivity of the medium, and Q(z, θ) corresponds to the added quantity of gas. A gas source placed at the origin is activated from θ = 1 to θ = 15. We use N = 100 sensors deployed uniformly in the region Z, each acquiring a time series of 15 measurements, between θ = 1 and θ = 15.

We consider a predetermined range of communication in the WSN. In this case, two nodes are connected when their distance is less than 0.38. For the stepsize parameters, we consider the stepsize η 1 = 0.0025 for the first principal axis and η 2 = 0.0005 for the second principal axis. We choose another stepsize for the second axis for a better convergence. As for the information theory, we consider η 1 = 0.025 and η 2 = 0.05 for both cooperative and noncooperative strategies.

Figure 2 and Figure 3 show the convergence of each strategy for the general expressions and for the information theory respectively, with the angles Θ 1 and Θ 2 , for the first and the second axes respectively, averaged over the N nodes. The confronted strategies are the noncooperative strategy, the combine-then-adapt (CTA) and adapt-then-combine (ATC) diffusion strategies, and the consensus implementation. For the diffusion strategies, we assumed that c kk = 1, and c lk = 0 for each k = l, with l, k = 1, • • • , 100. Results are shown in terms of the angle averaged over all the nodes. These learning curves show that the noncooperative strategy is outperformed by any diffusion strategy, independently of the combination rule. The analysis of the diffusion strategies shows that the adapt-then-combine strategy performs slightly better that the combine-then-adapt strategy. The overall efficiency of the diffusion strategies is shown in terms of stability.

Image processing application

We consider an image processing application, with the handwritten digit "1" given in 90 images of 28-by-28 pixels, each treated as a 784-dimensional vector. This time, the connectivity between nodes is tested for s = 2240. For the stepsize parameters, we consider the stepsize η 1 = 5.10 -8 for the first principal axis and η 2 = 1.10 -7 for the second principal axis. The same stepsize values are taken for the cooperative strategies. As for the information theory, we take η 1 = 0.05 and η 2 = 0.05 for both cooperative and noncooperative strategies. Figures 4 and5 show the convergence of each strategy for the general form and for the information theory respectively, with the angles averaged over the N nodes. Again, the noncooperative strategy is outperformed by all diffusion strategies, independently of the combination rule.

Fig. 1 .

 1 Fig. 1. Illustration of information processing within the proposed networks.

Fig. 2 .

 2 Fig. 2. Convergence analysis in the WSN settings.

Fig. 3 .

 3 Fig. 3. Convergence analysis of the strategies based on information theory, in the WSN settings.

Fig. 4 .

 4 Fig. 4. Convergence analysis for the image dataset.

Fig. 5 .

 5 Fig. 5. Convergence analysis of the strategies based on information theory, for the image dataset.

One may also include the communication complexity, which is O(N p) over a distance O(1).

Several orthogonalization techniques can be investigated within the proposed framework, including the Gram-Schmidt process, the deflation, and the symmetric orthogonalization. in order to extract multiple principal axes.

CONCLUSION

In this paper, we studied the issue of estimating the principal axes from PCA in networks. While taking into account the constraints imposed in WSNs and image processing applications, we proposed several strategies including noncooperative and diffusion strategies. Experimental results showed the relevance of these strategies. As a future work, we will also include the use of the spatial information within the study.