LOW-RANK APPROXIMATION-BASED DISTRIBUTED NODE-SPECIFIC SIGNAL
ESTIMATION IN A FULLY-CONNECTED WIRELESS SENSOR NETWORK
Amin Hassani, Alexander Bertrand, Marc Moonen
KU Leuven, Dept. of Electrical Engineering-ESAT,
Stadius Center for Dynamical Systems, Signal Processing and Data Analytics,
Address: Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
E-mail: amin.hassani, alexander.bertrand, marc.moonen@esat.kuleuven.be
ABSTRACT
In this paper, we consider the problem of distributed estima-tion of node-specific signals in a fully-connected wireless sen-sor network with multi-sensen-sor nodes. The estimation relies on a data-driven design of a spatial filter, referred to as the generalized eigenvalue decomposition (GEVD)-based multi-channel Wiener filter (MWF). In non-stationary or low-SNR conditions, this GEVD-based MWF has been demonstrated to be more robust than the original MWF due to an inher-ent GEVD-based low-rank approximation of the sensor signal correlation matrix. In a centralized realization where a fusion center has access to all the nodes’ sensor signal observations, the network-wide sensor signal correlation matrix and its low-rank approximation can be directly estimated from the sensor signals. However, in this paper we aim to avoid centralizing the sensor signal observations, in which case this network-wide correlation matrix cannot be estimated. We introduce a distributed algorithm which is able to significantly compress the broadcast signals while still converging to the centralized GEVD-based MWF as if each node would have access to all sensor signal observations.
Index Terms— wireless sensor networks (WSNs), dis-tributed estimation, low rank approximation, generalized eigenvalue decomposition (GEVD).
This work was carried out at the ESAT Laboratory of KU Leuven, in the frame of KU Leuven Research Council CoE PFV/10/002 (OPTEC), Con-certed Research Action GOA-MaNet, the Interuniversity Attractive Poles Programme initiated by the Belgian Science Policy Office IUAP P7/23 ‘Bel-gian network on stochastic modeling analysis design and optimization of communication systems’ (BESTCOM) 2012-2017, Research Project FWO nr. G.0763.12 ’Wireless Acoustic Sensor Networks for Extended Audi-tory Communication’, Project FWO nr. G.0931.14 ‘Design of distributed signal processing algorithms and scalable hardware platforms for energy-vs-performance adaptive wireless acoustic sensor networks’, and project HANDiCAMS. The project HANDiCAMS acknowledges the financial sup-port of the Future and Emerging Technologies (FET) Programme within the Seventh Framework Programme for Research of the European Commission, under FET-Open grant number: 323944. The scientific responsibility is as-sumed by its authors.
1. INTRODUCTION
Most spatial filtering or beamforming techniques use a fixed sensor array with a limited number of (often closely-spaced) wired sensors, resulting in only a local sampling [1], [2]. An alternative could be to deploy a wireless sensor network (WSN) [3], [4], with a larger number of sensor nodes, to collect more diverse information of the spatial field.
To process the sensor signal observations of a WSN, one possibility is to collect them in a fusion center, which we refer to as a centralized approach. However, this centralized pro-cessing requires a large communication bandwidth and com-putational workload. In this paper, we aim for a distributed approach, in which the nodes cooperate to solve an estima-tion task by sharing compressed sensor signal observaestima-tions, and by distributing the computational burden amongst them.
A large class of estimation problems in WSNs deals with the estimation of a common network-wide parameter or sig-nal of interest [5–7], which basically means that all nodes col-laborate to attain a global goal. In other estimation problems however, the parameters or signals of interest differ at each node, i.e., they are node-specific [8–11]. In some cases, these node-specific desired signals are related across the different nodes, e.g., when the signals of interest must be estimated as they are observed at a local sensor of each node to preserve the spatial properties in the signals [12–15].
The distributed adaptive node-specific signal estimation (DANSE) algorithm [16] is originally designed to estimate a node-specific desired signal at each node in a fully-connected WSN in a distributed fashion. In essence, DANSE can be viewed as a distributed realization of the centralized MWF and it considers the case where the node-specific desired sig-nals share a common (unknown) latent signal subspace. By exploiting this common interest of the nodes, DANSE signifi-cantly compresses the broadcast signals while still converging to the centralized linear minimum mean square error (MMSE) estimators as if each node would have access to all sensor sig-nal observations of the WSN.
Originally MWF has been designed based on a low-rank approximation of the signal correlation matrix with a
so-called column decomposition [17], [18]. However, in low-SNR conditions, and for highly non-stationary noise in par-ticular, the signal correlation matrix is often estimated poorly, which leads to suboptimal or even unstable filters [18]. Alter-natively, either an eigenvalue decomposition (EVD)-based or a generalized EVD (GEVD)-based low-rank approximation of the signal correlation matrix can be applied to improve the estimation performance in such cases. MWF with GEVD-based low-rank approximation has been shown to deliver the best performance, as it effectively selects the “mode” corre-sponding to the highest SNR [18]. The resulting spatial filter is referred to as the GEVD-based MWF.
In this paper, the objective is to design a DANSE-like algorithm that computes the GEVD-based MWF in a dis-tributed fashion in a fully-connected1WSN. We will refer to this as the GEVD-based DANSE algorithm. The proposed GEVD-based DANSE algorithm compresses the multi-sensor signals at each node into a smaller number of signal obser-vations which are then broadcast to the other nodes. Re-markably, even though the GEVD-based DANSE algorithm is not able to compute the network-wide signal correlation matrix (and its GEVD) from these compressed signal ob-servations, the algorithm does converge to the centralized GEVD-based MWF as if each node would have access to all (uncompressed) sensor signal observations.
The paper is organized as follows. The data model is pre-sented in Section 2. The centralized GEVD-based MWF is explained in Section 3. The GEVD-based DANSE algorithm and its convergence analysis is addressed in Section 4. Nu-merical simulations are presented in Section 5. Finally con-clusions are drawn in Section 6.
2. DATA MODEL AND MOTIVATION
We consider a fully-connected WSN with K multi-sensor nodes. Each node k ∈K= {1, . . . , K} is assumed to collect observations of a complex-valued Mk-channel sensor signal yk. Note that this also allows for a hierarchical WSN where K master nodes collect sensor signal observations from Mk slave nodes with a single sensor. The sensor signal ykcan be modeled as
yk = dk+ nk = Aks + nk (1)
where s is a latent S-channel signal defining S latent source signals, Ak is an unknown Mk × S complex-valued steer-ing matrix, and nk is additive noise. The sensor signal yk is assumed to satisfy short-term stationarity and ergodicity conditions. By stacking all yk, nk and dk, we obtain the network-wide M -channel sensor signals y, d and n, respec-tively, where M =PK
k=1Mkand hence y = d + n. The goal for each node k ∈Kis to denoise all Mk chan-nels of yk. Hence the desired signal to be estimated at each 1It is noted that all results in this paper can be extended to tree topology
networks, using similar strategies as in [19].
node is the Mk-channel signal dk. This means that the es-timation procedure will preserve the node-specific spatial in-formation in dkwhile reducing the noise nk.
3. CENTRALIZED GEVD-BASED MWF We first consider the centralized estimation problem. There-fore the objective for each node k is to estimate a complex-valued node-specific unknown Mk-channel desired signal dk, from the observations of all sensor signals in y. Node k uses an M × Mk linear estimator ˆWk to estimate dk as ˆdk =
ˆ
WH
ky, where superscript H denotes the conjugate transpose operator and where the hat (ˆ.) refers to the fact that the cen-tralized solution is considered. The MWF [17] computes ˆWk based on the minimum mean square error (MMSE) criterion, such that
ˆ
WMMSEk = arg min Wk E dk− WHky 2 (2)
where E{.} is the expected value operator. Assuming Ryy = E{yyH} has full rank, the unique solution of (2) is [17]:
ˆ WMMSE
k = R−1yyRdd (3)
where Rd = E{ddH}. We also define the network-wide noise covariance matrix Rnn = E{nnH}, where it is as-sumed that Rnnis either known a-priori or can be estimated from noise-only segments in the sensor signal observations. The latter can be performed in applications where the target signal has an on-off behavior, such as in speech enhancement where Ryy and Rnn can be estimated during “speech-and-noise” and “noise-only” segments, respectively, using a voice activity detection [17], [13]. The estimated correlation matri-ces will be denoted as ¯Ryy, ¯Rnnand ¯Rdd.
Assuming d and n are uncorrelated, the signal correla-tion matrix Rdd can be estimated as Ryy − Rnn. Note that in theory Rdd is a rank-S matrix, which can be verified by considering
Rdd= E{ddH} = AΨAH (4)
where A is the stacked version of all Ak steering matrices, and where Ψ = diag{ψ1, . . . , ψS} is an S-dimensional di-agonal matrix, where ψs = E{|st|2}, with t ∈ {1, . . . , S}. In practice, however, the estimated ¯Rddhas generally a rank greater than S, and it may even not be positive semi-definite due to the subtraction ¯Ryy − ¯Rnn. In this case, it has been demonstrated in [18] that incorporating a low rank approxi-mation based on either the eigenvalue decomposition (EVD) of ¯Rddor the generalized eigenvalue decomposition (GEVD) of ¯Ryy and ¯Rnnenhances the estimation performance of the MWF, especially in low-SNR conditions. The GEVD-based low-rank approximation has been shown to deliver the best performance, as it effectively selects the “mode” correspond-ing to the highest SNR [18]. In the rest of this section, the GEVD-based MWF solution is explained in detail.
In order to perform a GEVD of the ordered matrix pair ( ¯Ryy, ¯Rnn), each generalized eigenvector (GEVC) and its
corresponding generalized eigenvalue (GEVL), xmand λm (m = 1 . . . M ), respectively, must be computed such that
¯
Ryyxm= λmR¯nnxm[20], or equivalently ¯
RyyX = ¯RnnXΛ (5)
where X = [x1...xM] and Λ = diag{λ1. . . λM}. Note that when ¯Rnnis invertible, (5) can be written as a non-symmetric EVD as
¯
R−1nnR¯yy= XΛX−1. (6)
In the sequel, we assume w.l.o.g. that the GEVLs in Λ are sorted in descending order. Since the GEVCs are defined up to a scaling, we assume w.l.o.g. that all xm’s are scaled such that XHR¯nnX = IM where IM denotes the M × M iden-tity matrix. It is noted that the GEVD is equivalent to a joint diagonalization of ¯Ryy and ¯Rnn, i.e., it can be verified from (6) that
¯
Ryy = QΣQH, ¯Rnn= QΓQH (7)
where Q = X−H is a full-rank M × M matrix (not nec-essarily orthogonal), and where Σ = diag{σ1, . . . , σM} and Γ = diag{γ1, . . . , γM} are diagonal matrices. Note that (6) then implies that the GEVLs are equal to λm= σγm
m. Recon-sidering y = d + n and (7)), it follows that
¯
Rdd = R¯yy− ¯Rnn= Q Σ − ΓQH= Q∆QH(8) where ∆ = diag{δ1, ..., δM} with δm = σm− γm. The rank-R approximation of ¯Rdd becomes Q∆RQH with ∆R denoting the diagonal matrix ∆ with the M − R smallest diagonal entries set to zero. Ideally (but not necessarily), R is set to R = S, which is motivated by (4). By replacing ¯Rdd with its rank-R approximation in (3), the GEVD-based MWF is defined as
ˆ
Wk = ¯R−1yyQ∆RQHEk (9)
where Ekis a M × Mkmatrix which selects the Mkcolumns corresponding to node k. The next section explains how the GEVD-based DANSE algorithm obtains the signal estimates ˆ
dk = ˆWHk y, i.e., the outputs of (9) in a decentralized fashion.
4. GEVD-BASED DANSE
In this section, we briefly introduce the GEVD-based DANSE algorithm to obtain the same node-specific solution (9) at each node k ∈K, without accessing to the full signal y.
In GEVD-based DANSE, each node k ∈ K first opti-mally fuses its Mk-channel signal yk into a J -channel sig-nal zk = FHkyk with an Mk× J fusion matrix Fk (which will be defined later, see (13)), and then broadcasts observa-tions of zk to all other nodes. Consequently and compared to the centralized GEVD-based MWF, the algorithm reduces the required per-node communication bandwidth by a factor of max{(Mk/J ), 1}.
Considering z = [zT1. . . zTK] T, z
−kdenotes the vector z with zk omitted. Each node k in GEVD-based DANSE has
access to a Pk-channel signaleyk which is defined as eyk = [yTk zT−k]T, with Pk = Mk + J (K − 1). We use a similar notation for the desired and the noise component ofeyk, i.e., e
dkandenk.
In the DANSE algorithm [16], at iteration i, node q is the updating node where the local MMSE problem and its solu-tion take the form (the iterasolu-tion index i is omitted for concise-ness): f WMMSE q = arg min f Wq E ( dq− fWqHeyq 2) (10) f WMMSE q = ( ¯Ry˜qy˜q) −1R¯˜ dqd˜qEe (11) (compare with (2)-(3)) where ¯Ry˜qy˜q, ¯R˜nqn˜q and ¯Rd˜qd˜q are the Pk-dimensional correlation matrices corresponding re-spectively to ˜yq, ˜nqand ˜dqsignals, and where eE is a Pk×Mk matrix which selects the first Mkcolumns of ¯Rd˜qd˜q.
Similar to (5)-(8), here we locally perform a GEVD at node q on the matrix pair ¯Ry˜qy˜qand ¯Rn˜qn˜q. This leads to the corresponding local Pk-dimensional matrices eXq, eΛq, eQq,
e
Σq, eΓq and e∆q, where eQq = eX−Hq . When replacing ¯Rd˜qd˜q by its GEVD-based rank-R approximation, solution (11) be-comes f Wq= ( ¯Ry˜qy˜q) −1 e Qq∆eqRQe H q Ee (12) (compare with (9)) where e∆qRis the Pk-dimensional diago-nal matrix e∆q with the Pk− R smallest diagonal entries set to zero. The aforementioned fusion rule Fqat node q is then chosen as Fq= IMq 0 f Wq IJ 0 . (13)
Finally node q estimates its node-specific Mk-channel desired signal as dq = fWqHeyq. The resulting GEVD-based DANSE algorithm is described in Table 1.
Note that the fusion rule defined in (13) is the same as in the original DANSE algorithm, but now fWq is the result of a low-rank approximation-based MWF instead of a full-rank MWF. Due to this modification, the convergence proof of the original DANSE algorithm in [16] is not applicable anymore. Nevertheless, convergence of the GEVD-based DANSE algo-rithm can be proven under some technical conditions, as given by the following theorem:
Theorem I: If J = R and under some technical conditions, the GEVD-based DANSE algorithm converges for any ini-tialization of its parameters to the centralized GEVD-based MWF solution, i.e., wheni → ∞, ¯dk = ˆdk.
Proof:Omitted.
Remark I: The conditions stated above are as follows: 1) ¯
Ryy is rank-M 2) fWik, ∀k ∈ Kis rank-R in each iteration i of the GEVD-based DANSE algorithm. In practice, these conditions are usually satisfied . It is noted that both con-ditions are also required for the convergence of the original DANSE algorithm. However, the latter also requires strict conditions on the data model, i.e., the node-specific desired
Table 1. GEVD-based DANSE algorithm
1. Set i ← 0, q ← 1, and initialize all F0
kand fW0k, ∀ k ∈K,
with random entries.
2. Each node k ∈Kbroadcasts the J -channel fused signal of its N new observations
zk[iN + j] = Fi Hk yk[iN + j], j = 1 . . . N (14)
where the notation [.] denotes a sample index. 3. At node q: • Estimate ¯Ri ˜ yqy˜qand ¯R i ˜
nqn˜qvia sample averaging.
• Compute Qeiq and ∆eiq from the GEVD of ( ¯Ri
˜
yq˜yq, ¯Rinq˜ nq˜ ) similar to (5)-(8).
• Compute the local MWF with rank-R approximation of ¯Ri ˜ dqdq˜ as follows: f Wi+1q = ( ¯Riyq˜ yq˜ ) −1 e Qiq∆eiq RQe i H q Ee (15) • Update the fusion rule as
Fi+1 q =IMq 0 f Wi+1 q IJ 0 (16) 4. Other nodes k ∈K\ q update their parameters as fWi+1k =
f Wi
kand F i+1 k = Fik.
5. Each node k ∈Kestimates its Mk-channel signal dk, as
dk[iN + j] = fWi+1k yek[iN + j] (17) 6. i ← i + 1 and q ← (q mod K) + 1 and return to step 2.
signals dk should share a common latent signal subspace. Although the existence of such a subspace motivates the low-rank approximation of Rdd (see (4)), it is not a requirement as such for the GEVD-based DANSE algorithm to converge to the centralized GEVD-based MWF.
Remark II: It should be emphasized that for any choice of J = R, the GEVD-based DANSE algorithm converges to the centralized GEVD-based MWF, while only both are optimal in MMSE-sense when J = R = S. However note that it has been shown in [16] that the original DANSE algorithm only converges to the centralized MMSE-based MWF solution if J = S.
Remark III: It should be mentioned that GEVD-based DANSE can be shown to be equivalent (up to specific per-node transformations) to the DACGEE algorithm [21], based on an invariance-property of the GEVD with respect to row transformations. As a results, convergence of the latter can be exploited to prove convergence of the former. Although the proof of this relationship between both algorithms is not triv-ial, the mere fact that they are related may not be a complete surprise, since the GEVD-based MWF with rank-R approxi-mation implicitly also computes a GEVD.
5. NUMERICAL SIMULATIONS
A Monte-Carlo (MC) simulation scenario with K = 10 nodes and Mk = 15, ∀k ∈Kis considered. The observations of the latent S-channel signal s, and the entries of the Mk× S
steer-0 50 100 150 200 250 300 350 400 450 500 10-10
100
Avaraged MSE of denoised
signals 0 50 100 150 200 250 300 350 400 450 500 10-10 100 Iterations Avaraged MSE of MWF solutions R=J=2 R=J=1 R=J=2 R=J=1
Fig. 1. Convergence of GEVD-based DANSE ing matrix Ak, ∀k ∈Kare both independently drawn from a uniform distribution over the interval [−0.5; 0.5]. Two tar-get sources (S = 2) as well as two localized noise sources are assumed, where the target sources have an on-off behav-ior, while the noise sources are continuously active. In order to model sensor noise as well (spatially uncorrelated com-ponents), nk also contains an additive stochastic signal from which the observations are independently drawn from a uni-form distribution over the interval [−√0.2/2;√0.2/2].
Fig.1 illustrates the convergence results for two cases: 1) J = R = 2 and 2) J = R = 1, averaged over 200 MC runs (S = 2 in both cases). In the upper part, the mean squared errors (MSEs) between the entries of ˆdk and ¯dk (av-eraged over the nodes) are shown over the different iterations of the GEVD-based DANSE. Similarly, the bottom part il-lustrates the MSE between ˆWk and the corresponding filters in the case of GEVD-based DANSE. It is observed that for both cases, the GEVD-based DANSE algorithm converges (with a random initialization of its parameters) to the cen-tralized GEVD-based MWF solution. It is noted that the case J = R = 2 converges faster than the case J = R = 1, which can be explained by the larger number of degrees of freedom in each update step in the case of the former.
6. CONCLUSION
In this paper, we have proposed a distributed algorithm for the estimation of node-specific desired signals in a fully-connected wireless sensor network. The estimation has been based on a GEVD-based low-rank approximation of the cor-relation matrices within the MWF that is locally computed at each node. The resulting GEVD-based DANSE algorithm significantly compresses the broadcasting signals compared to a centralized approach with a fusion center. We have stated (without a proof) that the GEVD-based DANSE algorithm converges to the centralized GEVD-based MWF as if each node would have access to all the sensor signal observations.
7. REFERENCES
[1] S.U. Pillai and C.S. Burrus, Array signal processing, Signal Processing and Digital Filtering. Springer-Verlag, 1989.
[2] H. Krim and M. Viberg, “Two decades of array signal processing re-search: the parametric approach,” Signal Processing Magazine, IEEE, vol. 13, no. 4, pp. 67–94, 1996.
[3] D. Estrin, L. Girod, G. Pottie, and M. Srivastava, “Instrumenting the world with wireless sensor networks,” in Acoustics, Speech, and Sig-nal Processing, 2001. Proceedings. (ICASSP ’01). 2001 IEEE Interna-tional Conference on, 2001, vol. 4, pp. 2033–2036 vol.4.
[4] D. Culler, D. Estrin, and M. Srivastava, “Overview of sensor networks,” Computer, vol. 37, no. 8, pp. 41–49, Aug 2004.
[5] C.G. Lopes and A.H. Sayed, “Incremental adaptive strategies over dis-tributed networks,” Signal Processing, IEEE Transactions on, vol. 55, no. 8, pp. 4064–4077, Aug 2007.
[6] C.G. Lopes and AH. Sayed, “Diffusion least-mean squares over adap-tive networks: Formulation and performance analysis,” Signal Process-ing, IEEE Transactions on, vol. 56, no. 7, pp. 3122–3136, July 2008. [7] I.D. Schizas, G.B. Giannakis, and Zhi-Quan Luo, “Distributed
estima-tion using reduced-dimensionality sensor observaestima-tions,” Signal Pro-cessing, IEEE Transactions on, vol. 55, no. 8, pp. 4284–4299, Aug 2007.
[8] A. Hassani, A. Bertrand, and M. Moonen, “Distributed node-specific direction-of-arrival estimation in wireless acoustic sensor networks,” in Proceedings of the European Signal Processing Conference (EU-SIPCO), 2013.
[9] A. Hassani, A. Bertrand, and M. Moonen, “Cooperative integrated noise reduction and node-specific direction-of-arrival estimation in a fully connected wireless acoustic sensor network,” in Signal Process-ing, 2014.
[10] N. Bogdanovic, J. Plata-Chaves, and K. Berberidis, “Distributed incremental-based LMS for node-specific adaptive parameter estima-tion,” Signal Processing, IEEE Transactions on, vol. 62, no. 20, pp. 5382–5397, Oct 2014.
[11] R. Abdolee, B. Champagne, and A.H. Sayed, “Estimation of space-time varying parameters using a diffusion LMS algorithm,” Signal Processing, IEEE Transactions on, vol. 62, no. 2, pp. 403–418, Jan 2014.
[12] S. Markovich, S. Gannot, and I. Cohen, “A reduced bandwidth bin-aural MVDR beamformer,” in Proc. of the International Workshop on Acoustic Echo and Noise Control (IWAENC), Tel-Aviv, Israel, 2010. [13] S. Doclo, M. Moonen, T. Van den Bogaert, and J. Wouters,
“Reduced-bandwidth and distributed MWF-based noise reduction algorithms for binaural hearing aids,” Audio, Speech and Language Processing, IEEE Transactions on, vol. 17, no. 1, pp. 38–51, 2009.
[14] S. Markovich, S. Gannot, and I Cohen, “Multichannel eigenspace beamforming in a reverberant noisy environment with multiple inter-fering speech signals,” Audio, Speech, and Language Processing, IEEE Transactions on, vol. 17, no. 6, pp. 1071–1086, Aug 2009.
[15] A. Bertrand and M. Moonen, “Distributed signal estimation in sensor networks where nodes have different interests,” Signal Processing, vol. 92, no. 7, pp. 1679 –1690, July 2012.
[16] A. Bertrand and M. Moonen, “Distributed adaptive node-specific signal estimation in fully connected sensor networks part I: sequential node updating,” in IEEE Trans. Signal Processing, 2010, vol. 58, pp. 5277– 5291.
[17] S. Doclo and M. Moonen, “GSVD-based optimal filtering for single and multimicrophone speech enhancement,” in IEEE Trans. Signal Processing, 2002, vol. 50, pp. 2230–2244.
[18] Romain Serizel, Marc Moonen, Bas Van Dijk, and Jan Wouters, “Low-rank approximation based multichannel Wiener filtering algorithms for noise reduction in cochlear implants,” Audio, Speech and Language Processing, IEEE Transactions on, vol. 22, no. 4, pp. 785–799, 2014. [19] A. Bertrand and M. Moonen, “Distributed adaptive estimation of
node-specific signals in wireless sensor networks with a tree topology,” Sig-nal Processing, IEEE Transactions on, vol. 59, no. 5, pp. 2196–2210, 2011.
[20] C. F. Van Loan G. H. Golub, Matrix Computations, 3rd ed., Baltimore, MD: John Hopkins Univ. Press, 1996.
[21] A. Bertrand and M. Moonen, “Distributed adaptive generalized eigen-vector estimation of a sensor signal covariance matrix pair in a fully-connected sensor network,” Signal Processing, vol. 106, pp. 209–214, Jan. 2015.
