Greedy selection of sensors with measurements under correlated noise

Abstract

We address the sensor selection problem where linear measurements under correlated noise are gathered at the selected nodes to estimate the unknown parameter. Since finding the best subset of sensor nodes that minimizes the estimation error requires a prohibitive computational cost especially for a large number of nodes, we propose a greedy selection algorithm that uses the log-determinant of the inverse estimation error covariance matrix as the metric to be maximized. We further manipulate the metric by employing the QR and LU factorizations to derive a simple analytic rule which enables an efficient selection of one node at each iteration in a greedy manner. We also make a complexity analysis of the proposed algorithm and compare with different selection methods, leading to a competitive complexity of the proposed algorithm. For performance evaluation, we conduct numerical experiments using randomly generated measurements under correlated noise and demonstrate that the proposed algorithm achieves a good estimation accuracy with a reasonable selection complexity as compared with the previous novel selection methods.

1 Introduction

In wireless sensor networks, a large number of sensor nodes are spatially distributed and transmit their noise-corrupted measurements which are typically formulated by a linear combination of the known observation matrix ${\textbf{H}}$ and the parameter to be estimated. The sensor selection is conducted so as to optimize the estimation accuracy [1,2,3,4,5,6,7]: Convex relaxation [1]- and cross-entropy optimization [2]-based methods were presented with a prohibitive computational complexity for large sensor networks. Noting that greedy approach yields a feasible complexity, the sensor selection problem has been mostly tackled in a greedy manner. To guarantee near-optimality with regard to the mean squared estimation error (MSE), a submodular cost function called the frame potential was devised and a greedy removal method was proposed to find optimal sensor locations [3]. A QR factorization-based greedy selection method was proposed to minimize the estimation error [4]. To further reduce the complexity of the selection process, the log-determinant of the inverse estimation error covariance matrix was employed as a metric to present simple greedy methods instead of directly minimizing the estimation error [5, 6]. Specifically, analytic selection rules were established by proving the metric to be a monotone submodular function [5] and by employing the QR factorization [6], respectively. An efficient greedy algorithm was also developed for sensor placement by maximally projecting onto the minimum eigenspace of the dual observation matrix [7]. Clearly, the selection of the best subset of p sensor nodes corresponds to the construction of the matrix with the most informative p rows selected from the matrix ${\textbf{H}}$.

It should be noticed that most of the selection methods have been derived under the assumption of uncorrelated noise: That is, off-diagonal elements of the noise covariance matrix are assumed to be approximately zero or ignorable. The sensor selection problem in the presence of correlated noise is challenging since the Fisher information (equivalently, the inverse estimation error covariance matrix) is no longer linear in the selected sensors [8,9,10]. To solve the problem, various selection algorithms have been previously presented in [8,9,10,11,12]. A multi-step sensor selection strategy was proposed for linear dynamic systems under correlated noise [8]. Sparsity-aware sensor selection approaches in centralized and distributed versions were developed in the presence of correlated measurements using convex relaxation for the maximum likelihood estimation (MLE) [9]. Two selection algorithms were derived for estimation of random parameters to minimize the MSE based on convex relaxation and greedy approach [10], respectively. Recently, a greedy selection algorithm maximizing the log-determinant of the inverse estimation error covariance matrix has been presented to provide an additional increase in estimation accuracy [11, 12]. Whereas the sensor selection problem has been extensively studied for the additive Gaussian linear measurement model which is considered in this work, it has been also addressed for nonlinear measurement model [13, 14]. In [13], the sensor selection algorithm for target tracking was proposed based on the extended Kalman filtering for an additive Gaussian nonlinear model. In [14], sensor selection methods for a general nonlinear model were developed with an aid of convex relaxation technique for estimation of unknown parameters.

In this paper, we consider the scenario where a given number of sensor nodes with measurements corrupted by correlated noise are selected to estimate the parameter. We aim to find the subset of sensor nodes that minimizes the estimation error. We first formulate the estimation error covariance matrix as a function of the matrix ${\textbf{H}}_S$ with rows selected from ${\textbf{H}}$ and the covariance matrix ${\textbf{K}}_S$ of noise samples at the selected nodes. To expedite the selection procedure, we adopt the log-determinant of the inverse estimation error covariance matrix as a metric to be maximized, which allows us to avoid a large matrix inversion. We choose one node at each iteration in a greedy manner until the selection set of the desired cardinality is constructed. To simplify the metric, we factor the matrices ${\textbf{H}}_S$ and ${\textbf{K}}_S$ based on the QR and LU factorizations, respectively, to obtain the upper triangular matrix ${\textbf{R}}$ and the lower triangular matrix ${\textbf{M}}$. We show that the factored matrices ${\textbf{R}}$ and ${\textbf{M}}$ can be updated by simply appending the newly computed column or row vector to the last positions of those matrices at the previous iteration, respectively. This simple computation of the key matrices ${\textbf{R}}$ and ${\textbf{M}}$ leads to an analytic rule which enables an efficient selection of such a maximizing node at each iteration. We also make a complexity analysis of the proposed algorithm, resulting in a competitive complexity of the same order as that of [4]. We evaluate the performance of the proposed method through numerical experiments using randomly generated measurements under correlated noise. We demonstrate the advantage of the proposed technique over previous selection methods in terms of the estimation performance and the selection complexity.

Compared with [4, 11, 12], we have the new contributions in this paper:

• In [4], the MSE is minimized by applying the QR factorization to ${\textbf{H}}_S$ in the assumption of uncorrelated noise. However, in the presence of correlated noise, the MSE is not manageable using only the QR factorization. The main contributions are given in two aspects: First, in order to yield a simple selection process, the final selection rule should be expressed as the first term related with the previous iteration plus the second term determined by only the current selection. The LU factorization of the covariance matrix ${\textbf{K}}_S$ is proved to generate the desired expression of the final rule. Second, the overall derivation process is completely different from [4] except for the QR factorization of ${\textbf{H}}_S$.

• The only similarity with [11, 12] is to optimize the same metric which is the log-determinant of the inverse estimation error covariance matrix. Except for that, the optimization process to derive the proposed selection rule in this paper is totally different from [11, 12]. Furthermore, the proposed method is shown to operate faster than the method in [11, 12] in numerical experiments while preserving the same estimation performance.

This paper is organized as follows. The problem is formulated in Sect. 2 in which the metric is shown to be expressed in terms of the sampled matrix ${\textbf{H}}_S$, the covariance matrix ${\textbf{K}}_S$ and the statistics of the parameter. In Sect. 3, the QR and LU factorizations are employed to simplify the metric and a simple selection criterion is derived. The complexity analysis of the proposed algorithm is provided in Sect. 4.1. Extensive experiments are presented in Sect. 4.2 and conclusions in Sect. 5.

2 Problem Formulation

Suppose that N wireless sensors deployed in a sensor field generate the measurements ${\textbf{y}}\in {\mathbb {R}}^N$. We seek to estimate the parameter vector ${\theta }\in {\mathbb {R}}^p$ with a Gaussian distribution ${\mathcal {N}}({\textbf{0}},{{\varvec{\Sigma }}_{\theta }})$ by using $n(< N)$ measurements gathered by the n selected sensors in the set S. We assume that the measurements are corrupted by the additive correlated Gaussian noise ${\textbf{w}}\sim {\mathcal {N}}({\textbf{0}},{\textbf{K}})\in {\mathbb {R}}^N$ independent of ${\theta }$ where the covariance matrix ${\textbf{K}}$ is positive definite and symmetric. We also assume that the measurements are produced by a linear model with the known observation $N\times p$ full column rank matrix ${\textbf{H}}$ with N row vectors ${\textbf{h}}_i^\top , i\in {\mathcal {V}}=\{1 \cdots N\}$: Specifically,

$$\begin{aligned} {\textbf{y}}={\textbf{H}}{\theta }+{\textbf{w}} \end{aligned}$$

(1)

We denote the estimator by $\hat{{\theta }}({\textbf{y}}_S)$ where ${\textbf{y}}_S$ is the $|S|\times 1$ column vector with entries of ${\textbf{y}}$ indexed by S. In this work, we use an optimal Bayesian linear estimator (e.g., maximum a posteriori (MAP) estimator or minimum mean squared error (MMSE) estimator). Note that the MAP estimator is equivalent to the MMSE estimator in our formulation where the parameter and the noise are assumed to be Gaussian. Then, we can derive the estimator $\hat{{\theta }}({\textbf{y}}_S)$ and the estimation error covariance matrix ${\varvec{\Sigma }}(S)$ as follows [5, 15]:

$$\begin{aligned} \hat{{\theta }}({\textbf{y}}_S)= & {} \left( {\varvec{\Sigma }}_{\theta }^{-1}+{\textbf{H}}_S^\top {\textbf{K}}_S^{-1}{\textbf{H}}_S\right) ^{-1}{\textbf{H}}_S^\top {\textbf{K}}_S^{-1}{\textbf{y}}_S \end{aligned}$$

(2)

$$\begin{aligned} {\varvec{\Sigma }}(S)= & {} \left( {\varvec{\Sigma }}_{\theta }^{-1}+{\textbf{H}}_S^\top {\textbf{K}}_S^{-1}{\textbf{H}}_S\right) ^{-1} \end{aligned}$$

(3)

where ${\textbf{H}}_S$ is the matrix with rows of the matrix ${\textbf{H}}$ indexed by S and ${\textbf{K}}_S$ the $|S|\times |S|$ covariance matrix of the noise vector ${\textbf{w}}_S$ with entries of ${\textbf{w}}$ indexed by S. Notice that the estimation of deterministic and unknown parameters can be also conducted by letting ${{\varvec{\Sigma }}_{\theta }}^{-1}={\textbf{0}}$ in (2) and (3).

In this work, we aim to find the best subset S with $|S|=p$ that maximizes the log-determinant of the inverse estimation error covariance matric given by $\log \det \left( {\varvec{\Sigma }}(S)^{-1}\right)$ which has been employed as a metric for the sensor selection problem in previous work [1, 5, 11, 12] and shown to produce a good estimation performance with a reduced complexity. We also seek to select one node at each iteration in a greedy manner. More specifically, given the set $S_i$ of i nodes already selected until the ith iteration, we select one node from the set of the remaining nodes, $S_i^C\equiv ({\mathcal {V}}-S_i)$ that maximizes the intermediate metric $\log \det \left( {\varvec{\Sigma }}(S_{i+1})^{-1}\right)$ at the $(i+1)$th iteration where $S_{i+1}=S_i+\{j\}, j\in S_i^C$. The selection process is expressed by

$$\begin{aligned} j^*=\arg \max _{S_{i+1}=S_i+\{j\},j\in S_i^C} \log \det \left( {\varvec{\Sigma }}_{\theta }^{-1}+{\textbf{H}}_{S_{i+1}}^\top {\textbf{K}}_{S_{i+1}}^{-1}{\textbf{H}}_{S_{i+1}}\right) \end{aligned}$$

(4)

where ${\textbf{H}}_{S_{i+1}}$ is simply created from ${\textbf{H}}_{S_i}$ and the jth row vector ${\textbf{h}}_j^\top$ selected from ${\textbf{H}}$:

$$\begin{aligned} {\textbf{H}}_{S_{i+1}}^\top = \begin{bmatrix} {\textbf{H}}_{S_i}^\top&{\textbf{h}}_j \end{bmatrix} \end{aligned}$$

(5)

and ${\textbf{K}}_{S_{i+1}}$ is also constructed from ${\textbf{K}}_{S_i}$ and the jth node selected from $S_i^C$:

$$\begin{aligned} {\textbf{K}}_{S_{i+1}}= & {} \begin{bmatrix} {\textbf{K}}_{S_i} &{} {\textbf{k}}\\ {\textbf{k}}^\top &{} k \end{bmatrix}\nonumber \\ {\textbf{k}}_{i+1}^\top\equiv & {} \left[ {\textbf{k}}^\top \quad k\right] =\left[ k_{(1)j}\quad k_{(2)j}\quad \cdots \quad k_{(i)j}\quad k_{jj}\right] \end{aligned}$$

(6)

where the subscript (i) indicates the number of the node selected at the ith iteration and ${\textbf{k}}_{i+1}^\top$ denotes the $1\times (i+1)$ row vector with ${\textbf{k}}=[k_{(1)j}\quad \cdots \quad k_{(i)j}]$ and $k=k_{jj}$. Notice that $k_{(i)j}$ represents the covariance of the noise samples at the (i)th and jth nodes. The process in (4) is performed repeatedly until the selection set S with $|S|=p$ is constructed.

3 Method: efficient sampling algorithm

In this section, we present an analytic result for a simple selection process by manipulating the matrix ${\textbf{H}}_{S_{i+1}}^\top$ and the covariance matrix ${\textbf{K}}_{S_{i+1}}$ based on the QR and LU factorizations, respectively. Note that the Householder transformation is employed to perform the QR factorization since it has strength in terms of complexity and sensitivity to rounding error in comparison with the Gram–Schmidt orthogonalization [16]. First, noting that the covariance matrix ${\textbf{K}}_{S_{i+1}}$ is symmetric, we can factor it as follows:

$$\begin{aligned} {\textbf{K}}_{S_{i+1}}={\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \end{aligned}$$

(7)

where ${\textbf{M}}_{S_{i+1}}$ is the $(i+1)\times (i+1)$ lower triangular matrix. We also do the QR factorization of ${\textbf{H}}_{S_{i+1}}^\top ={\textbf{Q}}\bar{{\textbf{R}}}^{i+1}$ where ${\textbf{Q}}$ is the $p\times p$ orthogonal matrix with p column vectors ${\textbf{q}}_j, j=1,\ldots ,p$ and $\bar{{\textbf{R}}}^{i+1}$ the $p\times (i+1)$ matrix. Then, we manipulate the metric in (4) to derive a simpler form with assumption of ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p$ where ${\textbf{I}}_p$ indicates the $p\times p$ identity matrix. Specifically,

$$\begin{aligned}{} & {} \log \det \left( {\varvec{\Sigma }}(S_{i+1})^{-1}\right) \nonumber \\{} & {} \quad =\log \det \left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_p+{\textbf{Q}}\bar{{\textbf{R}}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}(\bar{{\textbf{R}}}^{i+1})^\top {\textbf{Q}}^\top \right) \nonumber \\{} & {} \quad =\log \det \left( {\textbf{Q}}\left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_p+\bar{{\textbf{R}}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}(\bar{{\textbf{R}}}^{i+1})^\top \right) {\textbf{Q}}^\top \right) \nonumber \\{} & {} \quad =\log \det \left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_p+\bar{{\textbf{R}}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}(\bar{{\textbf{R}}}^{i+1})^\top \right) \end{aligned}$$

(8)

where (8) follows from the notion that $\det {\textbf{Q}}\det {\textbf{Q}}^\top =1$. Note that $\bar{{\textbf{R}}}^{i+1}$ can be written by

$$\begin{aligned} \bar{{\textbf{R}}}^{i+1}= \begin{bmatrix} {\textbf{R}}^{i+1}\\ {\textbf{0}}_{(p-i-1)\times (i+1)} \end{bmatrix} \end{aligned}$$

(9)

where ${\textbf{R}}^{i+1}$ is the $(i+1)\times (i+1)$ upper triangular matrix and ${\textbf{0}}_{a\times b}$ indicates the $a\times b$ zero matrix. Then, plugging (9) into (8) yields

$$\begin{aligned}{} & {} \left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_p+\bar{{\textbf{R}}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}(\bar{{\textbf{R}}}^{i+1})^\top \right) \nonumber \\{} & {} \quad = \begin{bmatrix} {\textbf{R}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}({\textbf{R}}^{i+1})^\top +\frac{1}{\sigma _{\theta }^2}{\textbf{I}}_{i+1} &{} {\textbf{0}}_{(p-i-1)\times (i+1)}^\top \\ {\textbf{0}}_{(p-i-1)\times (i+1)} &{} \frac{1}{\sigma _\theta ^2}{\textbf{I}}_{p-i-1} \end{bmatrix} \end{aligned}$$

(10)

Hence, we can further simplify the log-determinant in (8) as follows:

$$\begin{aligned}{} & {} \log \det \left( {\varvec{\Sigma }}(S_{i+1})^{-1}\right) \nonumber \\{} & {} \quad =\log \det \left( {\textbf{R}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}({\textbf{R}}^{i+1})^\top +\frac{1}{\sigma _{\theta }^2}{\textbf{I}}_{i+1}\right) \nonumber \\{} & {} \qquad +\log \det \left( \frac{1}{\sigma _\theta ^2}{\textbf{I}}_{p-i-1}\right) \end{aligned}$$

(11)

$$\begin{aligned}{} & {} \propto \log \det \left( {\textbf{R}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}({\textbf{R}}^{i+1})^\top +\frac{1}{\sigma _{\theta }^2}{\textbf{I}}_{i+1}\right) \end{aligned}$$

(12)

where (12) follows since the second term in (11) is irrelevant in finding the maximizing node at $(i+1)$th iteration. In this paper, noting that $\det ({\textbf{A}})+\det ({\textbf{B}})\le \det ({\textbf{A}}+{\textbf{B}})$ for positive definite matrices ${\textbf{A}}$ and ${\textbf{B}}$ [17], we propose to maximize the lower bound of (12) to yield a low-complexity selection process: Specifically, the selection process in (4) is approximated by

$$\begin{aligned} j^*\approx & {} \arg \max _{j\in S_i^C}\log \left( \det \left( {\textbf{R}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}({\textbf{R}}^{i+1})^\top \right) +\det \left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_{i+1}\right) \right) \nonumber \\ \end{aligned}$$

(13)

$$\begin{aligned}= & {} \arg \max _{j\in S_i^C}\log \det \left( {\textbf{R}}^{i+1}\left( {\textbf{M}}_{S_{i+1}}{\textbf{M}}_{S_{i+1}}^\top \right) ^{-1}({\textbf{R}}^{i+1})^\top \right) \end{aligned}$$

(14)

where (13) follows from the notion that ${\textbf{R}}^{i+1}$ and ${\textbf{M}}_{S_{i+1}}$ are positive definite since the covariance matrix ${\textbf{K}}$ is positive definite and ${\textbf{H}}$ a full column rank matrix and (14) from the irrelevancy of $\det \left( \frac{1}{\sigma _{\theta }^2}{\textbf{I}}_{i+1}\right)$ in (13) to the selection at $(i+1)$th iteration. In the optimization perspective, the selection process in (14) corresponds to the methodology which makes the worst case as good as possible for the estimation of the parameters with equal variances (i.e., ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p$). Notably, the proposed process in (14) is equivalent to the selection process for the estimation of unknown and deterministic parameters (i.e., ${{\varvec{\Sigma }}_{\theta }}^{-1}={\textbf{0}}$ in (3)). Later in Sect. 4.2, the assumption of ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p$ is relieved to evaluate the proposed method in numerical experiments.

We prove a theorem which presents a simple criterion to enable an efficient selection of the node that maximizes the log-determinant in (14) at each iteration.

Theorem

Let the ($i+1$)th column vectors ${\textbf{r}}_{i+1}$ of ${\textbf{R}}^{i+1}$ and ${\textbf{k}}_{i+1}$ of ${\textbf{K}}_{S_{i+1}}$ be given by ${\textbf{r}}_{i+1}^\top =[{\textbf{r}}^\top \quad r]$ and ${\textbf{k}}_{i+1}^\top =[{\textbf{k}}^\top \quad k]$, respectively. Let the ($i+1$)th row vector ${\textbf{m}}_{i+1}^\top$ of ${\textbf{M}}_{S_{i+1}}$ be given by ${\textbf{m}}_{i+1}^\top =[{\textbf{m}}^\top \quad m]$. Then, the node at the $(i+1)$th iteration that maximizes the log-determinant in (14) is simply selected from the set $S_i^C$ consisting of $(N-i)$ remaining nodes:

$$\begin{aligned} j^*=\arg \max _{j\in S_i^C} \frac{r^2}{m^2} \end{aligned}$$

(15)

where $m^2=k-\parallel {\textbf{m}}\parallel ^2$.

Proof

We continue to simplify the metric in (14) to yield

$$\begin{aligned} \log \det \left( {\varvec{\Sigma }}(S_{i+1})^{-1}\right) \approx \log \left( \det \left( {\textbf{R}}^{i+1}\right) ^2/\det \left( {\textbf{M}}_{S_{i+1}}\right) ^2\right) \end{aligned}$$

(16)

where (16) follows from $\det ({\textbf{A}})=\det ({\textbf{A}}^\top )$ and $\det ({\textbf{A}}^{-1})=1/\det ({\textbf{A}})$. In order to compute $\det \left( {\textbf{R}}^{i+1}\right)$ and $\det \left( {\textbf{M}}_{S_{i+1}}\right)$, it is noted that the matrices ${\textbf{R}}^{i+1}$ and ${\textbf{M}}_{S_{i+1}}$ can be simply obtained from those at the previous iteration. Specifically, for each $j\in S_i^C$ we compute ${\textbf{K}}_{S_{i+1}}$ from (6) and also construct ${\textbf{R}}^{i+1}$ and ${\textbf{M}}_{S_{i+1}}$ as follows:

$$\begin{aligned} {\textbf{R}}^{i+1}= & {} \begin{bmatrix} {\textbf{R}}^i &{} {\textbf{r}}\\ {\textbf{0}}^\top &{} r \end{bmatrix} \end{aligned}$$

(17)

$$\begin{aligned} {\textbf{M}}_{S_{i+1}}= & {} \begin{bmatrix} {\textbf{M}}_{S_i} &{} {\textbf{0}}\\ {\textbf{m}}^\top &{} m \end{bmatrix} \end{aligned}$$

(18)

Here, ${\textbf{r}}_{i+1}$ is given by ${\textbf{r}}_{i+1}^\top =[{\textbf{r}}^\top \quad r]=[{\textbf{q}}_1^\top {\textbf{h}}_j \quad {\textbf{q}}_2^\top {\textbf{h}}_j \quad \cdots \quad {\textbf{q}}_{i+1}^\top {\textbf{h}}_j]$ where $r={\textbf{q}}_{i+1}^\top {\textbf{h}}_j$. Also, the row vector ${\textbf{m}}_{i+1}^\top =[{\textbf{m}}^\top \quad m]$ can be easily computed by the LU factorization of the symmetric matrix ${\textbf{K}}_{S_{i+1}}$:

$$\begin{aligned} {\textbf{K}}_{S_{i+1}}= \begin{bmatrix} {\textbf{K}}_{S_i} &{} {\textbf{k}}\\ {\textbf{k}}^\top &{} k \end{bmatrix}= \begin{bmatrix} {\textbf{M}}_{S_i} &{} {\textbf{0}}\\ {\textbf{m}}^\top &{} m \end{bmatrix} \begin{bmatrix} {\textbf{M}}_{S_i}^\top &{} {\textbf{m}}\\ {\textbf{0}}^\top &{} m \end{bmatrix} \end{aligned}$$

(19)

Then, we have from (19)

$$\begin{aligned} {\textbf{M}}_{S_i}{\textbf{m}}={\textbf{k}}, \quad m^2=k-\parallel {\textbf{m}}\parallel ^2 \end{aligned}$$

(20)

Therefore, the log-determinant in (16) can be further simplified as follows:

$$\begin{aligned}{} & {} \log \left( \det \left( {\textbf{R}}^{i+1}\right) ^2/\det \left( {\textbf{M}}_{S_{i+1}}\right) ^2\right) \nonumber \\{} & {} \quad =\log \left( \det \left( {\textbf{R}}^i\right) ^2/\det \left( {\textbf{M}}_{S_i}\right) ^2\right) + \log \frac{r^2}{m^2} \end{aligned}$$

(21)

$$\begin{aligned}{} & {} \propto \log \frac{r^2}{m^2} \end{aligned}$$

(22)

where (22) follows since the first term in (21) is irrelevant in finding the maximizing node at the ($i+1$)th iteration. $\square$

It should be noted that since ${\textbf{M}}_{S_i}$ is lower triangular, ${\textbf{m}}$ in (20) can be directly obtained by the forward substitution without computing the inverse of ${\textbf{M}}_{S_i}$. However, (20) requires i floating-point divisions repeated over $(N-i)$ remaining nodes at the $(i+1)$th iteration. Further, the division operation typically spends much more time than the multiplication. Hence, we avoid those divisions by computing ${\textbf{m}}$ as follows:

$$\begin{aligned} {\textbf{m}}={\textbf{M}}_{S_i}^{-1}{\textbf{k}}, \quad m^2=k-\parallel {\textbf{m}}\parallel ^2 \end{aligned}$$

(23)

where ${\textbf{M}}_{S_{i+1}}^{-1}$ can be easily computed from ${\textbf{M}}_{S_i}^{-1}$ without inversion of large matrices by using

$$\begin{aligned} {\textbf{M}}_{S_{i+1}}^{-1}= \begin{bmatrix} {\textbf{M}}_{S_i}^{-1} &{} {\textbf{0}}\\ -\frac{{\textbf{m}}^\top {\textbf{M}}_{S_i}^{-1}}{m} &{} \frac{1}{m} \end{bmatrix} \end{aligned}$$

(24)

Note that (24) can be verified by multiplying ${\textbf{M}}_{S_{i+1}}^{-1}$ by ${\textbf{M}}_{S_{i+1}}$ in (18). It is obvious that (23) and (24) runs faster than (20) since (23) without divisions is repeated over ($N-i$) remaining nodes and (24) with one division is conducted once at each iteration.

Initially, ${\textbf{q}}_1={\textbf{h}}_j/\parallel {\textbf{h}}_j\parallel$, ${\textbf{K}}_{S_1}=k=k_{jj}$ with ${\textbf{k}}=0$ for $j\in {\mathcal {V}}$. Then we have $r={\textbf{q}}_1^\top {\textbf{h}}_j=\parallel {\textbf{h}}_j\parallel$ and $m^2=k$ with ${\textbf{m}}=0$ and the first node is selected as follows:

$$\begin{aligned} j^*=\arg \max _{j\in {\mathcal {V}}} \frac{r^2}{m^2}=\arg \max _{j\in {\mathcal {V}}} \frac{\parallel {\textbf{h}}_j\parallel ^2}{k_{jj}} \end{aligned}$$

(25)

Once the first node is selected, we update ${\textbf{M}}_{S_{i+1}}^{-1}$ by (24) (e.g., ${\textbf{M}}_{S_1}^{-1}=1/m$) and ${\textbf{Q}}$ by the QR factorization. The updated matrices ${\textbf{M}}_{S_{i+1}}^{-1}$ and ${\textbf{Q}}$ are used at the next iteration. In what follows, the proposed selection algorithm is briefly explained.

Algorithm 1

Efficient greedy sensor selection algorithm

4 Results and discussion

4.1 Complexity analysis of proposed algorithm

Given the covariance matrix ${\textbf{K}}$ of the correlated noise vector ${\textbf{w}}$ in (1), the proposed algorithm aims to construct the best set $S^*$ by conducting two main tasks. First, computation of the last column vector ${\textbf{r}}_{i+1}$ of ${\textbf{R}}^{i+1}$ is performed by using the QR factorization of ${\textbf{H}}_{S_{i+1}}^\top ={\textbf{Q}}\bar{{\textbf{R}}}^{i+1}$. Specifically, given ${\textbf{Q}}$ and ${\textbf{R}}^i$ at the previous iteration, the last entry of ${\textbf{r}}_{i+1}$ is computed for each $h_j,j\in S_i^C$. Once the maximizing node is selected, ${\textbf{Q}}$ is updated for the next iteration by the Householder transformation. The second is to compute ${\textbf{m}}_{i+1}$ by using (23) given ${\textbf{M}}_{S_i}^{-1}$ and ${\textbf{k}}_{i+1}$ for each node $j\in S_i^C$. After the selection of the maximizing node, ${\textbf{M}}_{S_{i+1}}^{-1}$ is updated for the next iteration by (24). Since these two tasks are repeated $|S|-1$ times, the proposed algorithm yields the complexity of $O(Np|S|^2)$ which is the same complexity order as that of [4].

We evaluate three previous selection methods for the performance comparison. First, we consider the sensor selection method denoted by greedy sensor selection (GSS) [5] which seeks to maximize the log-determinant of the inverse error covariance matrix ${\varvec{\Sigma }}(S)^{-1}$. GSS assumed uncorrelated noise (equivalently, ${\textbf{K}}={\textbf{I}}_N$) and the metric is given by $\log \det {\varvec{\Sigma }}(S)^{-1}\approx \log \det {\textbf{H}}_S^\top {\textbf{H}}_S$ under the assumption of ${{\varvec{\Sigma }}_{\theta }}^{-1}={\textbf{0}}$. Secondly, We compare with the method denoted by greedy sensor selection based on QR factorization (GSS-QR) in [4] which has been developed by applying the QR factorization of the matrix ${\textbf{H}}_S^\top$ in the MSE in (3). Unlike GSS, it aims to directly minimize the MSE(S) with ${\textbf{K}}={\textbf{I}}_N$. For this case, the MSE(S) is given by $tr\left[ \left( {\textbf{H}}_S^\top {\textbf{H}}_S\right) ^{+}\right] =\parallel {\textbf{H}}_{S}^+\parallel _F^2$ when ${{\varvec{\Sigma }}_{\theta }}^{-1}={\textbf{0}}$ is assumed. Thirdly, we evaluate the method denoted by greedy sensor selection under correlated noise (GSS-CN) [11, 12] which maximizes the same metric as the proposed algorithm. In Table 1, the complexity and the metrics of the selection methods are summarized for comparison. As seen, the proposed algorithm, GSS-QR and GSS-CN offer the same complexity order for the case of $|S|=p$. In the following section, the estimation performance of the methods is experimentally investigated for the case of $|S|=p$.

Table 1 Comparison with various greedy selection methods

4.2 Experimental results

In this section, we conduct the extensive experiments to validate the proposed sensor selection algorithm by comparing with various selection methods in three different cases as follows:

• Case 1: Random matrix ${\textbf{H}}$ with Gaussian iid entries, $h_{ij} \sim {\mathcal {N}}(0,0.1^2)$.

• Case 2: Random matrix ${\textbf{H}}$ with Bernoulli iid entries, $h_{ij}$ which take binary values (0 or 1) with the probability 0.5.

• Case 3: Random matrix ${\textbf{H}}$ and covariance matrix ${\textbf{K}}$ generated by using the linear reduced-order modeling [11, 12].

In Case 1 and 2, we generate 50 different realizations for each type of ${\textbf{H}}\in {\mathbb {R}}^{N\times p}$ with $N=1000$ and $p=10,15,\ldots ,40$. For each realization of ${\textbf{H}}$, we generate 100 test samples of the parameter vector ${\theta }$ and the correlated noise ${\textbf{w}}$ drawn from ${\mathcal {N}}({\textbf{0}},{\varvec{\Sigma }}_{\theta })$ and ${\mathcal {N}}({\textbf{0}},{\textbf{K}})$, respectively. The covariance matrix ${\textbf{K}}$ is constructed from random data generated from uniform distribution over $[0\quad 1]$. In Case 3, 100 measurement vectors ${\textbf{y}}\in {\mathbb {R}}^{1000}$ are generated from the normal distribution ${\mathcal {N}}(0,1)$ and ${\textbf{H}}$ and ${\textbf{K}}$ are constructed by using the linear reduced-order modeling with the order equal to $p=|S|$ (see [11, 12] for the details). From the matrices ${\textbf{H}}$ and ${\textbf{K}}$, we construct the set S with the cardinality $|S|=p$ by using the various sensor selection methods such as GSS, GSS-QR, GSS-CN and the proposed algorithm. We then select the measurements $y_i, i\in S$ from the measurement vector ${\textbf{y}}$ and estimate the parameter vector ${\theta }$ by employing the optimal linear estimator in (2). For evaluation of the estimation performance, we compare the MSEs of the selection methods given by ${\mathbb {E}}\frac{\parallel {\theta }-\hat{{\theta }}\parallel ^2}{\parallel {\theta }\parallel ^2}$.

4.2.1 Performance evaluation with respect to parameter dimension

In this experiment, we generate the parameters by assuming ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p, \sigma _{\theta }=0.1$ and run the selection methods in order to construct the sets S with $|S|=p$ by varying the parameter dimension $p=10,15,\ldots ,40$ and compare the MSEs which are plotted in Figs. 1, 2 and 3. It is noted that the proposed method shows better performance than GSS and GSS-QR since unlike the methods, the proposed algorithm takes into account the correlation of the noise to construct the set S. As expected, the proposed algorithm and GSS-CN yield identical MSEs because both of them optimize the same metric. Notably, as the parameter dimension increases, the estimation accuracy becomes improved because the optimal linear estimator using more measurements can suppress the effect of correlated noise more efficiently. Furthermore, we investigate the estimation performance of the methods with $p=|S|=50$ for different signal-to-noise ratios (SNRs) by varying the standard deviation of the parameter, $\sigma _{\theta }$ from 0.1 to 0.5. In Fig. 4, the MSEs are demonstrated in Case 1 (random Gaussian matrix ${\textbf{H}}$). Obviously, the selection methods yield more accurate estimation performance with higher SNR. We also test the methods by relaxing the assumption of ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p$ and generate the parameters from the non-diagonal covariance matrix ${\varvec{\Sigma }}_{\theta }$ constructed by using random data uniformly distributed over $[0\quad 1]$. Notice that the p diagonal entries of the covariance matrix ${\varvec{\Sigma }}_{\theta }$ take much higher values than $\sigma _{\theta }^2=0.1^2$ used in the experiment for the case of ${\varvec{\Sigma }}_{\theta }=\sigma _{\theta }^2{\textbf{I}}_p$. The MSEs are provided in figure 5, showing a robust estimation performance of the proposed method for the case of the non-diagonal covariance matrix. The complexity of the selection methods is experimentally examined by measuring the execution time in second. Figure 6 plots the execution times of the methods in Case 1 (random Gaussian matrix ${\textbf{H}}$) with respect to the parameter dimension $p=|S|=10,\ldots ,40$. The proposed method offers a competitive complexity as compared with GSS-QR and GSS-CN. Specifically, the proposed method operates about twice as fast as GSS-CN in all three cases.

Fig. 1

Evaluation of estimation performance in Case 1 (random Gaussian matrix): the proposed algorithm is compared with different selection methods with $\sigma _{\theta }=0.1$ by varying the dimension of the parameter, $p=|S|$

Fig. 2

Evaluation of estimation performance in Case 2 (random Bernoulli matrix): the proposed algorithm is compared with different selection methods with $\sigma _{\theta }=0.1$ by varying the dimension of the parameter, $p=|S|$

Fig. 3

Evaluation of estimation performance in Case 3 (linear reduced-order model): the proposed algorithm is compared with different selection methods by varying the dimension of the parameter, $p=|S|$

Fig. 4

Evaluation of estimation performance in Case 1 (random Gaussian matrix): the proposed algorithm is compared with different selection methods with $p=|S|=50$ by varying the standard deviation of the parameter, $\sigma _{\theta }$

Fig. 5

Evaluation of estimation performance for non-diagonal covariance matrix of the parameter in Case 1 (random Gaussian matrix): the proposed algorithm is compared with different selection methods by varying the dimension of the parameter, $p=|S|$

Fig. 6

Evaluation of complexity in Case 1 (random Gaussian matrix): the various selection algorithms are evaluated in terms of the execution time in second by varying the dimension of the parameter, $p=|S|$

5 Conclusions

In a situation where the measurements at sensor nodes are contaminated by correlated noise, we considered a problem of finding the best set of sensor nodes which minimizes the estimation error computed by the signal values at the selected nodes. Instead of directly minimizing the estimation error, we focused on maximizing the log-determinant of the inverse estimation error covariance matrix and applied the QR and LU factorizations so as to present a simple analytic selection rule. We also provided a complexity analysis of the proposed algorithm to reveal a competitive complexity when compared with the previous novel selection methods. We finally investigated the performance of the proposed algorithm in various cases in terms of estimation performance and execution time and demonstrated that the proposed algorithm yields more accurate estimation with a reasonable complexity than the previous methods.