Two-stage source tracking method using a multiple linear regression model in the expanded phase domain

Abstract

This article proposes an efficient two-channel time delay estimation method for tracking a moving speaker in noisy and re-verberant environment. Unlike conventional linear regression model-based methods, the proposed multiple linear regression model designed in the expanded phase domain shows high estimation accuracy in adverse condition because its the Gaussian assumption on phase distribution is valid. Therefore, the least-square-based time delay estimator using the proposed multiple linear regression model becomes an ideal estimator that does not require a complicated phase unwrapping process. In addition, the proposed method is extended to the two-stage recursive estimation approach, which can be used for a moving source tracking scenario. The performance of the proposed method is compared with that of conventional cross-correlation and linear regression-based methods in noisy and reverberant environment. Experimental results verify that the proposed algorithm significantly decreases estimation anomalies and improves the accuracy of time delay estimation. Finally, the tracking performance of the proposed method to both slow and fast moving speakers is confirmed in adverse environment.

1. Introduction

Time delay estimation (TDE) plays key role in determining the steering capability of microphone array system which produces a direction of the target sound source required for performing spatial processing. Typical applications of microphone array system include teleconferencing, automatic speech recognition, speech enhancement, source separation and automatic auditory system for robots [1–6].

The problem of estimating relative time delay associated with a signal source and a pair of spatially separated microphones has been extensively studied [7–15]. Among TDE methods, the generalized cross-correlation (GCC) method is one of the most widely used because of its simplicity and acceptable performance [7–9]. In the GCC-based method, the time delay is calculated by finding a lag that maximizes the GCC function between acquired signals. The method has been enhanced by introducing a pre-filter or a weighting function such as maximum-likelihood (ML), phase transform (PHAT) and so on. The GCC-ML method derived from the assumption of the ideal single propagation situation is optimal in a statistical point of view in case the observed sample space is large enough. The GCC-PHAT is recognized as reasonably robust to reverberation though it is heuristically designed. Zhang et al. [16] verified that the GCC-PHAT could actually be derived from the ML-based algorithm in reverberant environment if noise level is low. Another technique relied on the identification of the minimum of the average magnitude difference function (AMDF) between two signals, which was recently modified by joint consideration of the AMDF and the average magnitude sum function (AMSF) to improve the performance in reverberant environment [13].

An adaptive filter-based algorithm utilizes the criterion of minimizing the mean-square error between the first channel signal and the filtered second channel signal to estimate relative time delay [17]. In [18], an adaptive eigenvalue decomposition algorithm was proposed to improve TDE performance in reverberant environment. It first identified the room impulse response (RIR) of each channel, and then the delay was determined by finding the direct paths from the two measured RIRs. A systematic overview of the stat-of-the-art of TDE techniques was summarized in the recent literature [14].

The TDE method using the inter-channel phase difference (IPD) has been attracted a lot since 1980s, thanks to its advantage on obtaining the result instantaneously [19–23]. Chan et al. [19] verified that a least-square (LS) estimator to the phase slope of cross power spectrum was equivalent to the ML estimator. They also proved that the distribution of IPD error followed Gaussian probability density function (pdf) if the signal and noise were zero mean Gaussian processes and uncorrelated each other. By raising the coherence issue between dual-microphone noises, Piersol provided the relationship between spatial coherence function and phase bias at specific frequency. Brandstein et al. [20] proposed a generalized cost function of the linear regression model of IPD by adopting a bi-weight function [23]. The method is particularly advantageous in reverberant environment, but there is no benefit in noisy environment. The performance of these approaches commonly degrades when phase wrapping occurs or the phase is corrupted by adverse environmental effects because the phase statistics cannot be modeled by a simple pdf. Since it is hard to find an ideal estimator for a non-Gaussian data set such as wrapped discrete phases, a phase unwrapping process needs to be included in the TDE method [22, 24, 25]. Tribolet [24] proposed an iterative phase unwrapping algorithm that adaptively integrated the derivative of the phase. Brandstein et al. [22] practically implemented a linear regression slope forced unwrapping method which recursively adjusted the estimated wrapping frequency using lower band phase observations. Since these methods commonly include heuristic parts, their performance vary depending on how they are implemented. Recently, recursive unwrapping methods such as maximizing a posteriori probability or adopting the expectation-maximization (EM) using the probability model of the observed phase data set are introduced [26, 27]. In those methods, a reliable phase unwrapping can be achieved at the expense of heavy computational burden.

This article proposes a multiple linear regression model-based instantaneous TDE method that uses the expanded IPD of two channel signals. An estimator designed for operating in the original phase domain, [-π ~ + π), can hardly be optimal because a phase can be wrapped corresponding to the inter-channel distance and the direction of arrival (DOA) angle. To solve the problem, a reasonable statistical model for the distribution of IPD error and its Gaussian approximation are presented. At first, a phase domain expansion method using frequency interpolation and phase shifting methodology is proposed. Conventional linear regression model of IPD can be considered as a multiple linear regression model in the proposed phase expansion framework. By applying the proposed method to TDE, an ambiguous factor due to phase wrapping is dismissed and the LS method results in an optimal estimator. This article also verifies that the proposed estimation method becomes a minimum variance estimator (MVE) in the expanded phase domain. The proposed TDE method is composed of two stages: an LS-based TDE method estimates an initial delay at the first stage, and the estimated delay is applied to the sequential recursive-LS (RLS) estimator. The proposed method is computationally simple since it does not need a minimum or maximum search stage as well as the phase unwrapping process. The proposed algorithm is fairly compared with the optimal GCC methods, a generalized linear regression estimator, and an AMDF method in noisy and reverberant environment. The performance of the candidate estimators is evaluated by detailed assessment items including the percentage of anomalies, the estimation bias for both low and high DOA angles, and the root-mean-squared error (RMSE). Experimental results show that the proposed method can be regarded as the most robust estimator for the outliers and is closer to the unbiased estimator than any other methods. Especially in the RMSE assessment, the proposed RLS-TDE shows the best performance in both noisy and reverberant environment. Finally, the superiority of the tracking performance of the proposed algorithm is verified to a moving source in low SNR conditions.

The contents of the article are divided into four parts. Conventional two-channel TDE is explained in Section 2. Section 3 describes the details of the proposed phase expansion method with a multiple linear regression model. The proposed two-step TDE method for a moving speaker is described in Section 4. Finally, various experimental results are given in Section 5.

2. Conventional TDE method

2.1. Input signal model

Assuming that signals radiated by a single source, s(t), impinge on two channel microphones, each received signal can be represented by the following frequency domain formula [16, 23]:

$$X_i\left(\omega \right)=S\left(\omega \right)H_i\left(\omega \right)+N_i\left(\omega \right),i=1,2,$$

(1)

where N _i (ω) is the noise sensed by the i th microphone, and H _i (ω) is the transfer function between source and i th microphone. H _i (ω) can be modeled as [28, 29]

$$\begin{array}{ll}\hfill H_1\left(\omega \right)& ={\alpha }_0+\sum _{k=1}^{\infty }{\alpha }_k{e}^{-j\omega {\tau }_{a,k}},\\ \hfill H_2\left(\omega \right)& ={\beta }_0{e}^{-j\omega {\tau }_{\theta }}+\sum _{k=1}^{\infty }{\beta }_k{e}^{-j\omega {\tau }_{\beta ,k}},\end{array}$$

(2)

where α _k and β _k are attenuation factors normally less than one, τ _θ is the time difference of arrival (TDOA) between two input signals, and τ _α,k , τ _β,k are time delays caused by reverberation. The first term in each of Equation 2 is a direct component from source to microphone while the second term is a reverberant component related to RIR. In a far-field source scenario assumption, the propagation time difference of two microphones relating to the direction θ is defined as τ _θ = d sin(θ)/c, where d is a distance between two microphones and c is the sound velocity in the air. This article initially assumes the single path signal model that considers only the direct path signal and the additive noise term in Equation 1, and then it is extended to the multi-path environment case later.

2.2. Linear regression model-based TDE

The IPD between two channel signals is computed by subtracting phase terms, ∠X₁(ω) - ∠X₂(ω), where ∠X₁(ω) and ∠X₂(ω) are phases of input signals, respectively. Practically, the IPD can be calculated by investigating the phase of cross spectrum, $\angle \left(X_1\left(\omega \right)X_2^{*}\left(\omega \right)\right)$, or the imaginary part of log-spectral distance, Im {lnX₁(ω) - lnX₂(ω)}, between two channel signals. Then, the IPD, ξ(ω), can be expressed as

$$\xi \left(\omega \right)=\omega {\tau }_{\theta }+2\pi m+\nu \left(\omega \right),$$

(3)

where m is an integer number and 2πm represents a phase wrapping factor which constrains a phase range [-π ~ + π). ν(ω) denotes the IPD error caused by N _i (ω), H _i (ω), and ignorable minor impact due to using a finite length of DFT, etc. In Equation 3, the TDE is now reformulated in terms of a linear regression problem in that the time delay is found by fitting a line to the observed IPD. Without considering the wrapping factor, a weighted LS method has been widely used as a regression cost function. Thus, the final TDE is given as follows:

$$\begin{array}{ll}\hfill \widehat{\tau }& =\underset{\tau }{\text{arg}\mathsf{\text{min}}}\sum _k\psi \left({\omega }_k\right)|{\omega }_k\tau -\xi \left({\omega }_k\right){|}^2\\ ={\left(\sum _k\psi \left({\omega }_k\right){\omega }_k^2\right)}^{-1}\sum _k\psi \left({\omega }_k\right)\xi \left({\omega }_k\right){\omega }_k,\end{array}$$

(4)

where k = 0, 1, ..., K - 1 is discrete frequency indices, ${\omega }_k=\frac{2\pi k}{K}$ and ψ(ω _k ) is a weight to normalize the disturbances. Equation 4 becomes the best linear unbiased estimator (BLUE) when ψ(ω _k ) equals to the reciprocal of IPD error variance. Moreover, it becomes an MVU estimator if the pdf of IPD error, ν(ω), follows Gaussian distribution [30]. The performance of the above LS-TDE for an acoustic signal is statistically analyzed in previous articles under the Gaussian assumption of IPD error distribution [19, 20]. If phase wrapping is considered, however, the distribution of ν(ω) does not follow Gaussian anymore unless an ideal phase unwrapping is performed as a pre-processing step. Generally, it is not an easy task to find wrapped frequencies and unwrapped phase values in noisy environment. In addition, the unwrapping process for the IPD requires time delay information before performing the TDE processing. In the next section, a novel pdf model of IPD error distribution under a noisy condition is introduced. A phase expansion method with a multiple linear regression model is also proposed, which is more efficient and generally applicable to IPD-based methodologies but does not require any complicated phase wrapping process.

3. Multiple linear regression model in the expanded phase domain

3.1. Generalized IPD distribution: sum of shifted gaussian pdfs

Without loss of generality, the multi-path effect caused by reverberation is ignored at first. Then, ν(ω) in Equation 3 can be considered as a random variable related to the phase deviations caused by N₁(ω) and N₂(ω). If we assume that S(ω) = 0, and N₁(ω) and N₂(ω) are independent zero mean Gaussian random variables, ν(ω) follows uniform distribution with $\frac{{\pi }^2}{3}$ variance in [-π ~ + π) range [19]. On the other hand, when the signal power is relatively larger than the noise one, the pdf of ν(ω) can be approximated by zero mean Gaussian, whose variance is represented by signal power and magnitude coherence function (MSC) [19, 26, 31]. These properties are useful to estimate a time delay that uses the IPD of two channel signals.

In this article, we modify the approximated Gaussian IPD error model using an SNR parameter. Though the idea was initially proposed by Said et al. [31], they only assumed a case when the signal was incident from the zero direction, so that there was no need to consider the phase wrapping effects. Figure 1a shows a complex diagram of IPD error model to generalize the conventional model to all possible DOA angle range. In the figure, e^jϕis regarded as a normalized cross spectrum of two channel source signal with unit power and phase ϕ, and N _ϕ (ω) is that of noise. Note that the real and imaginary components of N_ϕ(ω) are assumed as independent Gaussian random variables. The inner circle in Figure 1a represents the maximum range of erroneous phase distribution caused by N _ϕ (ω). The SNR of cross spectrum becomes (2γ²)^-1 if the variances of real and imaginary parts of the noise are γ², respectively. Therefore, the outer circle in the figure shows the maximum phase distribution when signal power is same as noise power. In this 0 dB SNR case, the absolute phase error is limited to a value of smaller than $\frac{\pi }{4}$ and it is presented in Figure 1b. The limited phase interval in Figure 1b is larger than +π; however, it is not a problem in the proposed expanded phase domain (it will be described in next subsection).

Figure 1

Gaussian assumption of IPD error model for DOA angle, ϕ , caused by uncorrelated noises and the actual IPD distribution in [- π ~ + π ): (a) complex diagram of IPD error model, (b) probability density function of IPD based on the sum of shifted Gaussian pdfs.

From Figure 1a, the pdf of IPD error for the true phase ϕ with the phase error ζ (omitting ω for simplicity) can be computed by following integral function:

$$p_{\varphi ,\zeta ,\gamma }={\int }_0^{\infty }\frac{r}{2\pi {\gamma }^2}{e}^{\frac{{\left(r\text{cos}\left(\varphi +\zeta \right)-\text{cos}\varphi \right)}^2+{\left(r\text{sin}\left(\varphi +\zeta \right)-\text{sin}\varphi \right)}^2}{-2{\gamma }^2}}dr,$$

(5)

which equals to (Appendix A)

$$p_{\varphi ,\zeta ,\gamma }=\frac{1}{2\pi }{e}^{-\frac{1}{2{\gamma }^2}}+\left(\frac{1}{\sqrt{2\pi {\gamma }^2}}{e}^{\frac{{\text{sin}}^2\zeta }{-2{\gamma }^2}}\right)\left(Q\left(\frac{\text{cos}\zeta }{-\gamma }\right)\text{cos}\zeta \right),$$

(6)

where Q-function is defined as $Q\left(x\right)={\left(2\pi \right)}^{-1/2}{\int }_x^{\infty }{e}^{-t^2/2}dt$. Equation 6 is composed of three components, additive positive constant, approximation of Gaussian pdf, and cosine multiplied Q-function terms. Finally, the IPD distribution for an arbitrary phase ϕ is expressed in the same way to the Said's method which forms a symmetric distribution focusing on ϕ [31]. It is increasingly flattened for higher level noise because the first term in Equation 6 becomes a principle factor, i.e. reducing the delay information contained in the IPD. However, if we assume that the SNR of cross spectrum is high (γ² ≪ 1) then majority of IPD error is concentrated on zero. In this assumption, the first term of Equation 6 can be disregarded and the approximations for sinusoidal function in the second term, sin(x) ≈ x and cos(x) ≈ 1, would be valid. Therefore, Equation 6 can be simplified as follows:

$$p_{\varphi ,\zeta ,\gamma }\approx \frac{1}{\sqrt{2\pi {\gamma }^2}}\text{exp}\left(\frac{{\zeta }^2}{-2{\gamma }^2}\right),{\gamma }^2\ll 1.$$

(7)

Equation 7 denotes Gaussian pdf with a variance of γ² which is related to the inverse of the SNR. Figure 2 depicts the comparison of the original pdf given in Equation 6 and its Gaussian approximation in Equation 7 in relatively low and high SNR conditions. The approximated IPD distribution depicted as the solid line is flatter than original IPD distribution due to the influence of the additive term and Q-function in -5 dB SNR. It is clear that the approximated pdf given in Equation 7 is getting closer to the original IPD distribution as the SNR increases. The actual IPD, however, is not normally distributed when there exists phase wrapping. As shown in Figure 1b, as ϕ closes to +π (or -π), it is likely that phase wrapping occurs. The solid line in Figure 1b is actual IPD distribution when phase wrapping occurs, which is obtained by the infinite sum of 2π shifted Gaussian pdfs of Equation 7 (circle markers) in -π to +π range. It is clear that the IPD distribution for the wrapped phase is non-symmetric and dense at erroneous arbitrary phase. Consequently, the actual shape of p _ϕ,ζ,γ cannot be regarded as Gaussian and completely depends on the actual phase at each frequency. In the following subsection, we derive a linearly interpolated phase expansion method to cope with the problem caused by the non-Gaussian IPD distribution. The IPD distribution in the expanded phase domain is shown as the dash-dotted line in Figure 1b.

Figure 2

Comparison between original pdf ( dotted-lines ) and its Gaussian approximation ( solid-lines ) in high and low SNR conditions.

3.2. Multiple linear regression model in the expanded phase domain

If phase wrapping occurs the Gaussian assumption becomes invalid thus a delay estimator which does not include a maximum searching process easily fails. Conventional linear regression model basically assumes that the phase is linear and always starts from zero at zero frequency. However, phase wrapping results in discontinuity due to the shifting phase term, ±2π given in Equation 3. The purpose of phase expansion proposed in this article is recovering linear parallel lines by shifting original phase terms and copying it to the interpolated frequency domain which is defined as the multiple linear regression model. Figure 3 depicts an example of phase expansion under the assumption that there exists at most one phase wrapping. It is a reasonable assumption because the second wrapping is hardly occurred in the tested speech signal band unless we use a very large microphone array, e.g., the second wrapping can be occurred at higher than 5.1 kHz when dual-microphone space is 0.1 m.

Figure 3

Three cases of the expanded phases: (a) no wrapping occurred case, (b) wrapped and positive slope case (c) wrapped and negative slope case.

Details of phase expansion stage are represented in Figure 4, where k and/are original and interpolated frequency indices and ξ _E (ω _l ) is the expanded discrete phase after applying the proposed interpolation process. First, the original phase is copied to 4-times of interpolated frequency, ω_4k. Then, it copies the +2π shifted interpolated phase to ω_4k+1and repeats it for -2π shifting to ω_4k+2. Therefore, a linear phase line starting from zero is recovered though there may exist two lines which lie on either zero to wrapping frequency or wrapping frequency to end. To make these two lines linear from zero to end, +4π (or -4π) shifting and copying process is needed only for the original phase which is smaller (or larger) than zero. Finally, the system determines a proper expanded domain which is shown as the widely shaded area in Figure 3. As we can see in Figure 3, only three possible multiple linear regression models are needed to be considered in our phase expansion method. The expanded phase is commonly distributed in 6π range though the expanded domains, Ω _d , d = -1, 0, 1 are not identically distributed. Moreover, there always exist ideal three linear parallel lines in Ω _d that make the LS-TDE derivation possible. The verification process will be followed in next section.

Figure 4

Details of the proposed linearly interpolated phase expansion.

4. A Framework of the proposed two-stage method

The multiple linear regression model-based LS method for IPD estimation is proposed in the expanded phase domain, Ω _d . The proposed method is composed of two stages: the multiple linear regression model-based LS-TDE at the first stage, and the RLS-based source tracking method using the delay information estimated at the first stage. After constructing an LS cost function for the TDE method based on the multiple linear regression model, it is verified that the proposed LS method is an ideal estimator which is unconstrained by phase wrapping. In the second stage, the RLS-TDE method is proposed which works very well for both fixed and moving source tracking. The proposed RLS method can be implemented by a simple equation, and it is also appropriate for conversational speech. Finally, a novel two-channel weighting method for noisy and reverberant environment is described.

4.1. First stage: multiple linear regression model-based TDE

In Section 3.2, the multiple linear regression model including three-linear lines in 6π interval is explained in detail. The proposed LS criterion using the multiple linear regression model is given as

$${\widehat{\tau }}_{E,d}=\text{arg}\underset{\tau }{\text{min}}\sum _{m=-1}^1\sum _l|\left({\omega }_l\tau +2m\pi -{\xi }_{E,d}\left({\omega }_l\right)\right){|}^2,$$

(8)

where d = -1, 0, 1 is the expanded domain index, l = 0, 1, ..., 4K -1 is the interpolated frequency index, and ξ_E,d(ω _l ) ∈ Ω _d is the expanded observation phase for each case in Figure 3. Then, the LS solution is derived by taking a derivative to the term τ as follows:

$$0=6\sum _l\left({\omega }_l^2\tau -{\omega }_l{\xi }_{E,d}\left({\omega }_l\right)\right)+4\pi \sum _{m=-1}^{1}m\sum _l{\omega }_l.$$

(9)

The second term in Equation 9 corresponding to phase shifting is equal to zero. Therefore, the proposed multiple linear regression model-based LS-TDE in the expanded phase domain is equivalent to the conventional LS equation given in Equation 4. Finally, the proposed LS solution is easily calculated by adopting a vector notation, ${\widehat{\tau }}_{E,d}={\left({\bar{\omega }}^H\bar{\omega }\right)}^{-1}{\bar{\omega }}^H{\bar{\xi }}_{E,d}$ where $\bar{\omega }$ and ${\bar{\xi }}_{E,d}$ are L _d × 1 vectors, L _d is the number of discrete frequencies satisfying ξ_E,d(ω _l ) ∈ Ω _d . A weighted solution which does not affect above derivation is given as

$${\widehat{\tau }}_{E,d}={\left({\bar{\omega }}^H\Psi \bar{\omega }\right)}^{-1}{\bar{\omega }}^H\Psi {\bar{\xi }}_{E,d},$$

(10)

where Ψ is a diagonal matrix composed by a reciprocal of IPD error variance related to the SNR of the input signal. The variance of IPD error at interpolated frequency is same as original variance. The proposed solution in the expanded phase domain, Equation 10, is not only unconstrained by the phase wrapping but also corresponding to the ideal LS solution of Equation 4. Furthermore, Equation 10 becomes an MVU estimator since the Gaussian assumption for the IPD error, Equation 7, is valid in the expanded phase domain. Finally, the estimator determines the most accurate delay among the estimated results in each expanded phase domain by measuring Euclidean distance between the estimated and the observed phases as follows:

$${\widehat{\tau }}_{LS}=\text{arg}\underset{d}{\text{min}}{\left(\sum _l{\left({\omega }_l{\widehat{\tau }}_{E,d}-{\xi }_{E,d}\left({\omega }_l\right)\right)}^2\right)}^{1/2},d=-1,0,1.$$

(11)

4.2. Second stage: RLS for moving speaker tracking

Generally, an LS-TDE in a single-frame-based process easily confronts the lack of data problem because the frame length for analyzing speech signal is only 20-30 ms and the sampling frequency is limited to the capacity of usual electronic devices. As the more data set is available, the performance of TDE becomes closer to the ideal lower bound such as Cramer-Rao bound (CRB) [30, 32]. To use multiple frames for TDE, however, non-stationarity of the speech signal and moving source case should be considered. This article proposes an RLS-TDE method which improves the performance of TDE by considering an arbitrarily moving speaker. At first, the LS-TDE result, ${\widehat{\tau }}_{LS}$, of the first stage is used to select the frequencies for the RLS processing as follows:

$$\left\{{\omega }_l\left|\right|{\omega }_l{\widehat{\tau }}_{LS}-{\xi }_E\left({\omega }_l\right)|<\pi \right\},l=0,1,\dots ,L-1.$$

(12)

Using the criterion given in Equation 12, the frequencies whose phases within a 2π interval around a straight line, $f\left({\omega }_l\right)={\omega }_l{\widehat{\tau }}_{LS}$, are selected as candidates for the second stage. Three new vectors are defined to simplify the equation such that, ${\bar{\omega }}_r\left(n\right)$ is the frequency vector satisfying Equation 12 at n th frame and ${\bar{\xi }}_r\left(n\right),{\Psi }_r\left(n\right)$ are related phase vector and diagonal matrix of weighting vector, respectively. Then, the RLS criterion is given as

$$J=\sum _{q=0}^Q{\delta }^q\left(\sum _{m=-1}^1{Ā}^T\left(m,n-q\right){\Psi }_r\left(n-q\right)Ā\left(m,n-q\right)\right),$$

(13)

where T means vector transpose, δ is a positive constant less than one, Q is the maximum number of observation frames. The criterion vector, $Ā\left(m,n\right)$, and the arbitrary vector, $Ī$, are defined as

$$Ā\left(m,n\right)=\left({\bar{\omega }}_r\left(n\right)+2\pi mĪ-{\bar{\xi }}_r\left(n\right)\right),Ī={\left[1,\dots ,1\right]}^T.$$

(14)

Finally, the RLS-TDE is represented by

$${\widehat{\tau }}_{\mathsf{\text{RLS}}}\left(n\right)=\frac{\sum _{q=0}^Q{\delta }^q\left({\bar{\omega }}_r^T\left(n-q\right){\Psi }_r\left(n-q\right){\bar{\xi }}_r\left(n-q\right)\right)}{\sum _{q=0}^Q{\delta }^q\left({\bar{\omega }}_r^T\left(n-q\right){\Psi }_r\left(n-q\right){\bar{\omega }}_r\left(n-q\right)\right)}.$$

(15)

Equation 15 is same as Equation 10 except the term δ^qwhich exponentially decreases the contribution of the past data set. In addition, a process is included such that all of the RLS vectors are initialized when long silence interval is included in the observation data. Experimental results described in detail later confirm that the performance of RLS-TDE is superior to conventional methods even for the fast moving speech source.

4.3. Weighting for LS-TDE in noisy and reverberant condition

In Section 3.1, it is shown that the IPD error distribution can be regarded as Gaussian with variance (2 × SNR)^-1. Actually, this property is implied in the ML TDE explained in the Knapp's method [9] that the ML weighting is derived from MSC. Note that MSC can be regarded as an SNR of the input signal. In practice, MSC must be estimated by the observed data set using a temporal averaging method [33]. However, it is hard to estimate accurate MSC for non-stationary data such as speech signal. The proposed method adopts an approximated-ML weighting which is roughly equivalent to the SNR evaluated from a single frame as follows [12, 22, 23]:

$$\psi \left({\omega }_k\right)=\frac{\left|X_1\left({\omega }_k\right)\right|\left|X_2\left({\omega }_k\right)\right|}{\left|N_1\left({\omega }_k\right){|}^2\right|X_2\left({\omega }_k\right){|}^2+\left|N_2\left({\omega }_k\right){|}^2\right|X_1\left({\omega }_k\right){|}^2}.$$

(16)

The proposed LS-TDE in the expanded phase domain given in Equation 10 with the weighting function above satisfies all the ML estimation conditions, e.g., the Gaussian assumption of IPD error and weighting of its variance reciprocal. The weighting given in Equation 16 is useful when the coherence between two noises of dual-sensor and the target speech signal are ignor-able. However, it cannot distinguish values of speech from other signals if we assume a reverberant environment. Piersol [20] paid attention to the spatial coherence between two-sensors and proved the effects to the TDE by lots of experimental results, which are consistent with the theoretical analysis. To design a practical two channel system under the reverberant environment, a substitutable method which can suppress the reverberation effect by signal-to-reverberation (SRR)-based weighting is introduced.

To estimate the power of the direct signal and reverberant components, a two-channel generalized side-lobe canceller (GSC) structure is adopted [34]. Figure 5 shows a simplified block diagram to estimate the direct signal power. In this method, the power envelop of the delay-and-sum beamformer (DSB) output, Q(ω, n), and the delay-and-subtract output used for a reference signal, U(ω, n), are obtained by using the first-order recursive equations:

Figure 5

Block diagram of the GSC-based direct signal power estimation.

$$\begin{array}{ll}\hfill {\lambda }_q\left(w,n\right)& =\eta {\lambda }_q\left(\omega ,n-1\right)+\left(1-\eta \right)|Q\left(\omega ,n\right){|}^2,\\ \hfill {\lambda }_u\left(\omega ,n\right)& =\eta {\lambda }_u\left(\omega ,n-1\right)+\left(1-\eta \right)|U\left(\omega ,n\right){|}^2,\end{array}$$

(17)

where n is frame index and η is a forgetting factor set close to, but less than, one. Then, the energy of reverberant residual components, ${\widehat{\lambda }}_r\left(\omega ,n\right)$ is obtained as follows:

$${\widehat{\lambda }}_r\left(\omega ,n\right)=W\left(\omega ,n\right){\lambda }_u\left(\omega ,n\right),$$

(18)

where W(ω, n) is a frequency dependent gain that is adaptively updated using a quadratic cost function, J _w = {λ _e (ω, n)}², where the error, λ _e (ω, n), is equal to ${\lambda }_q\left(\omega ,n\right)-{\widehat{\lambda }}_r\left(\omega ,n\right)$. Finally, the direct signal power is estimated using a spectral-subtraction method [35]:

$$|{Ŝ}_d\left(\omega ,n\right){|}^2=|Q\left(\omega ,n\right){|}^2-{\widehat{\lambda }}_r\left(\omega ,n\right).$$

(19)

In Habets's de-reverberation method [34], a post filter is applied to the DSB output, Q(ω, d), however, the spectral subtraction method, given in Equation 19, is good enough in our application because only the power envelop of the direct signal component is needed. Finally, the SRR is represented as follows (omitting frame index similar to Equation 16):

$$\psi \left({\omega }_k\right)=\frac{|{Ŝ}_d\left({\omega }_k\right){|}^2}{{\widehat{\lambda }}_r\left({\omega }_k\right)}.$$

(20)

The proposed method well suppresses the late reverberation but has no impact on the early reflected component which is the principle reason of bias for the IPD distribution. The bias caused by early reflection entirely depends on the physical conditions including the shape of room, sensor and source position, etc. It is still a challenging research area to deal with the early reflection blindly.

5. Experimental results

To verify the performance of the proposed algorithm, the performance of the proposed algorithm (p1(LS), p2(RLS)) is compared to the widely attracted methods that have reliable performance in noisy and reverberant environment. First, GCC-based methods that are preferred in a practical system are considered. Generalized GCC-TDE equation in frequency domain is as follows:

$${\widehat{\tau }}_{GCC}=\text{arg}\underset{\tau }{\text{max}}\left(\sum _k{\psi }_{GCC}\left({\omega }_k\right)G_{X_1{X}_2}\left({\omega }_k\right)e^{j{\omega }_k\tau }\right),$$

(21)

where $G_{X_1{X}_2}\left(\omega \right)$ is the cross spectrum of two channel signal, $X_1\left(\omega \right)X_2^{*}\left(\omega \right)$. The GCC-ML, ψ _ML (ω _k ) given in Equation 16, and the phase transform (PHAT), ${\psi }_{PHAT}\left({\omega }_k\right)=|X_1\left(\omega \right)X_2^{*}\left(\omega \right){|}^{-1}$, are well-known estimators used for noisy and reverberant environments, respectively.

Second, tests include the bi-weight (BIWT) method that are proposed to have robust performance especially for the outliers caused by the reverberation [23].

$${\widehat{\tau }}_{\mathsf{\text{BIWT}}}=\text{arg}\underset{\tau }{\text{min}}\left(\sum _k\rho \left(\frac{\xi \left({\omega }_k\right)-{\omega }_k\tau }{B\left({\omega }_k\right)}\right)\right),$$

(22)

where the bi-weighting function is given as

$$\rho \left(x\right)=\left\{\begin{array}{cc}\hfill -{\left(1-x^2\right)}^3/6,\hfill & \hfill \left|x\right|\le 1,\hfill \\ \hfill 0,\hfill & \hfill \left|x\right|>1.\hfill \end{array}\right.$$

(23)

The estimator given in Equation 22 can be regarded as a linear regression type for the cross spectrum phase. In fact, the weighted LS-TDE is a special case of the method given in Equation 22 with ρ(x) = x² and B(ω _k ) = ψ^-1/2(ω _k ). This alternative regression cost function shows the robust performance to the outliers by assigning a maximal error value to any scaled absolute residuals having larger than one. For a large value of B(ω _k ), spurious peaks in delay search range are diminished while the resolution of the TDE result is decreased. In this experiment, we set a constant value, B(ω _k ) = π/3, based on lots of simulations. Finally, a modified AMDF (m-AMDF) method which is robust to reverberant environment is considered [13]. The performance of the AMDF estimator is known as better than that of the GCC method in favorable noise conditions. The modified AMDF method is implemented in the frequency domain whose estimation equation is given as

$${\widehat{\tau }}_{\mathsf{\text{AMDF}}}=\text{arg}\underset{\tau }{\text{min}}\left(\frac{|X_1\left({\omega }_k\right)-X_2\left({\omega }_k\right)e^{j{\omega }_k\tau }|}{|X_1\left({\omega }_k\right)+X_2\left({\omega }_k\right)e^{j{\omega }_k\tau }|+\epsilon }\right),$$

(24)

where ε is a fixed positive number to prevent division overflow. The TDE of the modified AMDF, Equation 24, is determined by jointly considering the AMDF and the AMSF. The three reference TDE estimators commonly include a maximum (or minimum) searching process which requires a large amount of computation while the proposed method instantly estimates the time delay with an intra-sample precision.

In the experiment, four conversational speech signals from four different speakers, two-males and two-females are included into the test. An energy ratio-based voice activity detection (VAD) is designed and same voice active intervals are applied to different SNR conditions. The noise PSD of cross spectrum signal gathered in silence intervals is used to calculate the weighting term given in Equation 10. It is also used to GCC-ML to minimize weighting effect. The relative performance of the TDE was evaluated through a number of trials in a simulated rectangular room (12 × 10 × 3 m³). The microphone array is located at (3,3,2) and the distance from the source to the array is maintained 3 m for both fixed and moving source scenarios. We tested eight locations of the fixed source at intervals of 10° from 0° to 70°. The room environment is artificially generated by the modified frequency domain image source model (ISM) with negative reflection coefficients [28, 29]. The reverberation time, T₆₀, is measured by Lehmann's energy decay curve (EDC) [28]. The level of the additive white Gaussian noise (WGN) varies from 5 to 25 dB as the reverberation time is increased from 0 to 500 ms. The sampling frequency is 8000 Hz, 64 ms Hamming window is applied with 50% overlap and the space of microphone is set to 8 cm.

5.1. Fixed source case in noisy and reverberant environments

At first, it is verified whether the actual distribution of the expanded phase follows Gaussian pdf. In Figure 6, the dotted-line depicts a histogram of the expanded phase and the dashed-line shows the IPD of observed signals in the original phase domain at 1500 Hz in 5 dB SNR condition when true IPD is +2π/3. The IPD distribution in the original phase domain (dashed-line) is not symmetric and also a number of phases is concentrated in erroneous IPD near -π region. The solid-line is the approximated Gaussian pdf of Equation 7 with +2π/3 mean and (2 × SNR)^-1 variance. It shows that the IPD of real data is slightly biased to zero direction but we can confirm that the proposed Gaussian assumption in the expanded phase domain is quite reasonable. Figure 7 depicts the IPD distribution in the relatively high reverberant environment. The variance of Gaussian pdf cannot be evaluated from the proposed model in reverberant condition, so we set an appropriate value. In reverberant condition, the actual IPD is more biased to zero than the noisy environment and this phenomenon entirely depends on room environment.

Figure 6

Comparison of IPD distribution at 1500Hz in noisy environment, SNR = 5 dB.

Figure 7

Comparison of IPD distribution at 1500Hz in reverberant environment, SNR = 20 dB, T60 = 500 ms.

Figures 8, 9, 10 show the delay estimation results in various SNR conditions. The quantitative results by a percentage of outliers, bias and RMSE are evaluated to previously presented five techniques including the proposed LS and RLS methods. The bias and the RMSE are measured using the estimation result except for the outliers. In Figure 8, anomalies percentage is measured when the estimated time delay exceeded 20% of the overall delay range. The GCC-PHAT method shows the worst result, which has a number of severely erroneous estimation outliers in low SNR condition. The GCC-ML and the bi-weight method show similar performance while the bi-weight method shows slightly lower performance in low SNR environment. The AMDF method shows the best result among the comparing methods while it shows certain amount of outliers in low SNR conditions comparing to the proposed method. The proposed LS and RLS have similar anomalies percentage in the proposed algorithm structure, and both have superior performance to others such that the anomalies are suppressed less than 5% even in low SNR condition.

Figure 8

Anomalies percentage comparison in noisy environments.

Figure 9

Bias comparison in noisy environments: (a) bias for low DOA angle (0-40^°), (b) bias for high DOA angle (50-70^°).

Figure 10

TDE performance comparison in noisy environments.

The trend of estimation bias is represented in Figure 9 which shows the results in the low DOA angle and high DOA angle cases separately. The phase of high DOA angle cases are commonly wrapped because the wrapping is occurred when the DOA angle is lager than 32^° in our simulation condition. All of the tested algorithms are hardly biased when the source is located in front direction of dual-sensor as depicted in Figure 9a because the phase wrapping is less likely to occur for a low DOA angle incident case. As shown in Figure 9b, however, the estimation bias for a signal from the high DOA angle generally increases. Since the bias problem becomes more serious when the IPD is getting closer to +π (or -π) as we described in Figures 6 and 7. The proposed algorithm working in the expanded phase domain, however, does not suffer from the bias especially in noisy environment.

The final estimation performance is presented in Figure 10 which depicts RMSE results of averaged whole DOA angles. It is confirmed that the proposed method has superior performance to conventional ones in overall SNR conditions. The proposed LS and the AMDF methods show better performance than the GCC-ML and the bi-weight method while the performance of the bi-weight method and the AMDF method decrease in low SNR condition. The GCC-PHAT shows the worst performance in noisy environment.

Figures 11, 12, 13 show the performance of the test algorithms in reverberant environments. The reverberant signal is synthesized using the ISM model including 20 dB WGN for all conditions. Figure 11 depicts the anomalies percentage in reverberant environments. The number of severe outlier is rapidly increased by increasing T60 for the GCC-ML method while the GCC-PHAT, the AMDF and the bi-weight methods show robust performance even to the long reverberation time. Especially, the AMDF method shows the highest outlier suppression performance that does not affect by the reverberation. From Figures 8, 11, it is clear that the performance of these methods is sensitively affected by noise but robust to reverberation because it is originally designed to targeting reverberant conditions. The outlier suppression performance of the proposed method is similar to the bi-weight method that the anomaly percentage is limited by 10% in reverberant environments.

Figure 11

Anomalies percentage comparison in reverberant environments.

Figure 12

Bias comparison in reverberant environments: (a) bias for low DOA angle (0-40^°), (b) bias for high DOA angle (50-70^°).

Figure 13

TDE performance comparison in reverberant environments.

The estimated bias represented in Figure 12 shows a different trend comparing to the result in noisy environment such that the bias can occur regardless of the DOA angle. The GCC-PHAT method shows the most robust performance irrespective of the reverberation level while its performance also slightly degrades in the high DOA angle case. The other methods except for the GCC-PHAT show that the estimation bias is larger than the result in noisy environment and it is highly affected by the RIR.

Finally, the estimation error except for anomalies is depicted in Figure 13. The GCC-ML method has a relatively small error in low reverberation condition but the error dramatically increases as the reverberation increases. Among the methods immune to reverberation, the AMDF method shows the best performance in overall conditions. As with the previous RMSE results in noisy environment, the proposed two-step method with the SRR-based weighting shows the most accurate TDE results in reverberant environment comparing to the other methods.

Overall, it is verified that the proposed method shows the highest performance especially in the noisy environments, i.e. it has minimum error and the estimation anomalies is less than 5% even in low SNR condition. It is also verified that unlikely to other methods, the proposed multiple linear regression model-based TDE method is not biased by phase wrapping. It also shows the most accurate TDE results in reverberant environments. The proposed method shows similar results to the AMDF method which shows the best performance among the reverberation immune methods in the anomalies percentage and the bias measurements.

5.2. Source tracking scenario for slow and fast moving sources

The tracking performance of the proposed method is verified when the target speech source moves around. A conversational speech of 30 s length is tested in low SNR condition (5 dB). The RLS parameters for slow and fast moving sources are set δ = 0.9, Q = 9 in 64 ms in 50% OLA frame work that the maximum length of observation frames is 352 ms and the minimum weighting for last frame is 0.9⁹. The TDE results of the proposed LS and RLS for slow moving speaker are depicted in Figure 14. From the result, it is also confirmed that the RLS estimation tracks the true TDE (solid-line) better than the first step of the proposed estimation, LS. Note that the proposed RLS can find the true TDE value even there exists a long silence interval. Identical experiments are carried out for fast moving source and the results are depicted in Figure 15. The tracking performance for fast moving source is also good though there are some failed estimations around 11 and 24 s when the source moves from the high DOA angle directions. The proposed RLS, however, adapts the true TDE very quickly and tracks the speaker again even in these cases. The trends of tracking performance of the proposed two-stage method and the conventional methods are presented in Figure 16. For a fair comparison, a smoothing technique is applied to the result of the conventional TDE using a exponentially decaying sliding window that has a same fashion to the second stage of the proposed method. The tracking performance of the conventional methods shows a similar trend to the previous result for a fixed speaker. The proposed two-stage method (RLS) shows the best result in accuracy and the performance does not be affected by the velocity of a moving source.

Figure 14

Tracking performance of the proposed two-stage method for slow moving source: DOA estimation results of the first (LS) and the second stage (RLS).

Figure 15

Tracking performance of the proposed two-stage method for fast moving source: DOA estimation results of the first (LS) and the second stage (RLS).

Figure 16

Comparison of tracking performance between the proposed two-stage method (RLS) and the smoothed TDE of conventional methods in 5dB SNR condition.

6. Conclusion

A LS TDE method based on the multiple linear regression model via the interpolated phase expansion has been proposed. By the proposed phase expansion method, the IPD distribution between two channel signals becomes more advantageous in terms of pdf. It theoretically verified that the approximated Gaussian approaches to the actual IPD distribution for higher SNR and also confirmed it by various experimental results. The proposed TDE method which is composed of two stages shows superior performance especially in the anomalies percentage and RMSE results in both noisy and reverberant environments. It was also demonstrated that the bias to zero problem for high DOA angles could be mitigated in the proposed method. Finally, the superiority of the proposed algorithm in terms of tracking a moving source in low SNR condition was verified. The proposed method provides the explicit TDE solution that can be applied to a real time application. Future work involves improving the method in reverberant environments based on detailed investigation about the IPD statistics for a multi-path effects.

Appendix A: Simplifying the IPD pdf

The numerator of the power term in Equation 5 is simplified by the trigonometric identities as follows:

$$\begin{array}{c}{\left(r\text{cos}\left(\varphi +\zeta \right)-\text{cos}\left(\varphi \right)\right)}^2+{\left(r\text{sin}\left(\varphi +\zeta \right)-\text{sin}\left(\varphi \right)\right)}^2\\ =r^2-2r\left(\text{cos}\left(\varphi +\zeta \right)\text{cos}\left(\varphi \right)+\text{sin}\left(\varphi +\zeta \right)\text{sin}\left(\varphi \right)\right)+1\\ ={\left(r-\text{cos}\left(\zeta \right)\right)}^2+1-{\text{cos}}^2\left(\zeta \right).\end{array}$$

(25)

Then, the pdf function becomes

$$p_{\varphi ,\zeta ,\gamma }=\frac{1}{2\pi {\gamma }^2}\left(e^{\frac{{\text{cos}}^2\left(\zeta \right)-1}{2{\gamma }^2}}\right)\left({\int }_0^{\infty }r{e}^{\frac{{\left(r-\text{cos}\left(\zeta \right)\right)}^2}{-2{\gamma }^2}}dr\right).$$

(26)

By substituting t = r - cos(ζ) to solve the integral function, the pdf function can be simplified as follows:

$$\begin{array}{ll}\hfill p_{\varphi ,\zeta ,\gamma }& =\frac{1}{2\pi {\gamma }^2}\left(e^{\frac{{\text{cos}}^2\left(\zeta \right)-1}{2{\gamma }^2}}\right)\left({\int }_{-\text{cos}\left(\zeta \right)}^{\infty }t{e}^{\frac{t^2}{-2{\gamma }^2}}dt+{\int }_{-\text{cos}\left(\zeta \right)}^{\infty }\text{cos}\left(\zeta \right)e^{\frac{t^2}{-2{\gamma }^2}}dt\right)\\ =\frac{1}{2\pi {\gamma }^2}\left(e^{\frac{{\text{cos}}^2\left(\zeta \right)-1}{2{\gamma }^2}}\right)\left({\gamma }^2{e}^{\frac{-{\text{cos}}^2\left(\zeta \right)}{2{\gamma }^2}}+\text{cos}\left(\zeta \right){\int }_{-\text{cos}\left(\zeta \right)/\gamma }^{\infty }\gamma e^{\frac{t^2}{-2}}dt\right)\\ =\frac{1}{2\pi }\left(e^{\frac{-1}{2{\gamma }^2}}\right)+\frac{1}{\sqrt{2\pi {\gamma }^2}}\left(e^{\frac{{\text{sin}}^2\left(\zeta \right)}{-2{\gamma }^2}}\right)\left(\frac{\text{cos}\left(\zeta \right)}{\sqrt{2\pi }}{\int }_{-\text{cos}\left(\zeta \right)/\gamma }^{\infty }{e}^{\frac{t^2}{-2}dt}\right).\end{array}$$

(27)