Co-channel interference suppression for multi-cell MIMO heterogeneous network

Abstract

The heterogeneous network, contains a macro cell and a grid of low power nodes with the same frequencies, can improve the system capacity and spectrum efficiency. Configuring low-power nodes that share the same spectrum with macro cell to form heterogeneous networks makes it more likely to improve the system capacity and spectrum efficiency, but inevitably, strong co-channel interference is the main barrier to further improvement for heterogeneous networks. This paper proposes an algorithm which combines the triangular decomposition and signal to leakage and noise ratio (SLNR) (TD-SLNR) to suppress strong co-channel interference in multi-cell multiple input and multiple output (MIMO) heterogeneous networks. Firstly, the proposed algorithm can reduce the number of inter-cell interferences in half. As a result of triangular decomposition, an equivalent interference channel model is extracted to eliminate the rest of interferences using SLNR and interference suppression matrix. Theoretical analysis shows that the proposed algorithm provides a potential solution to suppress the co-channel interference with low complexity and reduce the computation complexity without adding extra interference suppression matrices and computation complexity at receivers. Furthermore, the simulation results show that TD-SLNR algorithm can improve system capacity and energy efficiency comparing with the traditional SLNR algorithm.

1 Introduction

In the new generation of broadband mobile communication systems, multiple input and multiple output (MIMO) technique [1, 2] and heterogeneous networks [3–5] are key methods to enable the fast and reliable wireless communication access. On one hand, MIMO provides increased channel capacity and data rate using multiple antennas on both base stations and terminals. On the other hand, heterogeneous networks use hybrid networks with lower power nodes (pico, femto cells) and main nodes (macro cells) to achieve high-frequency efficiency and system capacity. The topology of a typical heterogeneous network consists of one macro cell and several pico/femto cells sharing the same frequency within the one area. Such networks bring co-channel interferences (CCI) which need to be eliminated in wireless communications [6–8]. For instance, users of the pico/femto cells are affected by the strong power signal from the macro cell, whereas high power terminals at the edge of macro and pico/femto cells produce strong interferences to the surrounding pico/femto cells. It is worthwhile to investigate methods to reduce or eliminate the CCI especially after MIMO be introduced to the heterogeneous networks.

At present, interference coordination is the key technology of interference suppression for heterogeneous network. The inter-cell interference in traditional networks can be significantly reduced by adjusting the spectrum allocation and transmission power among cells [9–11]. The inter-cell interference coordination requires a central control node to achieve parameter sending and so on. However, in heterogeneous network, a large of low power nodes are randomly allocated according to the demands of users; thus, the low power nodes cannot use the X2 interface [12], which will cause strong uplink and downlink interference to adjacent cells, such as the downlink dead zone and uplink blocking [8, 13]. Therefore, the novel interference coordination schemes designed for heterogeneous network are proposed [14–16]. For example, in [10], a self-adaptive and flexible algorithm about spectrum utilization is proposed; this algorithm uses flexible spectrum utilization algorithm in cells formed by low power nodes while adopts fixed multiplexing schemes to manage interference in the macro cell. Meanwhile, there are many studies about interference suppression using various power control schemes for heterogeneous network [17, 18]. However, the interference coordination scheme based on the spectrum allocation cannot fully exploit the spectrum, and the interference coordination scheme based on the power control will result in higher computation complexity and a large number of signaling interactions. Furthermore, the application of interference coordination in actual deployment is usually restrained by many factors such as implementation complexity, central management, and distribution control; thus, we will focus on using the inter-cell interference elimination to suppress the interference in the heterogeneous network.

Currently, the interference elimination mainly includes two aspects, namely pre-coding design at transmitting end and the signal detection at receiving end, respectively. As for signal detection, it is usually limited by the achievement of channel state information, and it has high-complexity iterative detection process [19, 20]. As for the pre-coding design, CCI suppression can employ the block diagonalization pre-coding algorithm [21–24]. However, initially, this algorithm requires the number of transmitting antennas should be greater than the total number of all users’ receiving antennas, which is difficult to satisfy. At present, some regularized versions of BD-type algorithms can deal with the limitation of the number of transmitting antennas [25, 26], but they are not referred to the application in multi-cell environment. Furthermore, BD algorithm will not take the problems of noise amplification into consideration; thus, it is unrealistic to put it into practice. Therefore, the signal to leakage and noise ratio (SLNR) algorithm proposed by Sadek is chosen to be the criterion of pre-coding design [27]. This algorithm maximizes the ratio of signal over the sum of leakage and noise, decomposes the multi-user MIMO system into multiple collateral and independent single-user MIMO systems, so that it is not restricted by the number of antennas, and gets the pre-coding matrices of each user independently, so the better interference suppression performance and extensive application scenarios can be obtained [28–31]. Moreover, in order to reduce the inter-cell co-channel interference, the interference alignment technology is adopted in multi-cell MIMO network [32, 33]. Generally, the interference alignment technology designs the interference suppression matrices at receivers and aligns the interference signal to the corresponding zero zones of interference suppression matrices to suppress the inter-cell co-channel interference. Though the interference alignment technology is helpful to suppress the inter-cell and intra-cell interference as well as enhance the system capacity, it will undoubtedly increase the number of interference suppression matrices at receivers and its corresponding computation complexity.

Summarized the previous works, this paper propose an interference suppression algorithm combining triangular decomposition and SLNR to solve the problems of the co-channel interference suppression for multi-cell MIMO heterogeneous network. The contributions and innovations of this paper are summarized as follows:

• This paper firstly discusses the interference situation of the network and then proposes an interference suppression algorithm combining triangular decomposition and SLNR under the considered network. This algorithm can reduce the number of inter-cell interferences in half through using triangular decomposition to the joint channel matrix, before interference suppression process at receiver.

• Based on the equivalent interference model, combining with the SLNR algorithm, this paper computes the pre-coding matrices of each user in each cell and the corresponding closed-form interference suppression matrices in detail according to the different interference situations of each cell.

• Considering the computation complexity, we compare the proposed algorithm with the traditional SLNR as well as the algorithm adopting interference alignment technology.

• We verify that comparing the proposed algorithm with traditional SLNR, a great improvement of system capacity and energy efficiency can be achieved. Furthermore, the impact of different number of data streams and antennas on the system performance is further analyzed.

The remainder of this paper is organized as follows. Section 2 mainly discusses the interference situations in multi-cell MIMO heterogeneous network. Section 3 makes a detailed introduction about the proposed algorithm which combines the triangular decomposition and SLNR. Furthermore, the computation complexity of the proposed algorithm with the traditional SLNR as well as the algorithm adopting interference alignment was compared. The simulation results will be discussed in Section 5. Finally, conclusions are presented in Section 6.

2 The interference situation discussion for multi-cell MIMO heterogeneous network

Multi-cell MIMO heterogeneous network consists of multiple macro cells and micro cells, and usually, the users are uniformly and randomly distributed in the cells. Figure 1 describes a universal downlink interference model in heterogeneous network, where the macro BS represents the macro base station in the network, the MUE represents the user belonging to macro BS, the pico BS indicates the base station in pico cell, and PUS represents the user belonging to pico BS. The circular region circled by dotted line is the coverage area of pico cell.

Fig. 1

Downlink interference model

As shown in Fig. 1, there are two kinds of interferences in the model, including interference from macro BS to PUS and interference from pico BS to MUS. Due to the different path loss from pico BS to MUS, the received interference strength of different MUS is not the same. It is obvious that MUS1 may receive the strongest interference, MUS2 followed but also relatively strong, and MUS3 may receive the weakest interference strength. Furthermore, as the path loss from macro BS to MUS2 and MUS3 is much more than that to MUS1 and if we want to make the received signal to interference plus noise ratio (SINR) of each user the same, it is needed to increase the transmitting power of macro BS, which will result in strong interference from macro BS to PUS.

The uplink interference model in heterogeneous network is shown in Fig. 2. There are also two kinds of interference in the model, including interference from MUS to pico BS and interference from PUS to macro BS, respectively. According to the discussion of downlink, it may be considered that the interference from MUS1 and MUS2 to the uplink of pico BS is stronger enough: meanwhile, it is weaker to that of MUS3. However, the interference situations from MUS to the uplink of pico BS are quite different. If the macro BS requires the same receiving SINR from different MUS, but due to the different path loss of MUS1 and MUS2 to macro BS, the uplink transmission power of MUS2 should be much higher than that of MUS1, thus result in the interference from MUS2 to the uplink of pico BS is much stronger than the interference from MUS1 to the uplink of pico BS. Moreover, though MUS3 is far away from pico BS, the interference from MUS3 to the uplink of pico BS is the strongest. The path loss of MUS3 to macro BS is the largest one; so if macro BS requires the same receiving SINR from MUS3 and MUS1, the transmission power of MUS3 needs to be greatly enhanced; this results in the strongest interference from MUS3 to the uplink of pico BS. Similarly, the edge users of pico cell also need higher transmission power, which may result in interference to macro BS.

Fig. 2

Uplink interference model

In conclusion, the interference situations in multi-cell MIMO heterogeneous network are absolutely different with that in traditional macro cell. According to the above discussion, we know that it is not suitable for the interference suppression of multi-cell MIMO heterogeneous network through judging the interference strength and designing the corresponding interference coordination scheme just according to the users’ locations. Thus, this paper proposes an interference elimination algorithm for edge users, which can fully eliminate the interference according to the various interference situations on terminals.

For the convenience of analysis, the interference model has only one low-power cell. In the following, the system model of a multi-cell MIMO heterogeneous network is investigated, as shown in Fig. 3. Considering that there are K cells, including the low-power nodes such as macro cells and pico cells, they share the same spectrum. Thus, it is obvious that there are strong co-channel interferences exist in the system model, especially for the edge users. For the sake of convenience, it is supposed that there are only two edge users who not only receive the desired signals (the solid line represents in Fig. 3) from native cell but also receive the co-channel interference signals (the dotted line represents in Fig. 3) from other K-1 cells.

Fig. 3

System model

3 Co-channel interference suppression

Based on the system model considered above, supposing that the BS is equipped with N _t antennas, the user is equipped with N _r antennas, so constitutes a (K, N _t ) × (2, N _r ) MIMO interference channel model, as shown in Fig. 4. Due to the purpose of enabling the same signal space dimension that is provided by each transmitter, it can be assumed that each user has the same degrees of freedom d (Dof).

Fig. 4

Interference channel model

Thus, the received signal of ith user in kth cell can be expressed as

$$ {\mathbf{y}}_{i,k}={\left({\mathbf{u}}_{i,k}\right)}^H{\mathbf{H}}_b^{i,k}{\mathbf{w}}_k{\mathbf{x}}_k+{\left({\mathbf{u}}_{i,k}\right)}^H{\displaystyle \sum_{b\ne k,b=1}^K{\mathbf{H}}_b^{i,k}{\mathbf{w}}_b{\mathbf{x}}_b}+{\left({\mathbf{u}}_{i,k}\right)}^H{\mathbf{n}}_{i,k} $$

(1)

where, x _k  = [(x _1,k )^T, (x _2,k )^T]^T indicates the 2d × 1 data streams sent by the BS in kth cell; x _1,k , x _2,k are the d × 1 data streams sent by the BS in kth cell to users 1 and 2, respectively, satisfying the power constraint $E\left[{\mathbf{x}}_k^H{\mathbf{x}}_k\right]=P(k);\kern0.5em {\mathbf{H}}_b^{i,k}$ which represents the N _r  × N _t channel matrix between the BS in the bth cell and the ith user in the kth cell; u _i,k is the intra-cell interference suppression matrix of the ith user in the kth cell, and (u _i,k )^H u _i,k  = I _d ; w _k  = [w _1,k , w _2,k ] is the N _t  × 2d dimension pre-coding matrix; w _1,k and w _2,k are the N _t  × d dimension pre-coding matrices of users 1 and 2 in the kth cell, respectively; n _i,k is the N _r  × 1 dimension additive white Gaussian noise with zero-mean, unit variance, and $E\left[{\mathbf{n}}_{i,k}{\left({\mathbf{n}}_{i,k}\right)}^H\right]={\mathbf{I}}_{N_r}.$

3.1 Triangular decomposition of the equivalent channel

From the point of joint channel matrix of all cells, this paper exploits triangular decomposition to the joint channel matrix. It can be seen from the discussion about the downlink interference in Section 2 that different base stations have different transmission power, which results in different degree of interference on users, and the base stations with higher transmission power have stronger co-channel interference to edge users of other cells. Thus, due to the different extent of interference on different users, it is needed to rank the receiving signal powers of users in different cells then obtain the suitable joint channel matrix by adjusting the row vectors of joint channel matrix.

Here, by computing the norms ${\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2$ of channel matrices between the base stations and the native users, the row vectors of joint channel matrix can be adjusted and ranked according to the values of norms. Besides, ${\mathbf{H}}_b^{i,k}$ can be further expressed as $T{\mathbf{H}}_b^{\hbox{'}i,k}R,$, where T and R are the power gains of transmitter and receiver, respectively. So the ${\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2$ norm of channel matrices between base stations and native users can be calculated [34].

$$ {\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2={\displaystyle \sum_{j=1}^{N_t}{\left\Vert {h}_j\right\Vert}_2^2}={\displaystyle \sum_{j=1}^{N_t}{\lambda}_j\left({\left({\mathbf{H}}_k^{i,k}\right)}^H{\mathbf{H}}_k^{i,k}\right)} $$

(2)

where, λ _j is the jth eigenvalue of ${\left({H}_k^{i,k}\right)}^H{H}_k^{i,k}$ matrix whose rank is N _t . It is observed that the ${\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2$ norm reflects the magnitude of the desired signal power of the ith user in the cell and the situation of transmission power of native base stations as well. Therefore, the row vectors of joint channel matrix can be adjusted and ranked according to ${\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2.$ assuming that the ${\left\Vert {\mathbf{H}}_k^{i,k}\right\Vert}_F^2$ norms of the channel matrix of K cells are ranked as follows:

$$ {\left\Vert {\mathbf{H}}_a^{i,a}\right\Vert}_F^2\ge {\left\Vert {\mathbf{H}}_b^{i,b}\right\Vert}_F^2\ge \cdots \ge {\left\Vert {\mathbf{H}}_K^{i,K}\right\Vert}_F^2 $$

(3)

According to the ranking result, corresponding cells are named as cell 1, cell 2,…, cell K.

In the following, the triangular decomposition of the equivalent channel will be discussed. The received signal of the ith user in the kth cell without interference suppression operation can be expressed as

$$ \begin{array}{l}{\mathbf{y}}_{i,k}={\mathbf{H}}_b^{i,k}{\mathbf{w}}_k{\mathbf{x}}_k+{\displaystyle \sum_{b\ne k,b=1}^K{\mathbf{H}}_b^{i,k}{\mathbf{w}}_b{\mathbf{x}}_b}+{\mathbf{n}}_{i,k}\\ {}\kern1.5em ={\displaystyle \sum_{b=1}^K{\mathbf{H}}_b^{i,k}{\mathbf{w}}_b{\mathbf{x}}_b}+{\mathbf{n}}_{i,k}\end{array} $$

(4)

According to the above comparison result of channel norms, the joint received signals Y of the whole system can be written as [35]

$$ \mathbf{Y}=\left[\begin{array}{c}\hfill {\mathbf{y}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{y}}_2\\ {}\kern0.4em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{y}}_{\mathrm{K}}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,1}\hfill \\ {}\hfill {\mathbf{y}}_{2,1}\hfill \\ {}\hfill {\mathbf{y}}_{1,2}\hfill \\ {}\hfill \begin{array}{l}{\mathbf{y}}_{2,2}\\ {}\kern0.8em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{y}}_{1,\mathrm{K}}\hfill \\ {}\hfill {\mathbf{y}}_{2,\mathrm{K}}\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill {\mathbf{H}}_1^{1,1}\hfill & \hfill {\mathbf{H}}_2^{1,1}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{1,1}\hfill \\ {}\hfill {\mathbf{H}}_1^{2,1}\hfill & \hfill {\mathbf{H}}_2^{2,1}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{2,1}\hfill \\ {}\hfill {\mathbf{H}}_1^{1,2}\hfill & \hfill {\mathbf{H}}_2^{1,2}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{1,2}\hfill \\ {}\hfill {\mathbf{H}}_1^{2,2}\hfill & \hfill {\mathbf{H}}_2^{2,2}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{2,2}\hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \cdots \hfill & \hfill \vdots \hfill \\ {}\hfill {\mathbf{H}}_1^{1,\mathrm{K}}\hfill & \hfill {\mathbf{H}}_2^{1,\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{1,\mathrm{K}}\hfill \\ {}\hfill {\mathbf{H}}_1^{2,\mathrm{K}}\hfill & \hfill {\mathbf{H}}_2^{2,\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}}^{2,\mathrm{K}}\hfill \end{array}\right]\left[\begin{array}{c}\hfill {\mathbf{w}}_1{\mathbf{x}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{w}}_2{\mathbf{x}}_2\\ {}\kern1.1em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{w}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{n}}_{1,1}\hfill \\ {}\hfill {\mathbf{n}}_{2,1}\hfill \\ {}\hfill {\mathbf{n}}_{1,2}\hfill \\ {}\hfill \begin{array}{l}{\mathbf{n}}_{2,2}\\ {}\kern0.8em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{n}}_{1,\mathrm{K}}\hfill \\ {}\hfill {\mathbf{n}}_{2,\mathrm{K}}\hfill \end{array}\right]=\mathbf{H}\mathbf{X}+\mathbf{n} $$

(5)

where, $\mathbf{X}=\left[\begin{array}{c}\hfill {\mathbf{w}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\mathbf{w}}_2{\mathbf{x}}_2\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill {\mathbf{w}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right],{\mathbf{y}}_k=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,k}\hfill \\ {}\hfill {\mathbf{y}}_{2,k}\hfill \end{array}\right],{\mathbf{n}}_k=\left[\begin{array}{c}\hfill {\mathbf{n}}_{1,k}\hfill \\ {}\hfill {\mathbf{n}}_{2,k}\hfill \end{array}\right],k=1,2,\dots,\ K.$

Employing ${\mathbf{H}}_{bj}=\left[\begin{array}{c}\hfill {\mathbf{H}}_b^{1,j}\hfill \\ {}\hfill {\mathbf{H}}_b^{2,j}\hfill \end{array}\right]$ to merge the channel matrices, H _bj is the joint channel matrix between two users in the jth cell and the base station in the bth cell; its dimension is 2N _r  × N _t .. Thus, (5) can be rewritten as

$$ \begin{array}{l}\mathbf{Y}=\left[\begin{array}{c}\hfill {\mathbf{y}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{y}}_2\\ {}\kern0.5em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{y}}_{\mathrm{K}}\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill {\mathbf{H}}_{11}\hfill & \hfill {\mathbf{H}}_{21}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}1}\hfill \\ {}\hfill {\mathbf{H}}_{12}\hfill & \hfill {\mathbf{H}}_{22}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}2}\hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \cdots \hfill & \hfill \vdots \hfill \\ {}\hfill {\mathbf{H}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{H}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]\kern0.1em \left[\begin{array}{c}\hfill {\mathbf{w}}_1{\mathbf{x}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{w}}_2{\mathbf{x}}_2\\ {}\kern1.1em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{w}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{n}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{n}}_2\\ {}\kern0.5em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{n}}_{\mathrm{K}}\hfill \end{array}\right]\\ {}\kern1.1em =\mathbf{H}\mathbf{X}+\mathbf{n}\end{array} $$

(6)

where, $\mathbf{H}=\left[\begin{array}{cccc}\hfill {\mathbf{H}}_{11}\hfill & \hfill {\mathbf{H}}_{21}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}1}\hfill \\ {}\hfill {\mathbf{H}}_{12}\hfill & \hfill {\mathbf{H}}_{22}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}2}\hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \cdots \hfill & \hfill \vdots \hfill \\ {}\hfill {\mathbf{H}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{H}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]$ is the joint channel matrix of the whole system and its dimensions is 2KN _r  × KN _t .

Then the joint channel matrix H is adjusted by the following triangular decomposition

$$ \mathbf{H}=\left[\begin{array}{cccc}\hfill {\mathbf{H}}_{11}\hfill & \hfill {\mathbf{H}}_{21}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}1}\hfill \\ {}\hfill {\mathbf{H}}_{12}\hfill & \hfill {\mathbf{H}}_{22}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}2}\hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \cdots \hfill & \hfill \vdots \hfill \\ {}\hfill {\mathbf{H}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{H}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{H}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill {\mathbf{R}}_{11}\hfill & \hfill \hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{12}\hfill & \hfill {\mathbf{R}}_{22}\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{R}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{R}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]\mathbf{U}=\mathbf{R}\mathbf{U} $$

(7)

where, ${\mathbf{R}}_{bj}=\left[\begin{array}{c}\hfill {\mathbf{R}}_b^{1,j}\hfill \\ {}\hfill {\mathbf{R}}_b^{2,j}\hfill \end{array}\right]$ is the 2N _r  × N _t dimension matrix, and the dimension of ${\mathbf{R}}_b^{1,j}$ and ${\mathbf{R}}_b^{2,j}$ is N _r  × N _t , where R is the lower triangular matrix and its dimension is 2KN _r  × KN _t ,where U is unitary matrix and its dimension is KN _t  × KN _t . Thus, (6) can be equal to

$$ \mathbf{Y}=\mathbf{H}\mathbf{X}+\mathbf{n}=\mathbf{R}\mathbf{U}\mathbf{X}+\mathbf{n}=\left[\begin{array}{cccc}\hfill {\mathbf{R}}_{11}\hfill & \hfill \hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{12}\hfill & \hfill {\mathbf{R}}_{22}\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{R}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{R}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]\mathbf{U}\left[\begin{array}{c}\hfill {\mathbf{w}}_1{\mathbf{x}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{w}}_2{\mathbf{x}}_2\\ {}\kern1.1em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{w}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right]+\mathbf{n} $$

(8)

$$ \mathrm{Define}\kern0.5em \left[\begin{array}{c}\hfill {\mathbf{V}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\mathbf{V}}_2{\mathbf{x}}_2\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill {\mathbf{V}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right]=\mathbf{U}\left[\begin{array}{c}\hfill {\mathbf{w}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\mathbf{w}}_2{\mathbf{x}}_2\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill {\mathbf{w}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right] $$

(9)

Thus, equation (8) can be rewritten as

$$ \mathbf{Y}=\left[\begin{array}{c}\hfill {\mathbf{y}}_1\hfill \\ {}\hfill {\mathbf{y}}_2\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill {\mathbf{y}}_{\mathrm{K}}\hfill \end{array}\right]=\left[\begin{array}{cccc}\hfill {\mathbf{R}}_{11}\hfill & \hfill \hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{12}\hfill & \hfill {\mathbf{R}}_{22}\hfill & \hfill \hfill & \hfill \hfill \\ {}\hfill \vdots \hfill & \hfill \vdots \hfill & \hfill \ddots \hfill & \hfill \hfill \\ {}\hfill {\mathbf{R}}_{1\mathrm{K}}\hfill & \hfill {\mathbf{R}}_{2\mathrm{K}}\hfill & \hfill \cdots \hfill & \hfill {\mathbf{R}}_{\mathrm{K}\mathrm{K}}\hfill \end{array}\right]\kern0.1em \left[\begin{array}{c}\hfill {\mathbf{V}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\mathbf{V}}_2{\mathbf{x}}_2\hfill \\ {}\hfill \vdots \hfill \\ {}\hfill {\mathbf{V}}_{\mathrm{K}}{\mathbf{x}}_{\mathrm{K}}\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{n}}_1\hfill \\ {}\hfill \begin{array}{l}{\mathbf{n}}_2\\ {}\kern0.5em \vdots \end{array}\hfill \\ {}\hfill {\mathbf{n}}_{\mathrm{K}}\hfill \end{array}\right] $$

(10)

As can be known from (10), the received signals without the complex interference suppression processing can be equivalent to

$$ {\mathbf{y}}_1=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,1}\hfill \\ {}\hfill {\mathbf{y}}_{2,1}\hfill \end{array}\right]={\mathbf{R}}_{11}{\mathbf{V}}_1{\mathbf{x}}_1+{\mathbf{n}}_1 $$

(11)

$$ {\mathbf{y}}_2=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,2}\hfill \\ {}\hfill {\mathbf{y}}_{2,2}\hfill \end{array}\right]={\mathbf{R}}_{12}{\mathbf{V}}_1{\mathbf{x}}_1+{\mathbf{R}}_{22}{\mathbf{V}}_2{\mathbf{x}}_2+{\mathbf{n}}_2 $$

(12)

$$ {\mathbf{y}}_k=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,k}\hfill \\ {}\hfill {\mathbf{y}}_{2,k}\hfill \end{array}\right]={\mathbf{R}}_{1k}{\mathbf{V}}_1{\mathbf{x}}_1+{\mathbf{R}}_{2k}{\mathbf{V}}_2{\mathbf{x}}_2+\cdots +{\mathbf{R}}_{kk}{\mathbf{V}}_k{\mathbf{x}}_k+{\mathbf{n}}_k $$

(13)

where, ${\mathbf{R}}_{bj}{\mathbf{V}}_b{\mathbf{x}}_b=\left[\begin{array}{c}\hfill {\mathbf{R}}_b^{1,j}\hfill \\ {}\hfill {\mathbf{R}}_b^{2,j}\hfill \end{array}\right]{\mathbf{V}}_b{\mathbf{x}}_b=\left[\begin{array}{c}\hfill {\mathbf{R}}_b^{1,j}{\mathbf{V}}_b{\mathbf{x}}_b\hfill \\ {}\hfill {\mathbf{R}}_b^{2,j}{\mathbf{V}}_b{\mathbf{x}}_b\hfill \end{array}\right],$ b, j = 1, 2, …, K. When b = j, R _bj is the intra-cell equivalent channel matrix; When b ≠ j, R _bj is the inter-cell equivalent channel matrix. ${\mathbf{R}}_b^{1,j}$ and ${\mathbf{R}}_b^{2,j}$, whose dimension is N _r  × N _t , represents the equivalent channel matrices between base stations of bth cell and user 1, user 2 in the jth cell, respectively. V _1,k and V _2,k represent the equivalent pre-coding matrices of user 1 and user 2 in the kth cell, and the dimension is N _t  × d, V _k  = [V _1,k , V _2,k ], is N _t  × 2d dimension matrix, k = 1,2,…, K.

Through the above analysis, the interference channel model described in Fig. 4 can be equivalent to that in Fig. 5.

Fig. 5

Equivalent interference channel model

As can be known from (11), after using the triangular decomposition for joint channel matrix, the users of cell 1 in equivalent model do not receive the interference from other cells (inter-cell interference) but only receive the intra-cell interference. As can be known from (12), the users of cell 2 only receive the interference from cell 1 and intra-cell interference. As can be known from (13), the users in the kth cell receive the interference from cells 1, 2,…, k-1 and the intra-cell interference. Therefore, the number of inter-cell interferences in half is reduced through using triangular decomposition for joint channel matrix, without other complex interference suppressing operation, which reduces the computation complexity at receiver.

In the following, considering to employ SLNR to design the pre-coding matrix V _i,k with the goal of eliminating the rest of inter-cell interference is to consider the meaning of the equivalent interference channel model in another way of thinking. Combined with (11), (12), and (13), it can be found that part of power that originally should be sent to native users in cell 1 was leaked to the users in cells 2, 3,…, K; thus, the SLNR is used to design pre-coding matrix V _i,1 to eliminate the interferences in each of the cells caused by cell 1, then the interference suppression matrix u _i,1 is used to eliminate the intra-cell interference of cell 1. Similarly, the base station in kth cell leaks part of power which originally should be sent to native users to the users of cell k + 1, k + 2,…, K; thus, the SLNR is used to design pre-coding matrix V _i,k to eliminate the interferences in each of the cells caused by cell k then also use interference suppression matrix u _i,k to eliminate intra-cell interference of cell k. Meanwhile, as shown in equivalent model, the base stations in cell K do not leak power to users in other cells: thus, the equivalent pre-coding matrix V _i,K can be used to suppress the intra-cell interference of cell K, so that the receiver in cell K does not need to increase intra-cell interference suppression matrix. It can be seen that the ultimate interference channel model can be simplified as Fig. 6 after using triangular decomposition and SLNR.

Fig. 6

Triangular decomposition SLNR interference channel model

In conclusion, the algorithm reduce the number of inter-cell interferences in half using triangular decomposition, then the rest of interference is eliminated by using SLNR to design pre-coding matrix, so that the receiver does not need to design individual interference suppression matrix to eliminate inter-cell interference, which reduces the number of interference suppression matrices and simplifies the computation complexity at receiver. Besides, as can be known from (9) and matrix theory that dimension matching matrices satisfy span(w _k x _k ) = span(V _k x _k ), that is to say, they can be spanned to the same space, which indicates that the triangular decomposition will not impact the degree of freedom, so the obtaining for equivalent model will not change the degree of freedom either. There is a need to explain that the proposed algorithm is implemented at base stations; meanwhile, it is assumed that all stations in this system can get channel matrix information through central control or circular polling mechanism, etc.

3.2 Interference suppressing algorithm combining triangular decomposition and SLNR

According to the above analysis, it can be found that the number of inter-cell interferences in half can be eliminated by employing triangular decomposition. In order to further decrease the number of the matrices at the receivers and the corresponding computation complexity, simplify the receivers processing; SLNR algorithm is employed to further suppress the inter-cell interferences. Next, for the convenience of analysis, we will take three cells as example to analyze the process of suppressing the interference for each cell in detail.

3.2.1 Interference suppression of cell 1

The users of cell 1 utilize the interference suppression matrix u _i,1 to eliminate the intra-cell interference [35]. The received signal of the users in cell1 can be expressed as

$$ {\mathbf{r}}_1=\left[\begin{array}{c}\hfill {\mathbf{r}}_{1,1}\hfill \\ {}\hfill {\mathbf{r}}_{2,1}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{y}}_{1,1}\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{y}}_{2,1}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{R}}_1^{1,1}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{R}}_1^{2,1}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{n}}_{1,1}\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{n}}_{2,1}\hfill \end{array}\right] $$

(14)

Furthermore, the signal received by the ith user can be expressed as

$$ \begin{array}{l}{\mathbf{r}}_{i,1}={\left({\mathbf{u}}_{i,1}\right)}^H{\mathbf{R}}_1^{i,1}{\mathbf{V}}_1{\mathbf{x}}_{i,1}+{\left({\mathbf{u}}_{i,1}\right)}^H\mathbf{n}\\ {}\kern1.1em ={\left({\mathbf{u}}_{i,1}\right)}^H{\mathbf{R}}_1^{i,1}\left({\mathbf{V}}_{1,1}\mathbf{X}{}_{1,1}+{\mathbf{V}}_{2,1}\mathbf{X}{}_{2,1}\right)+{\left({\mathbf{u}}_{i,1}\right)}^H\mathbf{n}\end{array} $$

(15)

The user 1 needs to align the interference signal to the corresponding null space of the interference suppression matrix u _1,1; thus, it should satisfy the following constraint:

$$ \left\{\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{R}}_1^{1,1}{\mathbf{V}}_{2,1}=0\hfill \\ {}\hfill \mathrm{rank}\left[{\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{R}}_1^{1,1}{\mathbf{V}}_{1,1}\right]=d\hfill \end{array}\right. $$

(16)

Therefore, u _1,1 is the eigenvector corresponding to the minimum eigenvalue

$$ {\mathbf{u}}_{1,1}={v}_{\min}^d\left(\left({\mathbf{R}}_1^{1,1}{\mathbf{V}}_{2,1}\right){\left({\mathbf{R}}_1^{1,1}{\mathbf{V}}_{2,1}\right)}^H\right) $$

(17)

Similarly, the interference suppression matrix of the user 2 in cell 1 should satisfy the following constraints:

$$ \left\{\begin{array}{c}\hfill {\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{R}}_1^{2,1}{\mathbf{V}}_{1,1}=0\hfill \\ {}\hfill \mathrm{rank}\left[{\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{R}}_1^{2,1}{\mathbf{V}}_{2,1}\right]=d\hfill \end{array}\right. $$

(18)

Therefore, the interference suppression matrix is

$$ {\mathbf{u}}_{2,1}={v}_{\min}^d\left(\left({\mathbf{R}}_1^{2,1}{\mathbf{V}}_{1,1}\right){\left({\mathbf{R}}_1^{2,1}{\mathbf{V}}_{1,1}\right)}^H\right) $$

(19)

Considering formulas (11), (12), and (13), the users in cell 2 and cell 3 will receive the inter-cell interference from cell 1, combining the SLNR algorithm [36], that is to say, the user 1 in cell 1 not only leaks part of power to user 2 in cell 1 but also part of power to the users in cell 2 and cell 3. So the signal to leakage and noise ratio of use 1 in cell 1 is

$$ \mathrm{SLNR}=\frac{{\left\Vert {\left({\mathbf{u}}_{1,1}\right)}^H{\mathbf{R}}_1^{1,1}{\mathbf{V}}_{1,1}\right\Vert}_F^2}{{\left\Vert {\left({\mathbf{u}}_{2,1}\right)}^H{\mathbf{R}}_1^{2,1}{\mathbf{V}}_{1,1}\right\Vert}_F^2+{\displaystyle \sum_{i=1}^2{\left\Vert {\left({\mathbf{u}}_{i,2}\right)}^H{\mathbf{R}}_1^{i,2}{\mathbf{V}}_{1,1}\right\Vert}_F^2}+{\displaystyle \sum_{i=1}^2{\left\Vert {\mathbf{R}}_1^{i,3}{\mathbf{V}}_{1,1}\right\Vert}_F^2}+{N}_r{\sigma}_{1,1}^2} $$

(20)

Since the power of transmitting data is normalized and (u _i,k )^H u _i,k  = I _k ,, thus

$$ SLNR=\frac{{\left\Vert {\mathbf{R}}_1^{1,1}{\mathbf{V}}_{1,1}\right\Vert}_F^2}{{\left\Vert {\mathbf{R}}_1^{2,1}{\mathbf{V}}_{1,1}\right\Vert}_F^2+{\displaystyle \sum_{i=1}^2{\displaystyle \sum_{k=2}^3{\left\Vert {\mathbf{R}}_1^{i,k}{\mathbf{V}}_{1,1}\right\Vert}_F^2}}+{N}_r{\sigma}_{1,1}^2} $$

(21)

Under the certain power constraint, the standard of SLNR algorithm that determines the pre-coding vector is to maximize the SLNR, which is to find the equivalent pre-coding matrix V _1,1 satisfying the following two equations:

$$ \left\{\begin{array}{c}\hfill {\mathbf{V}}_{1,1}= \arg \kern0.2em \max \kern0.2em \mathrm{SLNR}\hfill \\ {}\hfill {\left\Vert {\mathbf{V}}_{1,1}\right\Vert}_F^2=d\hfill \end{array}\right. $$

(22)

Rewritten the (21) as

$$ \mathrm{SLNR}=\frac{{\left\Vert {\mathbf{R}}_1^{1,1}{\mathbf{V}}_{1,1}\right\Vert}^2}{{\left\Vert {\tilde{\mathbf{R}}}_1^{i,k}{\mathbf{V}}_{1,1}\right\Vert}^2+{N}_r{\sigma}_{1,1}^2} $$

(23)

According to the conclusions of generalized Rayleigh quotient, the pre-coding vectors are mutually orthogonal, the V _1,1 is the eigenvectors corresponding to the d largest generalized eigenvalues of the matrix $\left\{{\left({\mathbf{R}}_1^{1,1}\right)}^H{\mathbf{R}}_1^{1,1},{\left({\tilde{\mathbf{R}}}_1^{i,k}\right)}^H{\tilde{\mathbf{R}}}_1^{i,k}+{N}_r{\sigma}_{1,1}^2I\right\},$ which enable maximize the SLNR, that is

$$ {\mathbf{V}}_{1,1}\propto \max \kern0.2em .\mathrm{eigenvector}\left\{{\left({\mathbf{R}}_1^{1,1}\right)}^H{\mathbf{R}}_1^{1,1},{\left({\tilde{\mathbf{R}}}_1^{i,k}\right)}^H{\tilde{\mathbf{R}}}_1^{i,k}+{N}_r{\sigma}_{1,1}^2I\right\} $$

(24)

Similarly, the pre-coding matrix V _2,1 of the user 2 in cell 1 should satisfy

$$ {\mathbf{V}}_{2,1}\propto \max \kern0.2em .\mathrm{eigenvector}\left({\left({\mathbf{R}}_1^{2,1}\right)}^H{\mathbf{R}}_1^{2,1},{\left({\tilde{\mathbf{R}}}_1^{i,k}\right)}^H{\tilde{\mathbf{R}}}_1^{i,k}+{N}_r{\sigma}_{2,1}^2\mathbf{I}\right) $$

(25)

3.2.2 Interference suppression of cell 2

The received signal of the users in cell 2 can be expressed as

$$ \begin{array}{l}{\mathbf{r}}_2=\left[\begin{array}{c}\hfill {\mathbf{r}}_{1,2}\hfill \\ {}\hfill {\mathbf{r}}_{2,2}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{y}}_{1,2}\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{y}}_{2,2}\hfill \end{array}\right]\\ {}\kern1em =\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{R}}_1^{1,2}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{R}}_1^{2,2}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{R}}_2^{1,2}{\mathbf{V}}_2{\mathbf{x}}_2\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{R}}_2^{2,2}{\mathbf{V}}_2{\mathbf{x}}_2\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{n}}_{1,2}\hfill \\ {}\hfill {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{n}}_{2,2}\hfill \end{array}\right]\end{array} $$

(26)

The acquisition of the intra-cell interference suppression matrix of the users in cell 2 is similar to that of users in cell 1; the constraints can be expressed as

$$ \left\{\begin{array}{c}\hfill {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{R}}_2^{1,2}{\mathbf{V}}_{2,2}=0\hfill \\ {}\hfill \mathrm{rank}\left[{\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{R}}_2^{1,2}{\mathbf{V}}_{1,2}\right]=d\hfill \end{array}\right. $$

(27)

$$ \left\{\begin{array}{c}\hfill {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{R}}_2^{2,2}{\mathbf{V}}_{1,2}=0\hfill \\ {}\hfill \mathrm{rank}\left[{\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{R}}_2^{2,2}{\mathbf{V}}_{2,2}\right]=d\hfill \end{array}\right. $$

(28)

Thus, interference suppression matrix can be expressed as

$$ {\mathbf{u}}_{1,2}={v}_{\min}^d\left(\left({\mathbf{R}}_2^{1,2}{\mathbf{V}}_{2,2}\right){\left({\mathbf{R}}_2^{1,2}{\mathbf{V}}_{2,2}\right)}^H\right) $$

(29)

$$ {\mathbf{u}}_{2,2}={v}_{\min}^d\left(\left({\mathbf{R}}_2^{2,2}{\mathbf{V}}_{1,2}\right){\left({\mathbf{R}}_2^{2,2}{\mathbf{V}}_{1,2}\right)}^H\right) $$

(30)

Similarly, it can be seen from formula (13) that the user 1 in cell 2 leaks the part of power to the user 2 in cell 2 as well as the users in cell 3. Combined with the SLNR algorithm, the signal to leakage and noise ratio can be expressed as

$$ \mathrm{S}\mathrm{L}\mathrm{N}\mathrm{R}=\frac{{\parallel {\left({\mathbf{u}}_{1,2}\right)}^H{\mathbf{R}}_2^{1,2}{\mathbf{V}}_{1,2}\parallel}_F^2}{\parallel {\left({\mathbf{u}}_{2,2}\right)}^H{\mathbf{R}}_2^{2,2}{\mathbf{V}}_{1,2}{\parallel}_F^2+{\displaystyle \sum_{i=1}^2{\parallel {\mathbf{R}}_2^{i,3}{\mathbf{V}}_{1,2}\parallel}_F^2}+{N}_r{\sigma}_{1,2}^2} $$

(31)

Normalizing the power of transmitting data, and since (u _i,k )^H u _i,k  = I _k , thus

$$ \mathrm{SLNR}=\frac{{\left\Vert {\mathbf{R}}_2^{1,2}{\mathbf{V}}_{1,2}\right\Vert}_F^2}{{\left\Vert {\mathbf{R}}_2^{2,2}{\mathbf{V}}_{1,2}\right\Vert}_F^2+{\displaystyle \sum_{i=1}^2{\left\Vert {\mathbf{R}}_2^{i,3}{\mathbf{V}}_{1,2}\right\Vert}_F^2}+{N}_r{\sigma}_{1,2}^2} $$

(32)

The confirmation for the equivalent pre-coding matrix V _1,2 obtained from SLNR algorithm should satisfy the following two formulas

$$ \left\{\begin{array}{c}\hfill {\mathbf{V}}_{1,2}= \arg \kern0.2em \max \kern0.2em \mathrm{SLNR}\hfill \\ {}\hfill {\left\Vert {\mathbf{V}}_{1,2}\right\Vert}_F^2=d\hfill \end{array}\right. $$

(33)

Rewritten the (32) as

$$ \mathrm{SLNR}=\frac{{\left\Vert {\mathbf{R}}_2^{1,2}{\mathbf{V}}_{1,2}\right\Vert}^2}{{\left\Vert {\tilde{\mathbf{R}}}_2^{i,k}{\mathbf{V}}_{1,2}\right\Vert}^2+{N}_r{\sigma}_{1,2}^2} $$

(34)

V _1,2is the eigenvectors corresponding to the d largest generalized eigenvalues of the matrix $\left\{{\left({\mathbf{R}}_2^{1,2}\right)}^H{\mathbf{R}}_2^{1,2},{\left({\tilde{\mathbf{R}}}_2^{i,k}\right)}^H{\tilde{\mathbf{R}}}_2^{i,k}+{N}_r{\sigma}_{1,2}^2\mathbf{I}\right\},$ that will be able to maximize the SLNR, namely

$$ {\mathbf{V}}_{1,2}\propto \max \kern0.2em .\mathrm{eigenvector}\left({\left({\mathbf{R}}_2^{1,2}\right)}^H{\mathbf{R}}_2^{1,2},{\left({\tilde{\mathbf{R}}}_2^{i,k}\right)}^H{\tilde{\mathbf{R}}}_2^{i,k}+{N}_r{\sigma}_{1,2}^2\mathbf{I}\right) $$

(35)

In a similar way, the pre-coding matrix V _2,2 of the user 2 in cell 2 should satisfy

$$ {\mathbf{V}}_{2,2}\propto \max \kern0.2em .\mathrm{eigenvector}\left({\left({\mathbf{R}}_2^{2,2}\right)}^H{\mathbf{R}}_2^{2,2},{\left({\tilde{\mathbf{R}}}_2^{i,k}\right)}^H{\tilde{\mathbf{R}}}_2^{i,k}+{N}_r{\sigma}_{2,2}^2\mathbf{I}\right) $$

(36)

3.2.3 Interference suppression of cell 3

The received signal of the users in cell 3 can be expressed as

$$ {\mathbf{r}}_3=\left[\begin{array}{c}\hfill {\mathbf{r}}_{1,3}\hfill \\ {}\hfill {\mathbf{r}}_{2,3}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\mathbf{y}}_{1,3}\hfill \\ {}\hfill {\mathbf{y}}_{2,3}\hfill \end{array}\right]=\left[\begin{array}{c}\hfill {\mathbf{R}}_1^{1,3}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \\ {}\hfill {\mathbf{R}}_1^{2,3}{\mathbf{V}}_1{\mathbf{x}}_1\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{R}}_2^{1,3}{\mathbf{V}}_2{\mathbf{x}}_2\hfill \\ {}\hfill {\mathbf{R}}_2^{2,3}{\mathbf{V}}_2{\mathbf{x}}_2\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{R}}_3^{1,3}{\mathbf{V}}_3{\mathbf{x}}_3\hfill \\ {}\hfill {\mathbf{R}}_3^{2,3}{\mathbf{V}}_3{\mathbf{x}}_3\hfill \end{array}\right]+\left[\begin{array}{c}\hfill {\mathbf{n}}_{1,3}\hfill \\ {}\hfill {\mathbf{n}}_{2,3}\hfill \end{array}\right] $$

(37)

As can be seen from formulas (11), (12), and (13), after triangular decomposition, the power that is sent by the BS in cell 3 does not leak to other cells, namely, result in no interferences to other cells. Therefore, the pre-coding matrices of each user in cell 3 are used to eliminate the intra-cell interferences, so the pre-coding matrix should satisfy the following constraints

$$ \left\{\begin{array}{c}\hfill {\mathbf{R}}_3^{1,3}{\mathbf{V}}_{2,3}=0\hfill \\ {}\hfill {\mathbf{R}}_3^{2,3}{\mathbf{V}}_{1,3}=0\hfill \end{array}\right. $$

(38)

Moreover, the pre-coding matrix of the users 1, 2 in cell 3 can be expressed as

$$ \left\{\begin{array}{c}\hfill {\mathbf{V}}_{1,3}={v}_{\min}^d\left({\left({\mathbf{R}}_3^{2,3}\right)}^H{\mathbf{R}}_3^{2,3}\right)\hfill \\ {}\hfill {\mathbf{V}}_{2,3}={v}_{\min}^d\left({\left({\mathbf{R}}_3^{1,3}\right)}^H{\mathbf{R}}_3^{1,3}\right)\hfill \end{array}\right. $$

(39)

Obviously, the receivers in cell 3 do not need intra-cell interference suppression matrix to eliminate the intra-cell interferences. Finally, the final pre-coding matrix w _i,k can be obtained through each equivalent pre-coding matrix V _i,k .

4 Algorithm complexity analysis

4.1 Summarize the algorithm

The proposed algorithm that combines the triangular decomposition and SLNR can be implemented by three steps:

Step1. Employ the triangular decomposition for joint channel matrix H and then extract the equivalent interference channel model;

Step2. According to the equivalent interference channel model, exploit SLNR to compute the equivalent pre-coding matrices V _1,1, V _2,1, V _1,2, and V _2,2 to suppress the rest of inter-cell interference and then obtain w _1,1, w _2,1, w _1,2 and w _2,2;

Step3. Compute the intra-cell interference suppression matrices of users u _1,1, u _2,1, u _1,2 and u _2,2, respectively. Specially, the users in cell 3 use the equivalent pre-coding matrices V _1,3 and V _2,3 to eliminate the intra-cell interference, so the receiver does not need to add intra-cell interference suppression matrix any more.

4.2 Complexity analysis

Furthermore, the complexity of the triangular decomposition and SLNR algorithm is analyzed.

A and B are matrices whose dimensions are m × n, n × k, respectively; the complexity of operation A × B is ο(mnk). Thus, the complexity of operation ${\left({\mathbf{H}}_b^{i,k}\right)}^H{\mathbf{H}}_b^{i,k}$ is ο(N _t ² N _r ), where, ${\mathbf{H}}_b^{i,k}$ is the channel matrix whose dimension is N _r  × N _t , and the complexity of eigenvalues operation for N _t  × N _t dimension matrix C is ο(N _t ³). Therefore, the complexity of the algorithm that combines the triangular decomposition and SLNR can be concluded as follows:

Step1. Employ the triangular decomposition for H, the according complexity is $o\left(2\times {6}^2{N}_r^2\left(3{N}_t-6{N}_r/3\right)\right),$ namely, $o\left(3{N}_r^2{N}_t-2{N}_r^3\right);$

Step2. Exploit SLNR algorithm to calculate the equivalent pre-coding matrices V _1,1, V _2,1, V _1,2, and V _2,2 to suppress the rest of inter-cell interference, the according complexity is $o\left({N}_t^3\right)+o\left({N}_t^2{N}_r\right);$

Step3. Compute the intra-cell interference suppression matrices of users u _1,1, u _2,1, u _1,2 and u _2,2, respectively. Specially, the users in cell 3 use the equivalent pre-coding matrices V _1,3 and V _2,3 to eliminate the intra-cell interference; the total complexity is $o\left({N}_r^3\right)+o\left({N}_t^3\right)+o\left(d{N}_r^2\right)+o\left({N}_t^2{N}_r\right)+o\left(d{N}_r{N}_t\right).$

Therefore, according to above analysis, the total complexity of the proposed algorithm is $o\left({N}_r^3\right)+o\left({N}_t^3\right)+o\left({N}_r^2{N}_t\right)+o\left({N}_t^2{N}_r\right)+o\left(d{N}_r^2\right)+o\left(d{N}_r{N}_t\right).$

However, exploit the interference alignment algorithm to suppress the rest of inter-cell interference, corresponding complexity is $o\left({N}_r^2{N}_t\right)+o\left({N}_r^3\right)+o\left({d}^3{N}_r{N}_t^2\right)+o\left({d}^3{N}_t^3\right)$ [35]. Meanwhile, the complexity of the traditional SLNR without using triangular decomposition to the multi-cell MIMO interference system is $o\left({N}_r^3\right)+o\left({N}_t^3\right)+o\left(d{N}_r^2\right)+o\left(d{N}_r{N}_t\right)+o\left({N}_t^2{N}_r\right).$ It obviously seen that under the condition with certain configuration of antennas, the complexity of the proposed algorithm and the traditional SLNR are on the same order of magnitude, which means that the proposed algorithm without increasing complexity to the system compared with traditional SLNR algorithm. Furthermore, comparing with employing the interference alignment algorithm to compute the inter-cell interference suppression matrix at receiver [35], the proposed algorithm reduces the number of filters (matrices) at receiver, which is helpful to avoid more complex interference suppression process and have lower computation complexity as well as reduce costs of receiver.

5 Numerical results

In the downlink communication environment with co-channel interference of multi-cell MIMO heterogeneous network, the performance of the whole system will be affected by the antenna configurations both of transmitter and receiver; it will also be affected by the data streams sent by the transmitter. At present, the system capacity is usually used as one of the reference indexes to evaluate the system performance, and the energy consumption is a noteworthy problem as well. Thus, the system capacity and energy efficiency are chosen to be the indexes to evaluate the system performance. The system capacity is defined as C = log(1 + SINR _k ) (bps/Hz), and the energy efficiency is defined as the number of bits sent by in unit of energy and unit of bandwidth, namely, $\eta =\frac{ \log \left(1+{\mathrm{SINR}}_k\right)}{P_k}$ (bit/Hz/J), SINR _k represents the signal to interference plus noise ratio of kth user and P _k represents the transmission power of kth user. Aiming at network environment is formed by three cells sharing the same frequency resource, in which each cell has two edge users. Assuming that all channels are flat Rayleigh fading channels with the elements that are independent identically distributed Gaussian random variables whose mean is 0, variance is 1.

Figure 7a, b shows the comparing results of system capacity and energy efficiency when the antenna configurations are 5 × 4, 10 × 8, and 15 × 12, respectively, and the numbers of received data streams (the degree of freedom) of each user are d = 1, 2, 3, the degree of system freedom are Dof = 6, 12, 18. As can be seen from Fig. 7, the system capacity and energy efficiency are obviously promoted with the increase of numbers of antenna as well as data streams. The reason for this phenomenon is the system obtains more diversity gains and the transmitter sends multiple collateral data streams simultaneously, which is very helpful to improve the system performance.

Fig. 7

System performance analysis. a Comparison of system capacity. b Comparison of energy efficiency

Figure 8a, b shows the comparing results of system capacity and energy efficiency in the same simulation environment but d = 1, Dof = 6, and the antenna configurations are different. As can be known from Fig. 8, increasing the number of transmitting and receiving antenna can improve the system capacity and energy efficiency under the fixed Dof, especially the improvement of energy efficiency is more obvious. However, the further improvement of system performance is relatively slow with the increase of number of antenna. Thus, the antenna should be reasonably allocated according to the actual situation.

Fig. 8

Comparison of system performance with the same degree of freedom but different number of antennas. a Comparison of system capacity. b Comparison of energy efficiency

Figure 9a, b shows the comparing results of system capacity and energy efficiency with the same antenna configuration 10 × 8 and simulation environment, but different number of received data streams (the degree of freedom) d = 1,2,3, Dof = 6,12,18. As can be known from Fig. 9, the increasing of number of transmission data streams can improve the system capacity and energy efficiency obviously with fixed antenna configuration, and the improvement of energy efficiency is more obvious than system capacity as well. Therefore, it is needed to send multiple data simultaneously to improve system performance under the reasonable condition.

Fig. 9

Comparison of system performance with the same number of antennas but different number of degree of freedom. a Comparison of system capacity. b Comparison of energy efficiency

In the same simulation environment, Fig. 10a, b shows the results of system capacity and energy efficiency comparing the proposed triangular decomposition SLNR algorithm with traditional SLNR. As can be seen from Fig. 10, the proposed algorithm can greatly improve the system capacity comparing with the traditional SLNR under the same numbers of antennas and data streams. Meanwhile, it can be found from Fig. 10 that though the proposed algorithm does not have significant advantages in improving energy efficiency compared with the traditional SLNR, the proposed algorithm still has better performance than that of traditional SLNR in improving energy efficiency under the same condition.

Fig. 10

Comparison of system performance with different algorithms. a Comparison of system capacity. b Comparison of energy efficiency

It can be figured out that the traditional SLNR algorithm only reduce the inter-cell interference at receiver, the strong interference may weaken the interference suppression ability of SLNR, which causes the degradation of system performance. However, the proposed algorithm can reduce the number of inter-cell interferences in half before the interference suppression operation at receiver, so the interference environment at receiver is improved, and the application effect of the SLNR algorithm is enhanced; finally, the overall system performance is improved.

It should be noted that the analysis and studies of this paper are based on the ideal channel estimation and the actual method for obtaining channel state information is not further studied. Of course the acquisition of real-time channel state information under the actual channel condition can be studied in following. Furthermore, the performance of the proposed algorithm with real-time channel states information will be analyzed.

6 Conclusions

This paper firstly discusses the interference situations in multi-cell MIMO heterogeneous network. Aiming at the strong co-channel interference in multi-cell MIMO heterogeneous network, an algorithm that combines the triangular decomposition and SLNR has been proposed. The algorithm can reduce the number of inter-cell interferences in half through exploiting the triangular decomposition for equivalent channel matrix before the complex interference suppressing operation at receiver. Then based on the equivalent interference channel model extracted after triangular decomposition, the pre-coding matrices of each user in each cell and the corresponding closed-form interference suppression matrices are derived according to different interference situations in each cell. Furthermore, we compare the computation complexity of the proposed algorithm with traditional SLNR and interference alignment algorithm. Finally, the simulation results verify that the proposed algorithm can greatly improve the system capacity and energy efficiency compared with traditional SLNR algorithm. Meanwhile, the impact of different numbers of data streams and antennas on system performance is further analyzed.