Low complexity interference alignment algorithms for desired signal power maximization problem of MIMO channels

Abstract

In this article, we investigate the interference alignment (IA) solution for a K-user MIMO interference channel. Proper users’ precoders and decoders are designed through a desired signal power maximization model with IA conditions as constraints, which forms a complex matrix optimization problem. We propose two low complexity algorithms, both of which apply the Courant penalty function technique to combine the leakage interference and the desired signal power together as the new objective function. The first proposed algorithm is the modified alternating minimization algorithm (MAMA), where each subproblem has closed-form solution with an eigenvalue decomposition. To further reduce algorithm complexity, we propose a hybrid algorithm which consists of two parts. As the first part, the algorithm iterates with Householder transformation to preserve the orthogonality of precoders and decoders. In each iteration, the matrix optimization problem is considered in a sequence of 2D subspaces, which leads to one dimensional optimization subproblems. From any initial point, this algorithm obtains precoders and decoders with low leakage interference in short time. In the second part, to exploit the advantage of MAMA, it continues to iterate to perfectly align the interference from the output point of the first part. Analysis shows that in one iteration generally both proposed two algorithms have lower computational complexity than the existed maximum signal power (MSP) algorithm, and the hybrid algorithm enjoys lower complexity than MAMA. Simulations reveal that both proposed algorithms achieve similar performances as the MSP algorithm with less executing time, and show better performances than the existed alternating minimization algorithm in terms of sum rate. Besides, from the view of convergence rate, simulation results show that the MAMA enjoys fastest speed with respect to a certain sum rate value, while hybrid algorithm converges fastest to eliminate interference.

Introduction

Interference alignment (IA) technique is recently brought to show that each user can achieve half degree of freedom (DoF) in the K-user interference channel. It jointly optimizes precoding matrices for all transmitters, so that all interferences at one receiver fall into a reduced dimensional subspace. Then by multiplying decoding matrix orthogonal to this subspace, the certain receiver can extract the desired signals without interference. By utilizing the IA techniques, Cadambe and Jafar[1] showed that the achieved sum capacity of the K-user interference channel scales linearly with the number of users, in the high signal-to-noise-ratio (SNR) regime. Generally, the IA solutions are required to satisfy the following conditions simultaneously:

• (1) all the interferences are eliminated;

• (2) all the subspaces for desired signals are full rank;

• (3) precoders and decoders are required to be orthogonal.

The Yetis et al.[2] related the feasibility of the IA conditions in fully connected interference channel to the problem of determining the solvability of a multivariate polynomial system, with arbitrary antenna configurations. The achievable DoFs are also discussed based on this polynomial system, relating to users’ number of antennas and the number of users[3]. The analysis is further extended to that in partially connected channels[4–6].

From the signal processing point of view, the procedure of IA is to solve precoders and decoders according to the three conditions with a feasible IA system. However, the solution to this feasible problem is still not known in general. There are available closed form solutions only for certain cases, such as 3-user MIMO channel with N antennas each user equips with and N / 2 DoFs each user requires, and K-user channel where each user equips with K − 1 antennas and wishes to achieve 1 DoF. For general cases, the system is turned into an optimization problem minimizing the total leakage interference and preserving the orthogonality of precoders and decoders as constraints, which is denoted as leakage interference minimization (LIM) problem. With the solution, the IA condition 2 can be almost surely satisfied if channels have no special structures[7]. LIM problem is proved to be NP-hard when the number of antennas each user equips with are greater than 3[8]. Thus, iterative algorithms rather than the analytical solutions should be considered. Gomadam et al.[7] exploited the channel reciprocity and proposed an alternating minimization algorithm (AMA) to design precoders and decoders in a distributed way. Then each subproblem is equivalent to an eigenvalue problem with eigenvalue decomposition required. In the AMA, although the leakage interference can be perfectly canceled after convergence, its performance in terms of sum rate is not optimal. In fact, it is pointed out that for general interference channels the constructed LIM problem has a large number of different IA solutions obtained from different initial points, which lead to different achieved sum rate values[9]. The main reason is that the AMA only eliminates the interference in the desired signal space without considering the system sum rate, which results in a suboptimal sum rate achieved with finite signal power.

Gomadam et al.[7] also noticed the disadvantage of AMA, and then proposed the Max signal-to-interference-and-noise-ratio (SINR) algorithm as well. In each iteration, the basic idea of the Max-SINR algorithm is to choose the precoders and decoders stream by stream, with aim to maximize the SINR of each stream instead of minimizing the leakage interference. Due to the relaxation of the IA condition 3, the IA condition 2 does not hold anymore in this algorithm, which means the required DoFs might not be satisfied. This analysis accords with the performances shown in[7], that it achieves higher sum rate than the AMA sin intermediate SNR regime, however suffers from the loss of required DoFs in the high SNR regime. Many other algorithms can also be designed to perform like Max-SINR, however none of them can achieve the optimal DoFs without IA conditions[10]. Further, IA scheme provides the receivers interference-free subspaces, with which receivers completely get rid of complicated cancelation of interference. Thus, it is important to improve IA algorithms in general SNR scenarios.

Prior study in[11] proposes an iterative algorithm using the gradient descent method to solve the new IA model, where the utility function of either sum rate or the desired signal power is maximized with the IA conditions 1 and 3 as constraints. The corresponding MSP algorithm is shown to obtain higher sum rate than the AMA generally regardless of the initial point and higher than Max-SINR in high SNR regime. However, it requires a series of eigenvalue decompositions and compact singular value decompositions (SVDs), which lead to high computational complexity. Besides, the MSP algorithm has much slower convergence rate than the AMA.

Nevertheless, in practical systems, receivers have limited computational complexity, which might be a bottleneck for the complexity of the algorithms. Besides, channel reciprocity requires TDD operations which restrict the executing time of the algorithms. Therefore, the principal question here is how to design proper algorithms to balance the achieved sum rate with computational complexity and computing time. In this article, we aim to propose algorithms to maintain the advantage of the MSP algorithm with faster convergence, lower complexity and less executing time. Two efficient algorithms are proposed to solve the desired signal power maximization IA problem. First we propose a modified alternating minimization algorithm (MAMA) with Courant penalty function technique. Then, to further reduce the algorithm complexity, a new algorithm with Householder transformation (AHT) is proposed, where a two-dimensional subspace method is applied to solve subproblems. It acts to obtain precoders and decoders with low leakage interference in short time, and then the MAMA continues iterating to get perfect IA, which forms the hybrid algorithm.

The remaining article is organized as follows: In Section “Desired signal power maximization interference alignment model”, the desired signal power maximization model is presented. The two algorithms, MAMA and hybrid algorithm, are proposed in Sections “Modified alternating minimization algorithm” and “A hybrid algorithm”, respectively. The computational complexity of the proposed algorithms and MSP algorithm are analyzed in Section “Analysis of computational complexity”. Numerical results and further remarks are shown in Section “Simulations”.

Notation: Lowercase and uppercase boldface represent vectors and matrices, respectively. ℂ represents the complex domain. Re(a) means the real part of scalar a. tr(A) and ∥A∥ _F are the trace and the Frobenius norm of matrix A, respectively. I _d represents the d × d identity matrix.$\mathcal{K}$ represents the set of the user indices {1, 2, …, K}.$\mathcal{CN}$(μ,σ²) means the complex Gaussian distribution with mean μ and variance σ². And we use E _X (·) to denote the statistical expectation with the variable X. O(n) means the same order amount of n.

Desired signal power maximization IA model

Consider a K-user interference MIMO channel (M × N, d)^K as in Figure1, where each transmitter equips with M antennas, each receiver with N antennas and each user pair wishes to achieve d DoFs. Suppose s _k  ∈ ℂ^d×1 denotes the transmit signal vector of the k th user with power covariance as$E\left({\mathbf{s}}_k{\mathbf{s}}_k^H\right)=(P/d){\mathbf{I}}_d$, where P is the total transmit power of each user. For convenience we unify the transmit power of each stream, i.e. P / d = 1. After receiving the signal y _k , the k th receiver multiplies decoding matrix to it on the left, which is expressed as:

$${\mathbf{U}}_k^H{\mathbf{y}}_k=\underset{\text{desired signal}}{\underbrace{{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{kk}}{\mathbf{V}}_k{\mathbf{s}}_k}}+\underset{\text{interference}}{\underbrace{\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{s}}_l}}+\underset{\text{noise}}{\underbrace{{\mathbf{U}}_k^H{\mathbf{n}}_k}},$$

(1)

where H _kl  ∈ ℂ^N×M denotes the channel matrix between the k th transmitter and the l th receiver, and${\mathbf{V}}_k\in {ℂ}^{M\times d}$, U _k  ∈ ℂ^N×d represent the precoder and decoder of the k th user, respectively. The three terms of (1) on the right side represent the desired signal, the interference from other users and the noise with distribution of$\mathcal{C}\mathcal{N}(0,{\sigma }_k^2{\mathbf{I}}_N)$ at the k th receiver, respectively. In this article we assume all the noises have the same covariance, that is${\sigma }_k^2={\sigma }^2,k\in \mathcal{K}$.

Figure 1

K -user MIMO interference channel.

Following the feasible condition for IA system, IA scheme is defined as[7]:

$$\begin{array}{l}{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l=0,k\ne l,k,l\in \mathcal{K};\end{array}$$

(2)

$$\begin{array}{l}\text{rank}\left({\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{kk}}{\mathbf{V}}_k\right)=d,k\in \mathcal{K};\end{array}$$

(3)

$$\begin{array}{l}{\mathbf{V}}_k^H{\mathbf{V}}_k={\mathbf{I}}_d,{\mathbf{U}}_k^H{\mathbf{U}}_k={\mathbf{I}}_d,k\in \mathcal{K.}\end{array}$$

(4)

The original idea of IA only consists of (2) and (3). Noticing that for any precoding and decoding matrices {V _k , U _k , k ∈ $\mathcal{K}$} that satisfy these two conditions, {V _k P _k , U _k Q _k , k ∈ $\mathcal{K}$} also satisfy (2) and (3) as long as {P _k ,Q _k  ∈ℂ ^d×d,k ∈ $\mathcal{K}$} are all non-singular matrices. This indicates that the solutions of the IA system are not unique and the solution matrices form d-dimensional subspaces. Therefore, we require the columns of V _k ,U _k ,k ∈ $\mathcal{K}$ to be the orthogonal basis of the corresponding subspaces, which is the condition (4).

Besides the requirement of (2) and (4), we wish to maximize the desired signal power, in order to achieve sufficiently high sum rate. Suppose all the transmit signals are statistically independent of each other. We can obtain the expected total desired signal power

$$P^S=\sum _{k\in \mathcal{K}}{E}_{\mathbf{s}}(\parallel {\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{kk}}{\mathbf{V}}_k{\mathbf{s}}_k{\parallel }_2^2)=\sum _{k\in \mathcal{K}}\parallel {\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{kk}}{\mathbf{V}}_k{\parallel }_F^2.$$

Based on the above analysis, we present the desired signal power maximization (PM) model as follows:

$$\begin{array}{l}\underset{\begin{array}{c}{\mathbf{U}}_k,{\mathbf{V}}_k\\ k\in \mathcal{K}\end{array}}{\text{max}}{P}^S({\mathbf{U}}_k,{\mathbf{V}}_k)=\sum _{k\in \mathcal{K}}\parallel {\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}{\mathbf{V}}_k{\parallel }_F^2\end{array}$$

(5a)

$$\text{s.}\text{t.}{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l=0,k\ne l,k,l\in \mathcal{K},$$

(5b)

$${\mathbf{V}}_k^H{\mathbf{V}}_k={\mathbf{I}}_d,{\mathbf{U}}_k^H{\mathbf{U}}_k={\mathbf{I}}_d,k\in \mathcal{K}.$$

(5c)

This model was first brought in[11], in which its performance is compared with the sum rate maximization (SRM) model. The PM model can achieve similar performance as the SRM model, while its related optimization problem is much simpler. Therefore it is a good way to approximate the SRM model by the PM model[11]. Thus in this article we only focus on solving (5) to design proper precoders and decoders.

The sum of squares of the residuals of constraints (5b) is the total leakage interference, which is given by:

$$\begin{array}{ll}{P}^I\left({\mathbf{U}}_k,{\mathbf{V}}_k\right)& =\sum _{k\in \mathcal{K}}{E}_{\mathbf{s}}\left({∥\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{s}}_l∥}^2\right)\\ =\sum _{k\in \mathcal{K}}\sum _{l\ne k,l\in \mathcal{K}}\parallel {\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\parallel }_F^2.\end{array}$$

(6)

The essential idea of the AMA given by[7] is to minimize P^I alternately for U _k and V _k . From the formulation of (5), we know that the MSP algorithm in[11] solves it by increasing P^S while reducing P^I towards zero at the same time. Here we briefly introduce the framework of MSP. In each iteration for each precoder and decoder, an AMA step is first taken to reduce P^I. And then the iterative point go along the gradient direction of P^S to increase it, whose stepsize gradually shrinks to 0 in order to perfectly align the interference by simply taking AMA steps in the last few iterations. MSP algorithm requires a series of eigenvalue decompositions and SVDs, which bring in high computational complexity and complicated matrix computations. Therefore we wish to propose algorithms for (5) with lower complexity and less complicated computations.

Modified alternating minimization algorithm (MAMA)

In this section, we present our first low complexity algorithm, the MAMA. The main difficulty for solving the highly nonlinear nonconvex optimization problem (5) is to deal with the nonlinear constraint (5b). Noticing that (5a) and the penalty term (6) for (5b) have quite similar expressions, we can combine these two together by the Courant penalty function technique[12] as in the next section.

Courant penalty function technique

Courant penalty function technique is a classic penalty function technique, which avoids dealing with constraints by moving them to the objective function. The basic idea is to replace the constraints with a penalty term scaled by a parameter C. The penalty term is the sum of squares of all the constraint violations. In (5), we apply Courant penalty technique to (5b) and keep the constraints (5c) in order to obtain the following simple constrained problem:

$$\begin{array}{l}\underset{\begin{array}{c}{\mathbf{U}}_k,{\mathbf{V}}_k\\ k\in \mathcal{K}\end{array}}{\text{min}}P({\mathbf{U}}_k,{\mathbf{V}}_k,C)=C{P}^I({\mathbf{U}}_k,{\mathbf{V}}_k)-P^S({\mathbf{U}}_k,{\mathbf{V}}_k)\end{array}$$

(7a)

$$\begin{array}{l}\text{s.}\text{t.}{\mathbf{V}}_k^H{\mathbf{V}}_k={\mathbf{I}}_d,{\mathbf{U}}_k^H{\mathbf{U}}_k={\mathbf{I}}_d,k\in \mathcal{K},\end{array}$$

(7b)

where P^I is defined by (6).

It is well known that if (5) is feasible, the solution of (7) converges to that of (5) as C approaches infinity[13]. Thus the solution of (5) can be approximated by that of (7). As pointed out in[13], the approximated error between the solutions of (7) and (5) is of the level$O\left(\frac{1}{C}\right)$, which concerns about the choice of the penalty parameter C. Therefore, theoretically, (7) with larger C would get better approximation of (5). However, in real computations, large C may lead to ill-conditioned objective function and consequently bring in numerical calculation difficulties. To avoid such difficulties caused by unnecessarily large C in the first few iterations, we initially set C as a small positive number such as C = 1. If sufficient reduction in P^I is not achieved, we increase C to σ₀C, where σ₀ > 1 is a constant. The solution of (7), (U _k (C), V _k (C), k ∈ $\mathcal{K}$) provides a good initial point for solving (7) again when C is replaced by σ₀C. In this way we force P^I to reduce towards 0 eventually and the optimal solution of (5) can be obtained.

The overall algorithm

Via Courant penalty function technique our main task is now how to solve (7), which is quite similar to the LIM problem in which P^I is to be minimized. We can rewrite (7) as:

$$\begin{array}{l}\underset{\begin{array}{c}{\mathbf{U}}_k,{\mathbf{V}}_k\\ k\in \mathcal{K}\end{array}}{\text{min}}P({\mathbf{U}}_k,{\mathbf{V}}_k,C)\end{array}$$

(8a)

$$\begin{array}{l}=\sum _{k\in \mathcal{K}}\mathrm{tr}\left({\mathbf{U}}_k^H\left(C\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H-{\mathbf{H}}_{\mathit{\text{kk}}}{\mathbf{V}}_k{\mathbf{V}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}^H\right){\mathbf{U}}_k\right)\\ =\sum _{k\in \mathcal{K}}\mathrm{tr}\left({\mathbf{V}}_k^H\left(C\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l{\mathbf{U}}_l^H{\mathbf{H}}_{\mathit{\text{lk}}}-{\mathbf{H}}_{\mathit{\text{kk}}}^H{\mathbf{U}}_k{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}\right){\mathbf{V}}_k\right)\\ \mathrm{s.}\mathrm{t.}& {\mathbf{V}}_k^H{\mathbf{V}}_k={\mathbf{I}}_d,{\mathbf{U}}_k^H{\mathbf{U}}_k={\mathbf{I}}_d,k\in \mathcal{K.}\end{array}$$

(8b)

The only difference between the LIM problem and (8) is the objective function, as LIM minimizes the total leakage interference P^I subject to (8b). Thus, we can borrow the idea of the AMA in[7], to iterate V _k , k ∈ $\mathcal{K}$ and U _k , k ∈ $\mathcal{K}$ alternatively in each iteration. Fixing all V _k , k ∈ $\mathcal{K}$in (8), U _k , k ∈ $\mathcal{K}$ become independent of each other. In this case, (8) turns into K independent subproblems with formulas as:

$$\begin{array}{ll}\underset{{\mathbf{U}}_k}{\text{min}}& \mathrm{tr}\left({\mathbf{U}}_k^H\left(C\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H-{\mathbf{H}}_{\mathit{\text{kk}}}{\mathbf{V}}_k{\mathbf{V}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}^H\right){\mathbf{U}}_k\right)\hfill \\ \mathrm{s.}\mathrm{t.}& {\mathbf{U}}_k^H{\mathbf{U}}_k={\mathbf{I}}_d.\hfill \end{array}$$

(9)

We can obtain the closed-form solution of (9) as${\mathbf{U}}_k={\nu }_{\text{min}}^d\left({\mathbf{J}}_k\right),k\in \mathcal{K}$, which means the columns of U _k are eigenvectors corresponding to the d smallest eigenvalues of

$${\mathbf{J}}_k=C\sum _{l\ne k}{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H-{\mathbf{H}}_{\mathit{\text{kk}}}{\mathbf{V}}_k{\mathbf{V}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}^H.$$

Similar solutions of V _k can be achieved when fixing all U _k , k ∈ $\mathcal{K}$. Of course, we need a technique to update the penalty parameter C according to P^I during iterations.

Based on the above debold-scriptions, our MAMA for the PM model (5) is stated as follows (Algorithm 1).

Algorithm 1: modified alternating minimization algorithm

• 1. Set initial precoders V _k and decoders U _k , k ∈  $\mathcal{K}$, the initial penalty parameter C and σ ₀ > 1. ${\bar{P}}^I=P^I({\mathbf{U}}_k,{\mathbf{V}}_k)$.

• 2. For decoders: ${\mathbf{U}}_k={\nu }_{\mathit{\text{min}}}^d\left({\mathbf{J}}_k\right),k\in \mathcal{K}$ from above.

• 3. For precoders: ${\mathbf{V}}_k={\nu }_{\mathit{\text{min}}}^d\left({\tilde{\mathbf{J}}}_k\right),k\in \mathcal{K}$, where ${\tilde{\mathbf{J}}}_k=C\sum _{l\ne k}{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l{\mathbf{U}}_l^H{\mathbf{H}}_{\mathit{\text{lk}}}-{\mathbf{H}}_{\mathit{\text{kk}}}^H{\mathbf{U}}_k{\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kk}}}$.

• 4. If the algorithm converges, then stop and output V _k , U _k , k ∈  $\mathcal{K}$. If $P^I({\mathbf{U}}_k,{\mathbf{V}}_k)>{\bar{P}}^I-1/C$, then increase the penalty parameter C := σ ₀ C. If $P^I({\mathbf{U}}_k,{\mathbf{V}}_k)<{\bar{P}}^I$, let ${\bar{P}}^I=P^I({\mathbf{U}}_k,{\mathbf{V}}_k)$. Go to step 2.

In the original AMA of[7], eigenvectors corresponding to the d smallest eigenvalues of positive semi-definite matrices$\{{\sum }_{l\ne k}{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H,k\in \mathcal{K}\}$ and those of$\{{\sum }_{l\ne k}{\mathbf{H}}_{\mathit{\text{kl}}}^H{\mathbf{U}}_l{\mathbf{U}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}},k\in \mathcal{K}\}$ are required. In our MAMA, the matrices J _k and${\tilde{\mathbf{J}}}_k$ are not necessarily positive semi-definite, since the objective function value of (8) might be negative with feasible solution of quite low P^I and high P^S.

Although our MAMA avoids calculating SVD which is required by the MSP algorithm given by[11], eigenvalue decompositions are still required for solving subproblems. This lends an impetus to further improvement.

A hybrid algorithm

In order to further reduce the complexity of our MAMA, we propose a hybrid technique in this section. First, we give an AHT, which is a very low complexity algorithm for (5). This algorithm is free from complicated matrix computation, such as eigenvalue decomposition, SVD and QR factorization, and enjoys lower computational complexity than MAMA.

Algorithm with Householder transformation

In this algorithm, we still use the Courant penalty function technique. Thus we focus on problem (7). The only constraints of (7) are orthogonal constraints. We require all the iterative points U _k and V _k generated by our algorithms are feasible, namely (7b) are always satisfied. By requiring feasibility, basically we are solving an unconstrained optimization problem on the Grassmann manifold. Otherwise, if we allow iterations go outside the feasible region, we will have to adopt some technique, such as projection, to draw the iterations back to the feasible region, which can be complicated. For example, the projection to the set of orthogonal matrices can be a non-differentiable operator and time-consuming to compute.

Preserving orthogonality

Let${\mathbf{V}}_k^i$ denotes the precoder of the k th user in the i th iteration. Suppose both${\mathbf{V}}_k^i$ and${\mathbf{V}}_k^{i+1}$ are orthogonal, there should exist a unitary matrix${\mathbf{P}}_k^i\in {ℂ}^{M\times M}$ such that

$${\mathbf{P}}_k^i{\mathbf{V}}_k^i={\mathbf{V}}_k^{i+1}.$$

The straightforward approach is to search for all unitary matrices${\mathbf{P}}_k^i\in {ℂ}^{M\times M}$, but this can be computationally very expensive. We consider a special class of unitary matrices, the Householder matrices, which can be represented as P = I − 2pp^H with p^Hp = 1. The use of Householder transformation can not only preserve orthogonality, but also turn the problem from a matrix optimization into a vector optimization problem. In the i th iteration with fixed${\mathbf{V}}_k^i,{\mathbf{U}}_k^i,k\in \mathcal{K}$ as certain feasible solution of (7), we try to obtain orthogonal vectors V _k , U _k , k ∈ $\mathcal{K}$ and set

$${\mathbf{V}}_k^{i+1}=({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i,{\mathbf{U}}_k^{i+1}=({\mathbf{I}}_N-2{\mathbf{u}}_k{\mathbf{u}}_k^H){\mathbf{U}}_k^i,k\in \mathcal{K}$$

(10)

as the precoders and decoders in the (i + 1)th iteration. Here, V _k , U _k , k ∈ $\mathcal{K}$ are regarded as optimal (or suboptimal) solutions to minimize the objective function value of (7),$P({\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1},C)=P\left(\right({\mathbf{I}}_N-2{\mathbf{u}}_k{\mathbf{u}}_k^H){\mathbf{U}}_k^i,({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i,C)$.

As precoding and decoding matrices are updated according to Householder transformation in each iteration, the new algorithm is denoted as AHT.

Alternating directions

Substituting (10) into (7), we get (11) with variables V _k , U _k , k ∈ $\mathcal{K}$:

$$\begin{array}{ll}\underset{\begin{array}{c}{\mathbf{u}}_k,{\mathbf{v}}_k\\ k\in \mathcal{K}\end{array}}{\text{min}}& \sum _{k\in \mathcal{K}}\left(C\sum _{l\ne k,l\in \mathcal{K}}\parallel {\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{V}}_k^{i+1}{\parallel }_F^2-\parallel {\left({\mathbf{U}}_k^{i+1}\right)}^H{\mathbf{H}}_{\mathit{kk}}{\mathbf{V}}_k^{i+1}{\parallel }_F^2\right)\\ =\sum _{k\in \mathcal{K}}\left(C\sum _{l\ne k}\parallel {\left({\mathbf{U}}_l^i\right)}^H({\mathbf{I}}_N-2{\mathbf{u}}_l{\mathbf{u}}_l^H){\mathbf{H}}_{\mathit{\text{lk}}}({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i{\parallel }_F^2\right.\\ \left.-\parallel {\left({\mathbf{U}}_k^i\right)}^H({\mathbf{I}}_N-2{\mathbf{u}}_k{\mathbf{u}}_k^H){\mathbf{H}}_{\mathit{kk}}({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i{\parallel }_F^2\right)\\ \text{s.}\text{t.}& {\mathbf{v}}_k^H{\mathbf{v}}_k=1,{\mathbf{u}}_k^H{\mathbf{u}}_k=1,k\in \mathcal{K}.\end{array}$$

(11)

With the fact that

$$\begin{array}{l}\parallel {\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i{\parallel }_F^2\\ =& \mathrm{tr}\left(\right.{\left({\mathbf{V}}_k^i\right)}^H({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}\left({\mathbf{U}}_l^{i+1}\right)\\ {}^H{\mathbf{H}}_{\mathit{\text{lk}}}({\mathbf{I}}_M-2{\mathbf{v}}_k{\mathbf{v}}_k^H){\mathbf{V}}_k^i\left)\right.\\ =& 4\mathrm{tr}\left({\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k{\mathbf{v}}_k^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{v}}_k{\mathbf{v}}_k^H{\mathbf{V}}_k^i\right)\\ -2\mathrm{tr}\left(\right.{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k{\mathbf{v}}_k^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{V}}_k^i\\ +{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{v}}_k{\mathbf{v}}_k^H{\mathbf{V}}_k^i\left)\right.\\ +\mathrm{tr}\left({\left({\mathbf{V}}_k^i\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{V}}_k^i\right)\\ =& 4\left({\mathbf{v}}_k^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{v}}_k\right)\left({\mathbf{v}}_k^H{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k\right)\\ -2({\mathbf{v}}_k^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k\\ +{\mathbf{v}}_k^H{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{v}}_k)\\ +\mathrm{tr}\left({\left({\mathbf{V}}_k^i\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}{\mathbf{V}}_k^i\right),\end{array}$$

(12)

we can rewrite the objective function of (11) as:

$$\begin{array}{l}\sum _{k\in \mathcal{K}}\left[4\left({\mathbf{v}}_k^H{\mathbf{A}}_k{\mathbf{v}}_k\right)\left({\mathbf{v}}_k^H{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k\right)\right.\\ -2\left({\mathbf{v}}_k^H{\mathbf{A}}_k{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{v}}_k+{\mathbf{v}}_k^H{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H{\mathbf{A}}_k{\mathbf{v}}_k\right)\\ \left.+\mathrm{tr}\left({\left({\mathbf{V}}_k^i\right)}^H{\mathbf{A}}_k{\mathbf{V}}_k^i\right)\right],\end{array}$$

(13)

where${\mathbf{A}}_k=C{\sum }_{l\ne k,l\in \mathcal{K}}{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}-{\mathbf{H}}_{\mathit{kk}}^H{\mathbf{U}}_k^{i+1}{\left({\mathbf{U}}_k^{i+1}\right)}^H{\mathbf{H}}_{\mathit{kk}},k\in \mathcal{K}$.

Fixing all u _l , l ∈ $\mathcal{K}$ in (11),${\mathbf{U}}_l^{i+1},l\in \mathcal{K}$ are determined according to (10), and thus V _k , k ∈ $\mathcal{K}$ become independent of each other. Therefore, (11) is decomposed into K subproblems with the same form as follows:

$$\begin{array}{l}\underset{\mathbf{x}\in {ℂ}^M}{\text{min}}f\left(\mathbf{x}\right)=\left({\mathbf{x}}^H\mathbf{Ax}\right)\left({\mathbf{x}}^H\mathbf{Bx}\right)-{\mathbf{x}}^H\mathbf{Cx}\\ \text{s.}\text{t.}{\mathbf{x}}^H\mathbf{x}=1.\end{array}$$

(14)

Here x represents V _k .$\mathbf{A}=2\left(C{\sum }_{l\ne k,l\in \mathcal{K}}\right.{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l^{i+1}\left.{\left({\mathbf{U}}_l^{i+1}\right)}^H{\mathbf{H}}_{\mathit{\text{lk}}}-{\mathbf{H}}_{\mathit{kk}}^H{\mathbf{U}}_k^{i+1}{\left({\mathbf{U}}_k^{i+1}\right)}^H{\mathbf{H}}_{\mathit{kk}}\right)$,$\mathbf{B}=2{\mathbf{V}}_k^i{\left({\mathbf{V}}_k^i\right)}^H$ and C = (AB + BA)/2. Similarly, fixing all${\mathbf{v}}_k,k\in \mathcal{K}$, it yields K subproblems like (14) from (11). Based on the analysis above, the alternating direction method can be applied to optimize V _k , k ∈ $\mathcal{K}$ and U _k , k ∈ $\mathcal{K}$ alternatively.

Subproblem: 2D subspace method

Subproblem (14) is a nonlinear optimization on the Stiefel manifold in complex field, which is not easy to handle. With the aim of low complexity, we abandon the second order method where Hessian matrix information is required. Instead, we apply a 2D subspace method to solve it iteratively[14]. We define the subspace spanned by x₀ and$\tilde{\mathbf{g}}$ as

$$S({\mathbf{x}}_0,\tilde{\mathbf{g}})=\left\{\mathbf{x}:\mathbf{x}=b{\mathbf{x}}_0+a\tilde{\mathbf{g}}\mathrm{.a}\in [-1,1],b=\sqrt{1-a^2}\right\},$$

where x₀ is the current iterative feasible point and$\tilde{\mathbf{g}}$ represents its normalized gradient of the Lagrangian function of (14). In each step, the feasible domain of x shrinks from the Stiefel manifold x^Hx = 1 to$S({\mathbf{x}}_0,\tilde{\mathbf{g}})$.

Here$\tilde{\mathbf{g}}$ is calculated as follows. First, the Lagrangian function of (14) is expressed as:

$$L(\mathbf{x},\lambda )=f\left(\mathbf{x}\right)-\lambda ({\mathbf{x}}^H\mathbf{x}-1),$$

(15)

where λ is the Lagrange multiplier. Then the gradient of (15) is

$$\mathbf{g}={\mathbf{g}}_0-\lambda {\mathbf{x}}_0,$$

(16)

where g₀ is the gradient of the objective function f at the point x₀:

$${\mathbf{g}}_0=2\left[\right({\mathbf{x}}_0^H{\mathbf{Ax}}_0){\mathbf{Bx}}_0+({\mathbf{x}}_0^H{\mathbf{Bx}}_0){\mathbf{Ax}}_0-{\mathbf{Cx}}_0].$$

As a necessary condition of the first order optimality condition of (14), we require${\mathbf{g}}^H{\mathbf{x}}_0=0$. Thus$\lambda =2\left({\mathbf{x}}_0^H{\mathbf{Ax}}_0\right)\left({\mathbf{x}}_0^H{\mathbf{Bx}}_0\right)-{\mathbf{x}}_0^H{\mathbf{Cx}}_0$ is deduced due to${\mathbf{x}}_0^H{\mathbf{x}}_0=1$. Once λ is computed, g can be chosen by (16) and we can let$\tilde{\mathbf{g}}=\mathbf{g}/\parallel \mathbf{g}{\parallel }_2$.

To maintain$S({\mathbf{x}}_0,\tilde{\mathbf{g}})$ on the Stiefel manifold, any point$\mathbf{x}\in S({\mathbf{x}}_0,\tilde{\mathbf{g}})$ should satisfy${\mathbf{x}}^H\mathbf{x}=b^2\parallel {\mathbf{x}}_0{\parallel }^2+a^2\parallel \tilde{\mathbf{g}}{\parallel }^2=1$. Thus a² + b² = 1, a ∈ [−1, 1], namely$b=\pm \sqrt{1-a^2}$. For any a ∈ [−1, 1], the objective function of (14) share the same function value at$\sqrt{1-a^2}{\mathbf{x}}_0+a\tilde{\mathbf{g}}$ and$-\sqrt{1-a^2}{\mathbf{x}}_0-a\tilde{\mathbf{g}}$. Therefore, we only consider the case$b=\sqrt{1-a^2}$ to avoid redundant solutions.

Taking the expression of$\mathbf{x}\in S({\mathbf{x}}_0,\tilde{\mathbf{g}})$ into (14), it turns into a one dimensional constraint optimization problem with variable a:

$$\underset{a\in [-1,1]}{\text{min}}{p}_{1}a\sqrt{1-a^2}+p_2{a}^2+p_3{a}^3\sqrt{1-a^2}+p_4{a}^4,$$

(17)

where the current iterative point x₀ corresponds to a = 0, and

$$\begin{array}{lll}{p}_1& =\hfill & 2\left[\mathrm{Re}\right({\mathbf{x}}_0^H\mathbf{A}\tilde{\mathbf{g}}\left)\right({\mathbf{x}}_0^H{\mathbf{Bx}}_0)+\mathrm{Re}({\mathbf{x}}_0^H\mathbf{B}\tilde{\mathbf{g}}\left)\right({\mathbf{x}}_0^H{\mathbf{Ax}}_0)-\mathrm{Re}({\mathbf{x}}_0^H\mathbf{C}\tilde{\mathbf{g}}\left)\right],\hfill \\ p_2& =\hfill & \left({\mathbf{x}}_0^H{\mathbf{Ax}}_0\right)\left({\tilde{\mathbf{g}}}^H\mathbf{B}\tilde{\mathbf{g}}\right)+\left({\mathbf{x}}_0^H{\mathbf{Bx}}_0\right)\left({\tilde{\mathbf{g}}}^H\mathbf{A}\tilde{\mathbf{g}}\right)-2\left({\mathbf{x}}_0^H{\mathbf{Ax}}_0\right)\left({\mathbf{x}}_0^H{\mathbf{Bx}}_0\right)\hfill \\ +4\mathrm{Re}\left({\mathbf{x}}_0^H\mathbf{A}\tilde{\mathbf{g}}\right)\mathrm{Re}\left({\mathbf{x}}_0^H\mathbf{B}\tilde{\mathbf{g}}\right)+{\mathbf{x}}_0^H{\mathbf{Cx}}_0-{\tilde{\mathbf{g}}}^H\mathbf{C}\tilde{\mathbf{g}},\\ p_3& =\hfill & 2\left[\mathrm{Re}\right({\mathbf{x}}_0^H\mathbf{B}\tilde{\mathbf{g}}\left)\right({\tilde{\mathbf{g}}}^H\mathbf{A}\tilde{\mathbf{g}}-{\mathbf{x}}_0^H{\mathbf{Ax}}_0)+\mathrm{Re}({\mathbf{x}}_0^H\mathbf{A}\tilde{\mathbf{g}}\left)\right({\tilde{\mathbf{g}}}^H\mathbf{B}\tilde{\mathbf{g}}-{\mathbf{x}}_0^H{\mathbf{Bx}}_0\left)\right],\hfill \\ p_4& =\hfill & ({\tilde{\mathbf{g}}}^H\mathbf{A}\tilde{\mathbf{g}}-{\mathbf{x}}_0^H{\mathbf{Ax}}_0)({\tilde{\mathbf{g}}}^H\mathbf{B}\tilde{\mathbf{g}}-{\mathbf{x}}_0^H{\mathbf{Bx}}_0)-4\mathrm{Re}\left({\mathbf{x}}_0^H\mathbf{A}\tilde{\mathbf{g}}\right)\mathrm{Re}\left({\mathbf{x}}_0^H\mathbf{B}\tilde{\mathbf{g}}\right).\hfill \end{array}$$

Newton’s method with line search is applied to solve (17) and a local minimizer is guaranteed to be found[13].

The method to solve subproblem (14) is summarized as follows:

Subalgorithm: two dimensional subspace method

• 1. Given any initial feasible point x ₀ and the maximum iteration number T ^inner, t ₁ = 1.

• 2. Calculate $\tilde{\mathbf{g}}$ and construct subproblem (17).

• 3. Solve (17) by Newton’s method with line search and ${\mathbf{x}}_0:=\sqrt{1-a^2}{\mathbf{x}}_0+a\tilde{\mathbf{g}}$. If the objective function of (14) does not reduce or t ₁ = T ^inner, stop and output x ₀; else t ₁ := t ₁ + 1, go to Step 2.

The maximum iteration number T^inner relates to the efficiency of the algorithm. It should not be too large or too small, in order to avoid spending too much time for subproblem or insufficient reduction of the objective function value. We will discuss its specific choice in Section “Simulations”.

As the projection to the Stiefel manifold x^Hx = 1 is very easy to compute, projected gradient method is also a common approach to solve (14). In the following theorem and remark, we compare our 2D subspace method with the projected gradient method and show the advantage of our method.

Theorem 1

Denote the projected gradient step started from x ₀ with stepsize α as:

$$\mathbf{x}\left(\alpha \right)=\frac{{\mathbf{x}}_0-\alpha {\mathbf{g}}_0}{\parallel {\mathbf{x}}_0-\alpha {\mathbf{g}}_0\parallel }.$$

(18)

On one hand, for any vector$\mathbf{x}\in S({\mathbf{x}}_0,\tilde{\mathbf{g}})$, there exists a stepsize α  ∈ ℝ, such that x is expressed as (18); on the other hand, for any α  ∈ ℝ, the projected gradient step x ( α ) can be expressed as the linear combination of x ₀ and$\tilde{\mathbf{g}}$, which means$\mathbf{x}\left(\alpha \right)\in S({\mathbf{x}}_0,\tilde{\mathbf{g}})$.

Proof. See the Appendix.

Remark 1

Theorem 1 shows that step in certain 2D subspace and the projected gradient step can be mutually expressed. Thus the minimization of the objective function f on the 2D subspace $S({\mathbf{x}}_0,\tilde{\mathbf{g}})$ and that along the projected gradient step are essentially the same. The 2D subspace problem is easy to handle, as shown above; but the search for (18) may be quite difficult, because the projection operator would make the objective function nondifferentiable and lose the common Taylor expansion, consequently it is nontrivial to search for proper stepsize α . Due to these considerations, we use the 2D subspace method rather than the projected gradient method to solve (14).

Framework of AHT algorithm

According to the above discussions, the framework of the AHT algorithm can be concluded as follows (Algorithm 2).

Algorithm 2: algorithm with Householder transformation

• 1. Set initial precoders and decoders as ${\mathbf{V}}_k^0,{\mathbf{U}}_k^0,k\in \mathcal{K}$, parameter T ^inner in 2D subspace, the maximum inner iteration number T ^outer, the initial penalty parameter C, parameter σ ₀ > 1 for C and the stopping parameter ε. i = 0.

• 2. Solve (11) iteratively: fix ${\mathbf{U}}_k^i,{\mathbf{V}}_k^i,k\in \mathcal{K}$, and set the initial normalized vectors U _k , V _k , k ∈  $\mathcal{K}$. t = 1.

  • (2.1) For precoders: fix U _k , k ∈ $\mathcal{K}$. For k ∈ $\mathcal{K}$, construct A, B, C in (14) and solve it with 2D subspace method to obtain V _k .

  • (2.2) For decoders: fix V _k , k ∈ $\mathcal{K}$. Similarly to Step 2.1, obtain new${\mathbf{u}}_k,k\in \mathcal{K}$ by solving (14).

  • (2.3) Calculate${\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1},k\in \mathcal{K}$ from (10). If t = T^outer or$P^I({\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1})\le P^I({\mathbf{U}}_k^i,{\mathbf{V}}_k^i)-1/C$, go to Step 3; else t := t + 1, go to step 2.1.

• 3. If $P^I({\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1})>P^I({\mathbf{U}}_k^i,{\mathbf{V}}_k^i)-1/C$, increase the penalty parameter C := σ ₀ C. If $P^I({\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1})<\epsilon$, stop and output ${\mathbf{U}}_k:={\mathbf{U}}_k^{i+1},{\mathbf{V}}_k:={\mathbf{V}}_k^{i+1},k\in \mathcal{K}$; else i := i + 1, go to Step 2.

In our simulations, t is usually no more than 5 when$P^I({\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1})\le P^I({\mathbf{U}}_k^i,{\mathbf{V}}_k^i)-1/C$ is achieved in Step 2. T^outer, the maximum number of inner iterations inside Step 2, acts to avoid extreme situation with too much time to solve subproblems. Thus we suggest T^outer = 5.

Hybrid algorithm

Although the AHT algorithm enjoys lower complexity than MAMA as analyzed in the next section, it solves subproblem (14) inexactly by successive 2D subspace minimizations. This reduces the objective function value rapidly at first, however the speed becomes much more slowly when the iterative point approaches a local optimal solution. Simulation also verifies that it is quite difficult to align the interference perfectly with the AHT algorithm. In contrast, MAMA solves subproblem (9) exactly with closed form solution. The sequence generated by MAMA converges fast near the solution, which is complementary to that of AHT.

In order to make full use of the low complexity of AHT and the fast local convergence property of MAMA, it seems reasonable for us to combine these two methods together into a hybrid algorithm. We believe that such a hybrid algorithm will enjoy the properties of low complexity and fast local convergence, which are verified by our simulation tests reported in Section “Simulations”.

This hybrid algorithm consists of two stages. In the first stage, it uses the AHT algorithm to iterate from any starting point to a point near the solution. Then, the algorithm switches to MAMA, namely it applies MAMA to converge to the solution, by starting from the point obtained by AHT (Algorithm 3).

Algorithm 3: hybrid algorithm

• 1. Set initial precoders and decoders as ${\mathbf{V}}_k^0,{\mathbf{U}}_k^0,k\in \mathcal{K}$, the penalty parameter C = 1.

• 2. Input the initial iterative point and penalty parameter into the AHT algorithm. Set its stopping criterion as $P^I({\mathbf{U}}_k,{\mathbf{V}}_k)<0.01P^I({\mathbf{U}}_k^0,{\mathbf{V}}_k^0)$. Output the corresponding V _k , U _k , k ∈  $\mathcal{K}$ and the current penalty parameter C _AHT = C.

• 3. Input V _k , U _k , k ∈  $\mathcal{K}$ and C = C _AHT as the initial iterative point and penalty parameter of MAMA, respectively. Iterate until convergence by the MAMA. Output the corresponding precoders and decoders V _k , U _k , k ∈  $\mathcal{K}$.

In some sense, it can be viewed that the AHT algorithm acts as a method to rapidly find a good initial point with low interference for MAMA. Starting from the good initial point, MAMA can converge very fast to a local optimal solution with perfect IA.

Analysis of computational complexity

In this section, we analyze the computational complexity of IA algorithms for PM model, including our two proposed algorithms MAMA and AHT, as well as the MSP algorithm from[11].

Advantages of low complexity algorithms

Before presenting the detailed analysis on the computational complexities of different algorithms, we provide the reasons for constructing low computational complexity IA algorithms for the PM model.

• 1. No matter our algorithms or the existed IA algorithms, they all exploit the channel reciprocity to design the transmit precoders and receiver decoders in a distributed way. Such reciprocity is based on the time division duplex (TDD) operation with synchronized time-slot in the practical system. However, due to the time varying property of wireless channel, perfect reciprocity may be hard to achieve and result in residual interference at the receivers. This requires algorithms with short computing time to reduce the possibility of performance loss due to the imperfect reciprocity in practical systems [15].

• 2. Furthermore, that the receivers generally have limited computational complexity, might be a bottleneck for the complexity of the algorithms. Thus we need to design algorithms with low complexity and simple computation, to satisfy the computation restriction of communication equipments.

• 3. Also, algorithms with lower complexity are easier to be extended to large scale problems. For small scale problems, algorithms with low or high complexity do not differ much from each other. However as the problem dimension grows, high complexity algorithms can be very very slow for large scale problems because of too much computing time, while algorithms with lower complexity are more preferable.

Detailed analysis

Here we consider the number of complex multiplications as the complexity criterion. The AHT, MAMA and MSP algorithm are analyzed and compared. Their main computations in one iteration are listed as follows:

• 1. The computation of ${\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H$ and ${\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l{\mathbf{U}}_l^H{\mathbf{H}}_{\mathit{\text{lk}}},k,l\in \mathcal{K}$ are required for all the three algorithms. Let ${\tilde{\mathbf{H}}}_{\mathit{\text{kl}}}={\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l$. Then we can calculate ${\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H$ as ${\tilde{\mathbf{H}}}_{\mathit{\text{kl}}}{\tilde{\mathbf{H}}}_{\mathit{\text{kl}}}^H$. Similarly we can compute ${\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l{\mathbf{U}}_l^H{\mathbf{H}}_{\mathit{\text{lk}}}$ by introducing ${\bar{\mathbf{H}}}_{\mathit{\text{lk}}}={\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l$. As k and l traverse all the elements in $\mathcal{K}$, the entire complexity is K ²[(dMN + d N ²) + (dMN + d M ²)] = K ² d(2MN + N ² + M ²).

• 2. Besides term 1, the AHT algorithm requires to compute ${\mathbf{U}}_k{\mathbf{U}}_k^H$ and ${\mathbf{V}}_k{\mathbf{V}}_k^H$, with complexity of Kd(M ² + N ²).

• 3. P ^I is computed in both the AHT and MAMA. Noticing that

  $$P^I=\sum _{k\in \mathcal{K}}\sum _{l\ne k,l\in \mathcal{K}}\parallel {\mathbf{U}}_k^H{\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\parallel }_F^2=\sum _{k\in \mathcal{K}}\sum _{l\ne k,l\in \mathcal{K}}\parallel {\mathbf{V}}_k^H{\mathbf{H}}_{\mathit{\text{lk}}}^H{\mathbf{U}}_l{\parallel }_F^2,$$

we can compute$\parallel {\mathbf{U}}_k^H{\tilde{\mathbf{H}}}_{\mathit{\text{kl}}}{\parallel }_F^2$ if N < M, or$\parallel {\mathbf{V}}_k^H{\bar{\mathbf{H}}}_{\mathit{\text{lk}}}{\parallel }_F^2$ otherwise, where${\tilde{\mathbf{H}}}_{\mathit{\text{kl}}}$ and${\bar{\mathbf{H}}}_{\mathit{\text{lk}}}$ are the pre-computed parameters in term 1. With the above analysis, the corresponding complexity is$K(K-1)d^2\left(\text{min}\right(M,N)+1)$.

• 4. AHT algorithm requires to update precoders and decoders by computing ${\mathbf{U}}_k^{i+1},{\mathbf{V}}_k^{i+1},k\in \mathcal{K}$ in (10). With ${\mathbf{U}}_k^{i+1}={\mathbf{U}}_k^i-2{\mathbf{u}}_k\left({\mathbf{u}}_k^H{\mathbf{U}}_k^i\right)$, the complexity is K[(d ² + dM) + (d ² + dN)] = Kd(2d + M + N).

• 5. In each inner iteration of our 2D subspace method for (14) in the AHT algorithm, it mainly requires computation of Ax, Bx, Cx = [A(Bx) + B(Ax)] /2, Ag and Bg, the complexity of which is 6N ², where n is the dimension of x. Suppose for each 2D subspace method, there are mostly T ^inner inner iterations, we can see that each iteration of AHT requires the total complexity of 6K T ^inner (M ² + N ²).

• 6. One eigenvalue decomposition of matrix with dimension M × M requires 9M ³ complexity [16]. Both the MAMA and MSP algorithm need K eigenvalue decompositions of M × M matrices and K with dimension N × N. The entire complexity is 9K(M ³ + N ³).

• 7. The complexity of one compact SVD of matrix with dimension M × d(d < M) is 14M d ² + 8d ³ [16]. The MSP algorithm requires K compact SVDs for matrices with dimension M × d and K with dimension N × d, whose complexity is K d ²(14M + 14N + 16d).

Table1 shows the specific computational complexity of the three algorithms. Comparing the AHT and MAMA, besides the common terms, AHT owns term 2,4,5, while MAMA has term 6. With the fact that 1 ≤ d ≤ min(M, N), we can deduce that:

$$\begin{array}{l}\mathit{\text{Kd}}(2d+M+N)+\mathit{\text{Kd}}(M^2+N^2)\\ \le 2\mathit{\text{Fd}}(M+N)+\mathit{\text{Fd}}(M^2+N^2)\\ \le 3K(M^2+N^2)<3K(M^3+N^3).\end{array}$$

(19)

For term 5, as long as T^inner ≤ min(M, N), we have

$$6K{T}^{\text{inner}}(M^2+N^2)\le 6K(M^3+N^3).$$

(20)

Adding both sides of (19) and (20), we conclude that AHT has lower complexity than MAMA.

Table 1 Comparison of computational complexity

Similarly, we compare term 3 and term 7 to see the difference between the complexity of MAMA and MSP. As long as K ≤ 23 (which is usually the case in practical considered IA problems), the following inequality holds:

$$\begin{array}{l}\begin{array}{c}K(K-1)d^2\left(\text{min}\right(M,N)+1)\le 22K{d}^2\left(\text{min}\right(M,N)+1)\\ \le K{d}^2\left(22\text{min}\right(M,N)+22d)\le K{d}^2(14M+14N+16d).\end{array}\end{array}$$

So the complexity of MAMA is lower than that of the MSP algorithm. Thus the algorithms ranked from low to high complexity in one iteration are AHT, MAMA and MSP. This also implies that the hybrid algorithm has lower complexity than MAMA and MSP.

Our above analysis compares the complexity of each iteration of different algorithms. However, in order to compare the computational complexity of different algorithms, we need to estimate the total number of iterations of all the algorithms under considerations, which is not easy. Therefore, we try to explain it by convergence curves with consuming time in simulations.

Simulations

In this section, we analyze our proposed algorithms the MAMA, AHT and hybrid algorithm by simulations, and compare them with the existed AMA[7] and MSP algorithm proposed in[11]. (5 × 5, 2)⁴ interference channels are considered, that is, K = 4, M = N = 5, d = 2, which satisfy the general feasibility condition[2, 3]. Each component of H _kl , k, l ∈ $\mathcal{K}$ is i.i.d complex Gaussian distribution$\mathcal{CN}$(0, 1).

We use sum rate as the measure of quality of service. The sum rate of the K-user MIMO interference channel is expressed as follows:

$$\begin{array}{ll}R& =\sum _{k\in \mathcal{K}}log\left|{\mathbf{I}}_N+{\mathbf{F}}_k^{-1}{\mathbf{L}}_{\mathit{kk}}\right|\\ =\sum _{k\in \mathcal{K}}log\left|{\mathbf{I}}_N+{\left({\sigma }^2{\mathbf{I}}_N+\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{L}}_{\mathit{\text{kl}}}\right)}^{-1}{\mathbf{L}}_{\mathit{kk}}\right|,\end{array}$$

(21)

where${\mathbf{L}}_{\mathit{\text{kl}}}={\mathbf{H}}_{\mathit{\text{kl}}}{\mathbf{V}}_l{\mathbf{V}}_l^H{\mathbf{H}}_{\mathit{\text{kl}}}^H,k,l\in \mathcal{K}$,${\mathbf{F}}_k={\sigma }^2{\mathbf{I}}_N+\sum _{l\ne k,l\in \mathcal{K}}{\mathbf{L}}_{\mathit{\text{kl}}}$ and σ² is the covariance of the additive white Gaussian noise at the receivers. Here we define the SNR as SNR = P / σ² = d / σ².

In the AHT algorithm, we set T^outer = 5 and initially C = 1. For each figure, 250 random realization of different channel coefficients are generated to evaluate the average performance. For each realization the initial values of precoders and decoders${\mathbf{V}}_k^0,{\mathbf{U}}_k^0,k\in \mathcal{K}$ are randomly generated and remain the same in the compared algorithms.

Parameter analysis

As mentioned before, the choice of T^inner relates to the efficiency of the AHT algorithm and also the hybrid algorithm. In Figure2 we plot the convergence curves of the hybrid algorithms with parameter T^inner = 3, 5, 10, respectively. SNR equals 30 dB and σ₀ = 20 here.

Figure 2

Convergence of the average sum rate of the hybrid algorithms with T^inner = 3, 5, 10.

Figure2 reveals that, both the converged average sum rate and the convergence rate of the hybrid algorithms with different T^inner are quite similar. The curve with T^inner = 10 seems to be the worst, which accords with our complexity analysis as T^inner > min(M, N) = 5. In the upcoming simulations we set T^inner = 5 with which the hybrid algorithm obtains the highest sum rate, to improve the efficiency of our proposed algorithms.

Besides T^inner, we also compare the performances of the hybrid algorithms with different σ₀. σ₀ determines the growth speed of the penalty parameter C, which represents the weight of P^I in the objective function of (7). If σ₀ is too small, it would take many iterations for C to grow large, and thus affects the efficiency of the hybrid algorithm. On the other side, too large σ₀ forces C to grow too fast, which makes (7) degenerate into the LIM problem in short time and that P^S cannot be increased efficiently.

Based on the aforementioned analysis, we plot the convergence curves of the hybrid algorithms with σ₀ = 10, 20, 50 in Figure3. SNR is also set as 30 dB here. The three curves are very close to each other, which indicates that the algorithm is not sensitive to σ₀. In the following simulations we set σ₀ = 20.

Figure 3

Convergence of the average sum rate of the hybrid algorithms with σ₀ = 10, 20, 50.

AHT algorithm

In this section, the relative iteration number to achieve certain leakage interference of the AHT and MSP algorithm are displayed in Figure4, to show the property of AHT algorithm. The stopping criterion is P^I ≤ 0.01. The average number of iterations of the MSP and AHT algorithm to solve one problem are 1233 and 583, respectively. To analyze the relative convergence performance, we use the normalized iterations, the ratio of the iterations at the certain point and the total iterations, as the x-axis. And the y-axis represents the total leakage interference P^I. Figure4 implies that to achieve the interference of 0.5 from the initial point, the AHT algorithm requires 7.5% of its total iterations while MSP requires 45%. Compared to the MSP algorithm, the AHT algorithm iterates rapidly to a point with low leakage interference P^I. Actually the main consuming time of AHT algorithm is used to reduce P^I from less than 1 towards 0.01, and the convergence rate becomes much slower when P^I is smaller. The performances here accords with the previous analysis.

Figure 4

Comparison of AHT and MSP: total leakage interference versus normalized iteration number.

Comparison of different IA algorithms

In this section, the performances of different IA algorithms are compared and analyzed, including our proposed MAMA, AHT and hybrid algorithm, as well as the existed MSP and AMA. The number of iterations of each algorithm to solve each problem is set as 3000, in order to perfectly align the interference. To take full advantage of AHT, we set its stopping parameter$\epsilon =0.01P^I({\mathbf{U}}_k^0,{\mathbf{V}}_k^0)$, where${\mathbf{V}}_k^0,{\mathbf{U}}_k^0,k\in \mathcal{K}$ are the initial precoders and decoders.

The average sum rate and relative computing time of the four algorithms (MAMA, Hybrid, MSP and AMA) with respect to different SNRs are depicted in Figures5 and6, respectively. Both our proposed algorithms show good performances, especially the hybrid algorithm. As reflected in Figure5, the curves representing the average sum rate of MSP, MAMA and the hybrid algorithms are quite close to each other. The solutions of the three algorithms are quite different, whereas they get similar sum rate under the same SNR. And all the three algorithms gain about 5 bps/Hz higher sum rate than the AMA under medium and high SNR. In Figure6, both of our proposed two algorithms require much less computing time than the MSP algorithm. Particularly, the hybrid algorithm gain almost as high sum rate as MSP with as little time as the AMA.

Figure 5

Comparison of MAMA, Hybrid, MSP and AMA: average sum rate versus SNR.

Figure 6

Comparison of MAMA, Hybrid, MSP and AMA: average relative computing time versus SNR. Time of AMA is set as standard.

To further compare the convergence performances of these four algorithms as well as AHT algorithm, we plot their convergence curves of sum rate with respect to the running time in Figure7. As AHT algorithm is difficult to converge and eliminate interference, it is shown as the first stage of the hybrid algorithm. Its output point is used to continue iteration in the hybrid algorithm. SNR are set as 5, 15 and 30 dB to represent the scenarios of low, medium and high SNR, respectively. In all three scenarios, both MAMA and the hybrid algorithm achieve similar converged average sum rate as the MSP algorithm, which gain more sum rate than the AMA. To achieve a certain sum rate, MAMA consumes the least time. We also observe an interesting phenomenon, that in the low and medium SNR scenario each of the MAMA, hybrid and MSP algorithm achieves high sum rate during iteration before convergence, after that it reduces and converges to a lower rate. This may be relevant to the value of σ². As shown at the beginning of this section, SNR value is inversely proportional to σ². For the low SNR scenario, σ² is considerably high and σ²I _N takes the main part of F _k in (21). Then, the sum rate mainly increases with large P^S. All the three concerned algorithms are designed to increase P^S in the first few iterations and gradually reduce the weight of this requirement. This may lead to the phenomenon that P^S increases to a peak value and then decreases gradually, which explains the similar phenomenon of sum rate in Figure7.

Figure 7

Convergence of the average sum rate of MAMA, AHT, Hybrid, MSP and AMA with respect to SNR = 5, 15, 30 dB.

Similar as Figure7, for the five compared algorithms (MSP, AHT, Hybrid, MAMA and AMA), we also plot the convergence curves of the leakage interference with respect to the running time in Figure8. As the convergence curves of the same algorithm are quite close for different SNR scenarios, here we only display the scenario that SNR is 30 dB for representation. Among the algorithms, the hybrid algorithm converges fastest to a point with interference less than 10⁻⁴. The MAMA also converges faster than the MSP and AMA.

Figure 8

Convergence of the leakage interference of MAMA, AHT, Hybrid, MSP and AMA with respect to SNR = 30 dB.

Comparing the proposed MAMA and hybrid algorithm from both aspects of computing time and convergence rate in Figures6,7 and8, we should admit that the improvement of the hybrid algorithm over MAMA is limited in the examples that we tested. The main reason of this phenomenon is due to the quite small scale of the test problems, for which lower complexity algorithm does not have much gain. Nevertheless, as the dimension of the problem increases, the hybrid algorithm will save more consuming time and benefit more. Moreover, in similar applications in other fields, there might be large scale similar matrix optimization problems, we believe that the hybrid algorithm will improve MAMA greatly.

Remark 2

We have pointed out in Section “A hybrid algorithm” that the sequence generated by the hybrid algorithm converges to a local optimal solution. Although it is not guaranteed to be global optimal, the obtained local optimal solution performs with high sum rate (almost the same as MSP algorithm and higher than the AMA, as depicted in Figure 5 ) and perfect IA. Such solution satisfies our requirement for an IA solution.

The performances of MAMA and the hybrid algorithm in this section indicates that the Courant penalty function technique is an effective way to combine the leakage interference and the desired signal power together. Through this technique, both algorithms achieve high average sum rate in short time. In the last few iterations when C becomes very large, the algorithm essentially tries to minimize P ^I with the part of − P ^S being nearly ignored. Therefore, MAMA (also the hybrid algorithm) eventually behaves very similar to the original AMA, which can perfectly align the interference.

Conclusion

This article proposed two low complexity algorithms for the desired signal power maximization IA problems of MIMO channels. The IA constraints are added to the objective function and combined with the desired signal power via the Courant penalty function technique. First, a MAMA is proposed following the similar approach of the AMA. Then, a hybrid algorithm is proposed to further reduce complexity. In the hybrid algorithm, the AHT is proposed to iterate rapidly from any initial point to precoders and decoders with low leakage interference. This step provides a good point around the local optimal solution. From this point, MAMA is applied to converge fast to the local optimal solution satisfying our requirement. Analysis shows that, among the compared algorithms the hybrid algorithm has the lowest computational complexity, followed by MAMA, with MSP being the highest. Simulations indicate that both the hybrid algorithm and MAMA achieve similar sum rate as the MSP algorithm with less computing time, and higher sum rate than the AMA.

Appendix

Proof of Theorem 1

As stated in (16), g₀ is the linear combination of x₀ and$\tilde{\mathbf{g}}$. Suppose

$${\mathbf{g}}_0=c_1{\mathbf{x}}_0+c_2\tilde{\mathbf{g}},$$

where c₁ and c₂ are scalars.

First prove that the projected gradient step can be expressed by 2D subspace step. For the projected gradient step x(α) with any α ∈ ℝ, we would like to find corresponding scalars a and b, such that

$$\mathbf{x}\left(\alpha \right)=\frac{{\mathbf{x}}_0-\alpha {\mathbf{g}}_0}{\parallel {\mathbf{x}}_0-\alpha {\mathbf{g}}_0\parallel }=\frac{(1-c_1\alpha ){\mathbf{x}}_0-c_2\alpha \tilde{\mathbf{g}}}{\parallel (1-c_1\alpha ){\mathbf{x}}_0-c_2\alpha \tilde{\mathbf{g}}\parallel }=b{\mathbf{x}}_0+a\tilde{\mathbf{g}}.$$

(22)

Let the coefficients of x₀ and$\tilde{\mathbf{g}}$ remain unchanged, then

$$a=\frac{-c_2\alpha }{\parallel (1-c_1\alpha ){\mathbf{x}}_0-c_2\alpha \tilde{\mathbf{g}}\parallel },b=\frac{1-c_1\alpha }{\parallel (1-c_1\alpha ){\mathbf{x}}_0-c_2\alpha \tilde{\mathbf{g}}\parallel }.$$

Next show that 2D subspace step can also be expressed by the projected gradient step. For any$\mathbf{x}\in S({\mathbf{x}}_0,\tilde{\mathbf{g}})$, we have$\mathbf{x}=b{\mathbf{x}}_0+a\tilde{\mathbf{g}}$. As we wish to find α ∈ ℝ such that x = x(α), then (22) holds. Similar to the proof above, we can induce that

$$\frac{b}{a}=\frac{c_1\alpha -1}{c_2\alpha },\alpha =\frac{a}{c_{1}a-c_{2}b}.$$

Thus we have proved Theorem 1.