A High-resolution DOA Estimation Method with a Family of Nonconvex Penalties

The low-rank matrix reconstruction (LRMR) approach is widely used in direction-of-arrival (DOA) estimation. As the rank norm penalty in an LRMR is NP-hard to compute, the nuclear norm (or the trace norm for a positive semidefinite (PSD) matrix) has been often employed as a convex relaxation of the rank norm. However, solving a nuclear norm convex problem may lead to a suboptimal solution of the original rank norm problem. In this paper, we propose to apply a family of nonconvex penalties on the singular values of the covariance matrix as the sparsity metrics to approximate the rank norm. In particular, we formulate a nonconvex minimization problem and solve it by using a locally convergent iterative reweighted strategy in order to enhance the sparsity and resolution. The problem in each iteration is convex and hence can be solved by using the optimization toolbox. Convergence analysis shows that the new method is able to obtain a suboptimal solution. The connection between the proposed method and the sparse signal reconstruction (SSR) is explored showing that our method can be regarded as a sparsity-based method with the number of sampling grids approaching infinity. Two feasible implementation algorithms that are based on solving a duality problem and deducing a closed-form solution of the simplified problem are also provided for the convex problem at each iteration to expedite the convergence. Extensive simulation studies are conducted to show the superiority of the proposed methods.