UC San Diego

As the demand for high-speed, reliable wireless communication among interconnected devices rises, the need for robust next-generation wireless MIMO systems becomes crucial.This dissertation is motivated by the need to address the challenges inherent in the development of such systems, specifically focusing on two key aspects: (i) Robust block-sparse mmWave channel modeling and (ii) Robust detection in the presence of few-bit MIMO systems. Central to this thesis is the integration of model-based methods with deep neural network (DNN)-aided approaches, leveraging the synergy between these two paradigms to enhance system performance and mitigate the impact of model inaccuracies and mismatches.

The first part of this dissertation focuses on the spatial modeling of mmWave channels, to capture the heterogeneous scattering behavior, through block-sparse signal recovery.Despite the promise of block-sparse signal processing for channel modeling in the angular domain, a key challenge is block-patterned estimation without knowledge of block sizes and boundaries. This work propose a novel total variation sparse Bayesian learning (TV-SBL) method for block-sparse signal recovery under unknown block patterns. Unlike conventional approaches that employ block-promoting regularization on signal components, this method introduces two classes of hyperparameter regularizers for the SBL cost function inspired by total variation (TV) denoising. The first class relies on a conventional TV difference unit, allowing iterative SBL inference through convex optimization, thus facilitating the use of various numerical solvers. The second class integrates a region-aware TV penalty to penalize signal and zero blocks differently, thereby enhancing performance. An alternating optimization algorithm based on expectation-maximization is derived for computationally efficient parallel updates for both regularizer classes. Going beyond model-based methods, this work also presents a basis for extension to DNN-aided block-sparse signal recovery for 1-D and 2-D signals.

The second part of this dissertation focuses on designing detection algorithms for signal recovery in few-bit MIMO systems, beginning with a detailed analysis of one-bit MIMO systems. This begins by analyzing the smoothness and convexity of the one-bit likelihood function, based on the Gaussian CDF, for signal recovery. This culminates in an improved gradient descent (GD) algorithm for one-bit MIMO, and ensuing convergence analysis. The accelerated GD method is applied to one-bit MIMO recovery, further improving convergence. The analysis is extended to an effective surrogate function for the Gaussian CDF, i.e., the logistic regression (LR), explaining the enhanced performance when utilized as a surrogate likelihood. Constrained optimization, incorporating detection from a finite M-QAM constellation, is addressed by the introduction of a \textit{learnable} Gaussian denoiser to project detected symbols onto the M-QAM subspace.

Another class of DNN-aided regularizers is proposed for one-bit MIMO, utilizing a regularized gradient descent update. A novel constellation-aware loss function is incorporated to tailor the DNN loss function to M-QAM symbol recovery. The key utility of a generalized DNN-aided GD update is for detection in mmWave channels, where there is a higher contrast in per-user channel powers, presenting a challenge for joint multi-user detection. Leveraging a general parametric DNN structure enables the development of a novel hierarchical detection training algorithm, ensuring network design for equitable detection in mmWave channels, where users with higher channel powers experience improved recovery performance.

The final part of this dissertation extends the research to two-bit MIMO detection. In few-bit MIMO systems like the two-bit MIMO receiver, the quantization noise exhibits distinct characteristics positioned between the fully saturated one-bit scenario and the independently additive noise observed in higher resolutions. However, limited research has focused on accurately characterizing this unique quantization noise profile. These properties of quantization noise, along with the constraints of optimization for MIMO signal recovery, make DNNs ideally suited for the signal recovery. The DNN-augmented receiver algorithm developed attempts to learn this noise behavior to dequantize the signal, without the need for explicit analytical characterization, thereby enhancing signal recovery in few-bit MIMO systems.