 Main
Modern Problems in Mathematical Signal Processing: Quantized Compressed Sensing and Randomized Neural Networks
 Nelson, Aaron Andrew
 Advisor(s): Saab, Rayan
Abstract
We study two problems from mathematical signal processing. First, we consider problem of approximately recovering signals on a smooth, compact manifold from onebit linear measurements drawn from either a Gaussian ensemble, partial circulant ensemble, or bounded orthonormal ensemble and quantized using \(\Sigma\Delta\) or distributed noiseshaping schemes. We construct a convex optimization algorithm for signal recovery that, given a Geometric MultiResolution Analysis approximation of the manifold, guarantees signal recovery with high probability. We prove an upper bound on the recovery error which outperforms prior works that use memoryless scalar quantization, requires a simpler analysis, and extends the class of measurements beyond Gaussians.
Second, we consider the problem of approximation continuous functions on compact domains using neural networks. The learning speed of feedforward neural networks is notoriously slow and has presented a bottleneck in deep learning applications for several decades. For instance, gradientbased learning algorithms, which are used extensively to train neural networks, tend to work slowly when all of the network parameters must be iteratively tuned. To counter this, both researchers and practitioners have tried introducing randomness to reduce the learning requirement. Based on the original construction of B.~Igelnik and Y.H.~Pao, single layer neuralnetworks with random inputtohidden layer weights and biases have seen success in practice, but the necessary theoretical justification is lacking. We begin to fill this theoretical gap by providing a (corrected) rigorous proof that the Igelnik and Pao construction is a universal approximator for continuous functions on compact domains, with \(\varepsilon\)error convergence rate inversely proportional to the number of network nodes; we then extend this result to the nonasymptotic setting using a concentration inequality for MonteCarlo integral approximations. We further adapt this randomized neural network architecture to approximate functions on smooth, compact submanifolds of Euclidean space, providing theoretical guarantees in both the asymptotic and nonasymptotic cases.
Main Content
Enter the password to open this PDF file:













